UTF-8 with BOM for ID3 and Vorbis Tags

[giorgos]

Is there an option/setting for UTF-8 with Byte Order Mark (BOM) ID3 and Vorbis tags encoding, rather than UTF-8 without BOM?

My need for this comes in connection to the Fiio M5 DAP which does not (correctly) support UTF-8 without BOM tags when the tags contain non-standard characters, such as Cyrillic, Greek etc. ones.

Thank you,
George

[Spoon]

Re: UTF-8 with BOM for ID3 and Vorbis Tags

UTF-8 does not have BOM as it is stored as encoded 8 bit, not 16 bit (which is UTF-16).

[giorgos]

Re: UTF-8 with BOM for ID3 and Vorbis Tags

UTF-8 does not have BOM as it is stored as encoded 8 bit, not 16 bit (which is UTF-16).

The UTF-8 standard neither requires nor recommends BOM, but it does allow for it. And in my use case it is needed to be present. So, this is why I am asking if there is such an option, rather that saying that this is some kind of error of this particular software.

[giorgos]

Re: UTF-8 with BOM for ID3 and Vorbis Tags

The UTF-8 standard neither requires nor recommends BOM, but it does allow for it. And in my use case it is needed to be present. So, this is why I am asking if there is such an option, rather that saying that this is some kind of error of this particular software.

PS: From the Unicode standard:

It is the BOM use as a UTF-8 signature that is needed in some cases, like in the case that I need it for.

[Spoon]

Re: UTF-8 with BOM for ID3 and Vorbis Tags

https://en.wikipedia.org/wiki/Byte_order_mark

This is the first 2 characters of a text file to indicate what its encoding is. ID Tags would not have a 2 character header as the format is indicated either by the standard (vorbis is utf8) or by a tag block header (id3v2 for example, it is possible to have ansi, utf8, unicode 16).

[mville]

Re: UTF-8 with BOM for ID3 and Vorbis Tags

My need for this comes in connection to the Fiio M5 DAP which does not (correctly) support UTF-8 without BOM tags when the tags contain non-standard characters, such as Cyrillic, Greek etc. ones.

Have you contacted Fiio about this?

[giorgos]

Re: UTF-8 with BOM for ID3 and Vorbis Tags

https://en.wikipedia.org/wiki/Byte_order_mark

This is the first 2 characters of a text file to indicate what its encoding is. ID Tags would not have a 2 character header as the format is indicated either by the standard (vorbis is utf8) or by a tag block header (id3v2 for example, it is possible to have ansi, utf8, unicode 16).

Could you please point me to the section of the Unicode standard that identifies that the BOM is applicable only for text files, not for Unicode text streams in general? I have tried to locate that myself following your above response, but I couldn't. It could help me (and many other users) to bring into Fiio's attention such a point in the Unicode standard if it exists.

Thank you.

[giorgos]

Re: UTF-8 with BOM for ID3 and Vorbis Tags

Have you contacted Fiio about this?

I wouldn't be here looking for a workaround if Fiio would acknowledge and fix the issue in its source.

[Spoon]

Re: UTF-8 with BOM for ID3 and Vorbis Tags

Could you please point me to the section of the Unicode standard that identifies that the BOM is applicable only for text files, not for Unicode text streams in general? I have tried to locate that myself following your above response, but I couldn't. It could help me (and many other users) to bring into Fiio's attention such a point in the Unicode standard if it exists.

Thank you.

So Fiio state that every tag, should have a BOM at the start of the tag? even for UTF-8 which the wiki link above stated:

The Unicode Standard permits the BOM in UTF-8,[3] but does not require or recommend its use.[4] Byte order has no meaning in UTF-8

[Dat Ei]

Re: UTF-8 with BOM for ID3 and Vorbis Tags

I wouldn't be here looking for a workaround if Fiio would acknowledge and fix the issue in its source.

That doesn't surprise me, being a FiiO M11 owner myself, at all. FiiO has phantastic hardware, but the software is poor. Have you ever thought about a change of the player software? Since I've changed from FiiO's music player to "USB Audio Player Pro" I'm happy again.


Dat Ei

[giorgos]

Re: UTF-8 with BOM for ID3 and Vorbis Tags

That doesn't surprise me, being a FiiO M11 owner myself, at all. FiiO has phantastic hardware, but the software is poor. Have you ever thought about a change of the player software? Since I've changed from FiiO's music player to "USB Audio Player Pro" I'm happy again.


Dat Ei

Thank you for taking the time to comment on this and suggest a possible workaround . Unfortunately the M5 is a single/fixed SW/APP device.

[giorgos]

Re: UTF-8 with BOM for ID3 and Vorbis Tags

So Fiio state that every tag, should have a BOM at the start of the tag? even for UTF-8 which the wiki link above stated:

The Unicode Standard permits the BOM in UTF-8,[3] but does not require or recommend its use.[4] Byte order has no meaning in UTF-8

Unless a SW or HW implementation claims full support of any and all options allowed by all applicable standard(s) - which they don't - when/if (big IF as I have no evidence to the contrary myself) the applicable standards allow for either, should and shouldn't are actually equally wrong. And, when one is looking for a workaround, what is strictly right or wrong by the standard9s) isn't the essence of everything as that would be the case when pursuing a fix, not when looking for a workaround.

Be that as it may, I really thank you for your time and your input. I understand that the implied answer to my original question is NO.

[Spoon]

Re: UTF-8 with BOM for ID3 and Vorbis Tags

Yes, we would not make an option just for one device which does not follow the standard, if we did, in the 15 years dBpoweramp has existed there would be 1000's of options.

The only time we would work around another programs bug is if it is used by many people.