Track and album names displayed with replacement character. How to resolve?

[Yukster]

(24-01-2024 10:02)simbun Wrote:  It looks like simply saving the tracks in Mp3tag might remove those NUL characters based on a similar problem discussed in the Mp3tag forums, but without a sample I don't know how to identify them.

I tried to install Media Centre 17.0.189 from their archive but it wouldn't install into a sandbox.

I guess you could truncate the track with SOX and upload it?

Code:

"C:\Program Files (x86)\sox-14-4-2\sox.exe" "input.flac" "output.flac" trim 0 2

NOTE: Don't use the output from fsutil, use the original track as input to sox.


EDIT: I guess it might be worth trying to update in Jriver first, although I have no idea if you can perform this operation in bulk like you could with Mp3tag.

Many thanks for your help so far

I'll try to open and save in mp3tag and report back.

I haven't had jriver Mesia Cebtre v 17 installed for some years. I'm running v.23 currently.

[simbun]

(24-01-2024 13:10)Yukster Wrote:  I haven't had jriver Mesia Cebtre v 17 installed for some years. I'm running v.23 currently.

I'm just trying to create a file that exhibits the same behaviour to see if I can identify the NUL in Mp3tag, so that rather than re-save your entire collection you can target only those affected.

If you load a problem album into Mp3tag then paste the following code in the Filter at the bottom of the screen:

Code:

"#ifgreater(#strstr(%title%,#char(00)),0,yes,no)" IS yes

NOTE: I've had to replace $ with # in the above code for the forum to accept it, so switch them back before pasting into Mp3tag.

Do the problem tracks remain in the grid or do they disappear? If they remain then it's found the NUL in the title field (you could double check by loading a recent album and verifying that the tracks are removed). This means that you could load your entire collection (depending on how big it is!) into Mp3tag, apply the filter (which would reduce the selection to only those affected), then perform the re-save.

Up until now you've been dealing with FLAC files, if you have any MP3 files and they exhibit the same issue you'll need to configure Mp3tag so it retains the correct ID3 tag version (2.3 vs 2.4), but let's address that if you have any.

As always, when performing any big change make sure you have a backup.

[simoncn]

(24-01-2024 14:20)simbun Wrote:  Up until now you've been dealing with FLAC files, if you have any MP3 files and they exhibit the same issue you'll need to configure Mp3tag so it retains the correct ID3 tag version (2.3 vs 2.4), but let's address that if you have any.

This issue will probably not affect MP3 files because ID3v2 text strings are NUL-terminated.

[simbun]

(24-01-2024 16:58)simoncn Wrote:  

(24-01-2024 14:20)simbun Wrote:  Up until now you've been dealing with FLAC files, if you have any MP3 files and they exhibit the same issue you'll need to configure Mp3tag so it retains the correct ID3 tag version (2.3 vs 2.4), but let's address that if you have any.

This issue will probably not affect MP3 files because ID3v2 text strings are NUL-terminated.

I didn't think it could happen (although whilst I did realise 2.4 is NUL separated I didn't realise they were NUL teriminated), but wanted to stay well clear as the MP3 waters are quite murky

[paultaylor]

(24-01-2024 17:49)simbun Wrote:  (although whilst I did realise 2.4 is NUL separated I didn't realise they were NUL teriminated)

Hi, you are correct they are not null-terminated just null separated (although there are some taggers that incorrectly null-terminate all strings)

e.g from spec for text information frames and the variant of TXXX frames

Quote:4.2. Text information frames

The text information frames are often the most important frames,
containing information like artist, album and more. There may only be
one text information frame of its kind in an tag. All text
information frames supports multiple strings, stored as a null
separated list, where null is reperesented by the termination code
for the charater encoding. All text frame identifiers begin with "T".
Only text frame identifiers begin with "T", with the exception of the
"TXXX" frame. All the text information frames have the following
format:

<Header for 'Text information frame', ID: "T000" - "TZZZ",
excluding "TXXX" described in 4.2.6.>
Text encoding $xx
Information <text string(s) according to encoding>



4.2.6. User defined text information frame

This frame is intended for one-string text information concerning the
audio file in a similar way to the other "T"-frames. The frame body
consists of a description of the string, represented as a terminated
string, followed by the actual string. There may be more than one
"TXXX" frame in each tag, but only one with the same description.

<Header for 'User defined text information frame', ID: "TXXX">
Text encoding $xx
Description <text string according to encoding> $00 (00)
Value <text string according to encoding>

Look at the definition of the Description field, if <text string according to encoding> was meant to include terminating byte(s) you would have it twice because it has been explicity defined in the definition.

[simoncn]

Text fields are NUL-terminated in ID3v2.4. Here is the relevant section:

Code:

4.   ID3v2 frame overview

   .................................

   Frames that allow different types of text encoding contains a text
   encoding description byte. Possible encodings:

     $00   ISO-8859-1 [ISO-8859-1]. Terminated with $00.
     $01   UTF-16 [UTF-16] encoded Unicode [UNICODE] with BOM. All
           strings in the same frame SHALL have the same byteorder.
           Terminated with $00 00.
     $02   UTF-16BE [UTF-16] encoded Unicode [UNICODE] without BOM.
           Terminated with $00 00.
     $03   UTF-8 [UTF-8] encoded Unicode [UNICODE]. Terminated with $00.

[paultaylor]

(25-01-2024 12:45)simoncn Wrote:  Text fields are NUL-terminated in ID3v2.4. Here is the relevant section:

If that were true according to the spec a custom frame (TXXX) body using ISO-8859 with description CUSTOM and value Fred would be encoded as:

Code:

\0CUSTOM\0\0Fred\0

but it is not most do this

Code:

\0CUSTOM\0Fred

some incorrectly do this

Code:

\0CUSTOM\0Fred\0

but that makes no sense because both Description and Value are defined as being text string according to encoding and additionally there is null terminator (based on encoding) defined between the two.

The spec is confusing but Im sure it is just saying that for a particular encoding this is the null termination to use when you need a null terminator

so if you were using UTF8 in the above example you should have

Code:

\3CUSTOM\0Fred

but if using UTF16BE would be (not showing multi bytes of CUSTOM or Fred here)

Code:

\2CUSTOM\0\0Fred

If CUSTOM contained two values Fred and Jim then you would have

Code:

\3CUSTOM\0Fred\0Jim

or

Code:

\2CUSTOM\0\0Fred\0\0Jim

[simoncn]

I checked a selection of MP3 files sent to me by users (so not all ripped by the same ripper). They included both ID3v2.3 and ID3v2.4 and different encodings for text strings. Most files had text strings with a NUL at the end but some did not. From this it seems the NUL terminator is optional and the server must be prepared to accept strings either with or without a NUL terminator, as MinimServer does.

[Yukster]

(24-01-2024 14:20)simbun Wrote:  If you load a problem album into Mp3tag then paste the following code in the Filter at the bottom of the screen:

Code:

"#ifgreater(#strstr(%title%,#char(00)),0,yes,no)" IS yes

NOTE: I've had to replace $ with # in the above code for the forum to accept it, so switch them back before pasting into Mp3tag.

Do the problem tracks remain in the grid or do they disappear? If they remain then it's found the NUL in the title field (you could double check by loading a recent album and verifying that the tracks are removed). This means that you could load your entire collection (depending on how big it is!) into Mp3tag, apply the filter (which would reduce the selection to only those affected), then perform the re-save.

Up until now you've been dealing with FLAC files, if you have any MP3 files and they exhibit the same issue you'll need to configure Mp3tag so it retains the correct ID3 tag version (2.3 vs 2.4), but let's address that if you have any.

As always, when performing any big change make sure you have a backup.

I pasted the code into mp3tag filter the tracks disappeared. I saved all 4 tracks individually and the unwanted characters are no longer

[simbun]

(25-01-2024 18:02)Yukster Wrote:  I pasted the code into mp3tag filter the tracks disappeared. I saved all 4 tracks individually and the unwanted characters are no longer

It's good news that saving the files in Mp3tag removed the problem characters.

I think you're telling me though that when you pasted the following code (replacing # with $) into the Filter in Mp3tag, all the bad tracks disappeared.

Code:

"#ifgreater(#strstr(%title%,#char(00)),0,yes,no)" IS yes

Which isn't great - although there's no checking of the syntax so it could be that it was pasted in incorrectly.

In order to be able to help you in identifying the problem tracks, I'd need you to create a short version of one of the tracks so I can test it this end.

If you download and install SOX, then run the following command against one of your problem files (one that hasn't been fixed or processed with fsutil) and attach it to a post.

Code:

"C:\Program Files (x86)\sox-14-4-2\sox.exe" "file name.flac" "output.flac" trim 0 2

Alternatively you'd need to load your entire collection into Mp3tag and save them all.