Missing files --> Error: couldn't access file (with special characters) [SOLVED]

Missing files --> Error: couldn't access file (with special characters) [SOLVED] - Printable Version

+- MinimServer Forum (https://forum.minimserver.com)
+-- Forum: MinimServer (/forumdisplay.php?fid=1)
+--- Forum: Support (/forumdisplay.php?fid=4)
+--- Thread: Missing files --> Error: couldn't access file (with special characters) [SOLVED] (/showthread.php?tid=2179)

Missing files --> Error: couldn't access file (with special characters) [SOLVED] - blinkeye - 15-02-2015 12:27

I'm using mp3tag with the following tag-to-filename conversion settings:

Code:

%artist% - %composer% - %album% - $num(%track%,2) - %title%

This results in filenames with special chararacters which cannot be parsed by minimserver:

Code:

Error: couldn't access file Chicago Symphony Orchestra - Sergey Rachmaninov - Rachmaninov Rhapsody on a Theme of Paganini - 27 - Noches en los jardines de Espa�a I. En el Generalife.flac

After logging in into the QNAP NAS I think to have the proper locale settings:

Code:

# locale -a

C

en_US.utf8

POSIX

and the filenames (even with special chars) are properly displayed:

Code:

# ls -l

-rw-r--r--    1 1004     1015     100574204 Feb 15 11:57 Chicago Symphony Orchestra - Sergey Rachmaninov - Rachmaninov Rhapsody on a Theme of Paganini - 27 - Noches en los jardines de España I. En el Generalife.flac

Is there anything I can do besides renaming all the files?

RE: Missing files --> Error: couldn't access file (with special characters) - blinkeye - 15-02-2015 12:29

This also happens with German Umlaute characters:

Code:

Error: couldn't access file  Chicago Symphony Orchestra - Richard Wagner - Reiner Conducts Wagner - 02 - Die Meistersinger von N�rnberg, opera, WWV 96 Act III Prelude.flac

RE: Missing files --> Error: couldn't access file (with special characters) - simoncn - 15-02-2015 15:59

(15-02-2015 12:29)blinkeye Wrote: This also happens with German Umlaute characters:

Code:

Error: couldn't access file Chicago Symphony Orchestra - Richard Wagner - Reiner Conducts Wagner - 02 - Die Meistersinger von N�rnberg, opera, WWV 96 Act III Prelude.flac

What output do you get from "locale" (not "locale -a")?

What is the platform default charset displayed in the MinimServer log?

Do you have 0.8.3 installed?

RE: Missing files --> Error: couldn't access file (with special characters) - blinkeye - 15-02-2015 17:36

(15-02-2015 15:59)simoncn Wrote: What output do you get from "locale" (not "locale -a")?

Code:

# locale

LANG=en_US.UTF-8

LC_CTYPE="en_US.UTF-8"

LC_NUMERIC="en_US.UTF-8"

LC_TIME="en_US.UTF-8"

LC_COLLATE="en_US.UTF-8"

LC_MONETARY="en_US.UTF-8"

LC_MESSAGES="en_US.UTF-8"

LC_PAPER="en_US.UTF-8"

LC_NAME="en_US.UTF-8"

LC_ADDRESS="en_US.UTF-8"

LC_TELEPHONE="en_US.UTF-8"

LC_MEASUREMENT="en_US.UTF-8"

LC_IDENTIFICATION="en_US.UTF-8"

LC_ALL=en_US.UTF-8

(15-02-2015 15:59)simoncn Wrote: What is the platform default charset displayed in the MinimServer log?

Do you have 0.8.3 installed?

Code:

# head -n 5 /share/Public/minimserver.log

MinimServer 0.8.3 update 57, Copyright (c) 2012-2014 Simon Nash. All rights reserved.

MinimStreamer 0.5.1, Copyright (c) 2012-2015 Simon Nash. All rights reserved.

Java(TM) SE Embedded Runtime Environment (build 1.8.0_06-b23)

Java HotSpot(TM) Embedded Client VM (build 25.6-b23, mixed mode)

Platform default charset is UTF-8

Language setting is 'auto'

starting MinimServer

RE: Missing files --> Error: couldn't access file (with special characters) - simoncn - 15-02-2015 18:28

It looks like MinimServer is working correctly. This would mean that the problematic filenames don't contain UTF-8 characters but something else.

I presume you created these files using Mp3tag on Windows. If your Windows charset is Windows-1252, you might have filenames containing Windows-1252 characters instead of UTF-8 characters. How did you copy these files from Windows to the QNAP? If you browse these files using QNAP File Station, do the filenames look correct or incorrect?

RE: Missing files --> Error: couldn't access file (with special characters) - blinkeye - 15-02-2015 18:58

You're right, it's a filename encoding issue:

Code:

$ isutf8 Chicago\ Symphony\ Orchestra\ -\ Sergey\ Rachmaninov\ -\ Rachmaninov\ Rhapsody\ on\ a\ Theme\ of\ Paganini\ -\ 27\ -\ Noches\ en\ los\ jardines\ de\ España\ I.\ En\ el\ Generalife.flac 

Chicago Symphony Orchestra - Sergey Rachmaninov - Rachmaninov Rhapsody on a Theme of Paganini - 27 - Noches en los jardines de España I. En el Generalife.flac: line 2, char 1, byte offset 1: invalid UTF-8 code

I'm using mp3tag on Linux with Wine, but the file encoding seems to be latin1. What I now did was using convmv to convert files from iso-8859-1 to utf8:

Code:

convmv --notest -f iso-8859-1 -t utf8 file-with-encoding-issues.flac

As always, thank you Simon for the prompt and valid replies!

RE: Missing files --> Error: couldn't access file (with special characters) [SOLVED] - MikeL - 02-06-2016 19:25

Funny how problems multiply! I downloaded a couple of new flacs and could not understand why they were not indexing. Investigating led here: then I discovered that there were more than a hundred tracks failing that I had never noticed were missing. I think that the problem in my case leads back to unpredictable characters coming from cddb which I use to make the filenames.

Anyway, I wrote a little script to parse the log file and make the fixes. Here it is if anyone else wants it. It basically wraps convmv and uchardet (widely available).

Code:

#!/bin/bash

# uncomment or define on command line to make it happen

#DOIT="--notest"

LOG=/mnt/terabyte/music/utils/minimserver/data/minimserver.log

IFS=$'\n'

# convert all file names to UTF8 that appear as an error in log

#   File name as it appears in log is altered by error char

#     so make a stub /a/path/to/nn-* then get the real name

#     by eval'ing an ls on it.

#     Assumes that all files start with /nn-

for  i in $(awk '/access file/{print $5 }' $LOG);do

    STUMP=$(echo $i|sed 's/\(\/[0-9]\+[-_]\).*/\1*/')

    FILE=$(eval "ls $STUMP")

    FROM=$(echo $FILE|uchardet)

    echo $FROM

    if [[ -f $FILE ]]; then 

        convmv -f $FROM -t UTF8 $FILE $DOIT 

    fi

done

Obviously the LOG path needs to match the target system. Either uncomment #DOIT in the script or provide it on the command line change talk into action

Code:

chmod 777 ./fixnames.sh

DOIT=--notest ./fixname.sh