About japanesse chars about forceutf8

[tegansnyder]

Do you know what charset your are working with? I just tried this extension on some Shift-JS charset characters and it failed to encode them in UTF-8. I think this extension is not detecting Shift-JS correctly.

Try doing it manually with iconv.

$string_to_encode = '質量';
$encoded_str = iconv('shift-jis','utf-8'.'//TRANSLIT', $string_to_encode);
echo $encoded_str;

[neitanod]

The main function of this package is toUTF8(), which encodes Latin1 to UTF8 but detects characters that are already UTF-8 encoded and keeps them as they are, avoiding the usual problem of double encoding them.

FixUTF8 is an auxiliary function that fix Latin1 characters on double-encoded UTF8 strings, converting, for instance, "República" back to "República", but losing all non-Latin1 characters.

In other words, do not try to use it with japanese charactes (or any character outside the Latin1 set). It will break them.
Do not use it in production either. It's a hacky and slow function designed to be used in manual, human assisted batch processes by people that only use characters in the Latin1 set.

[didix16]

Oh okay thanks for the explanation! Anyway I wrote a function to handle the problem with it. I post it here if anyone want to use it or improve it. $nickname represents an unicode string wich means it has X pairs of bytes. In my case I'm working with BigEndian format. This function sucessfully returns any unicode string by getting its hexadecimal value.

function fixNick($nickname){
        $fixNick ="";
        $i=0;
        $strlen = strlen($nickname)-2;
        $unicode = (ord($nickname[$i+1]) << 8)+ord($nickname[$i]);
        while($i<$strlen and  $unicode>0){
            $i=$i+2;
            $fixNick.=mb_convert_encoding('&#'.$unicode, 'UTF-8', 'HTML-ENTITIES');
            $unicode = (ord($nickname[$i+1]) << 8)+ord($nickname[$i]);
        }

        return $fixNick;
    }