non-UTF8 characters fail to parse on Windows about data.table

[joshhwuu]

Seems like the code doesn't compile properly wherever I used the Japanese characters in the tests, should be able to get around this with strrep, however I'm not confident that the tests will pass even if the code compiles, similar to the issues we had with AppVeyor encoding in #5096. In that case, should I try to encapsulate the tests for the Windows platform and manually change the system locale to pass the tests? Although testing that may be a bit tricker -- I don't use Windows to build usually.

I was able to reproduce the failure on my Linux system by setting locales to match the one in the GHA, and refactoring the tests to not explicitly use the Japanese characters + wrapping the set of tests in Sys.setlocale() then restoring allow the tests to pass. I saw some tests before that also did it this way, just need a way to test it with the Windows GHA, how could I go about doing this? @MichaelChirico Thanks!

[joshhwuu]

Also wanted to add that while I was working on this, test(2194.7) was also failing with my locale settings:

Tue May 21 19:36:42 2024  endian==little, sizeof(long double)==16, longdouble.digits==64, sizeof(pointer)==8, TZ==unset, Sys.timezone()=='America/Vancouver', Sys.getlocale()=='C', l10n_info()=='MBCS=FALSE; UTF-8=FALSE; Latin-1=FALSE; codeset=ANSI_X3.4-1968', getDTthreads()=='OpenMP version (_OPENMP)==201511; omp_get_num_procs()==16; R_DATATABLE_NUM_PROCS_PERCENT==unset (default 50); R_DATATABLE_NUM_THREADS==unset; R_DATATABLE_THROTTLE==unset (default 1024); omp_get_thread_limit()==2147483647; omp_get_max_threads()==16; OMP_THREAD_LIMIT==unset; OMP_NUM_THREADS==unset; RestoreAfterFork==true; data.table is using 8 threads with throttle==1024. See ?setDTthreads.', zlibVersion()==1.2.11 ZLIB_VERSION==1.2.11
  Error in stopf("%d error(s) out of %d. Search %s for test number(s) %s. Duration: %s.",  : 
    1 error(s) out of 11223. Search tests/tests.Rraw for test number(s) 2194.7. Duration: 15.2s elapsed (17.3s cpu).
  Calls: test.data.table -> stopf -> raise_condition -> signal
  In addition: Warning message:
  In readLines(testDir("issue_563_fread.txt")) :
    invalid input found on input connection '/home/joshhwuu/work/data.table/data.table.Rcheck/data.table/tests/issue_563_fread.txt'

Seems to be the same test described in #5484, so just wanted to note that it is reproducible