Is the script used to generate the indices available? about zhongwen

[metzkorn]

The way the indices are calculated seems kind of bizarre (at least from what my text editors are showing me). There seems to be an extra character in every line at first, but then that breaks down.

[metzkorn]

Most lines seem to count as an extra character, but not every line. Going to try using a different editor. This might be affecting the actual app too? At least some characters towards the bottom of cedict_ts.u8 aren't detectable. For instance, 𬭳 isn't detectable as well as the other 10 final entries.

[metzkorn]

Upon further investigation, this seems to be an error with the encoding scheme used. The one extra character per line comes from \r\n. But other extra characters appear because characters like 𪨊 get rendered as ��.

[metzkorn]

I have written a python script that updates the indices. I'll consider refactoring it so it's easy to use for others, but everything anyone would need to update indices in updateidx.py in my fork of the project.