Panic hunspell returned non-utf8 sequence about cargo-spellcheck

[drahnr]

Since version 0.12.0 .gitgnore files are considered, where you could put the offending file as a workaround.

The referenced code hints that hunspell (the C library itself) returns invalid results. I'd buckle up - it might be a while before that is fixed. Could you upload the offending file triggering the behavior?

[lopopolo]

The branch here reliably panics for me with 0.12.0: https://github.com/artichoke/focaccia/tree/lopopolo/spellcheck

[drahnr]

Could you provide your native hunspell lib version?

[lopopolo]

I'm not sure how to do that. I'm on a mac and I don't think I've installed hunspell.

I did a cargo install cargo-spellcheck today. I don't think I passed --locked.

[drahnr]

That's sufficient info, thanks!

[drahnr]

It doesn't crash for me with 0.12.0, I guess that's because I am on Fedora/Linux.

Can reproduce on Fedora with 0.12.0.

[drahnr]

So I released 0.3.1 of hunspell-rs, this will not panic anymore but print some context. Note that this uses eprintln! for printing, 0.4.0 will return results in a few API calls that will be released soon and should mediate the issue itself.

Another possible fix is not using the bundled hunspell lib but the host one, 1.7.1 contains a few memory fixes that might contain a fix, untested.

Note that this smells some off by one error or memory corruption inside hunspell. There have been a few instances like this where adding locks will make the issue disappear. Tempted to go down the rabbit hole of compiling it to wasm and transpiling reverse as part of build.rs to avoid any kind of memory madness in the spirit of firefox.