Comments (5)
CC @lopopolo that was the issue at hand, the suggestion should accept 0xC2 0xA9 as valid since it is valid itself, but returns garbage suggestions instead. It's still present in 0.12.2 but will only be a verbose message that will be handled in the next release.
from cargo-spellcheck.
Oh awesome. That's in the generated headers of the Unicode files. Great catch @drahnr and thanks for debugging!
from cargo-spellcheck.
@lopopolo I realized you have custom dict, with a -
- removing the single char items from the list resolves the issue. It seems that trips the parser and makes it's way into the lut inside hunspell and then surfaces with some byte sequences.
from cargo-spellcheck.
I just tried this workaround and that seemed to work! I get no warning messages in cargo-spellcheck 0.12.1
.
from cargo-spellcheck.
The core issue is due to the fact that Hunspell
uses the encoding used in the affix file, for the dictionaries as well. For en_us.aff
(both builtin and Fedora 36) this was latin-1
encoding rather than utf-8
. Hunspell
then treats all inputs of encoding equiv to the affix file, implicitly by the used prefix tree.
Solution would be to use i.e. encoding_rs to re-encode the dictionaries to UTF-8 and only afterwards feed them to Hunspell
or reject all encodings set besides utf-8
.
from cargo-spellcheck.
Related Issues (20)
- Word "C++" is tokenized incorrectly and can not be whitelisted HOT 3
- Handle mdbook `book.toml`
- reflow sub command transposes `//` and leading space HOT 7
- Panic hunspell returned non-utf8 sequence HOT 7
- Broader CI/Docker usage support HOT 6
- How to ignore files HOT 7
- Build failure due to clang HOT 6
- How to add words? HOT 2
- Is there way to skip over doc links in comments? HOT 2
- Interactive fixing seems buggy HOT 3
- Support cargo-binstall HOT 9
- Reduce release artefact download size HOT 2
- Spellcheck doesn't list files when edition is inherited HOT 3
- Adding a footnote gives false positives HOT 2
- Is there a way to ignore commented out code? HOT 2
- Help identifying panic: `assertion failed: plain_range.start <= plain_range.end` HOT 15
- Non determinism with hunspell backend
- Installing the crate in a macbook machine HOT 8
- Don't flag valid identifiers as misspelled in doc comments HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cargo-spellcheck.