研究 is broken down into 研 and 究 about kuromoji.js

[takuyaa]

@katspaugh I released fixed version of 0.1.2! You could check it on demo site: https://takuyaa.github.io/kuromoji.js/demo/tokenize.html
Please try it.
FYI @iapyeh @jrorsini

[katspaugh]

I confirm the issue is fixed in my app, too. Many thanks! 👍

[iapyeh]

Excuse me, then, how to change the behavior for "研究" to be parsed as a single token? (Disable the Viterbi algorithm?)

[jrorsini]

Maybe if you try this it should work:
let path = tokenizer.tokenizeForSentence(txtToGloss);

[katspaugh]

@jrorsini nope, the same.
I've just tried it in my project https://kuromoji.fluentcards.com/ – changed tokentize to tokenizeForSentence but it's still breaking 研究 mid-word.

[jrorsini]

@katspaugh, Maybe you should add this file,
https://github.com/takuyaa/kuromoji.js/blob/master/build/kuromoji.js
That's the one that contains the prototype method.

[katspaugh]

@jrorsini I mean, tokenizeForSentence worked, it's defined. It just behaves the same way as tokenize with regard to 研究.

[takuyaa]

This seems to be a bug.
#25 will fix this issue.

[katspaugh]

Cool! I'll test it with my app and let you know if the fix helped with this issue.