Detection accuracy about langid.py

[vamseekm]

Sorry my mistake, it was encoding issue. Thanks for the great tool.

[saffsd]

No worries. Indeed, my own check verifies that zh is correctly detected

**功夫是一门博大精深的武学艺术 , **功夫app , 介绍**功夫的分类、特点、器材、门派等与**功夫有关的内容！让广大读者能够更完整的了解**功夫的精华！
('zh', -1414.5709274662972)

Could you provide me some detail on how you used the tool? It would be good if I can detect potential encoding issues beforehand and try to address them, to make the tool as simple as possible for the end user.

[vamseekm]

I forgot to supply utf8 encoding option to mysql connection while trying to get some unicode text from db. So I was essentially passing garbled mess to langid.py and asking it to identify the language. Again langid.py is a great tool it was my stupid mistake. :)