mayday-wpf / sharptoken Goto Github PK
View Code? Open in Web Editor NEWThis project forked from dmitry-brazhenko/sharptoken
SharpToken is a C# library for tokenizing natural language text. It's based on the tiktoken Python library and designed to be fast and accurate.
Home Page: https://www.nuget.org/packages/SharpToken
License: MIT License