- ๐ Hi, Iโm kathir
- ๐ Iโm interested in Deep Learning...
- ๐ฑ Iโm learning Language Models and their applications...
kathir-ks / indictrans2-tokenization Goto Github PK
View Code? Open in Web Editor NEWA codebase to split text into sentences and tokenize them in batches for inference using IndicTrans2