MiniGPT, inspired by NanoGPT Inspired by NanoGPT (https://github.com/karpathy/nanoGPT), MiniGPT (created May 2023) was trained on a 1.3GB dataset that we created to eliminate offensive and less intelligent language found in the GPT-2 training dataset (Open Crawl), which originally consisted of approximately 120GB of data. To enhance efficiency, we developed miniText, a 1.3GB dataset containing carefully selected content.
The miniText dataset will be released soon.