graphcore / gpt-j

Notebook for running GPT-J/GPT-J-6B – the cost-effective alternative to ChatGPT, GPT-3 & GPT-4 for many NLP tasks. Available on IPUs as a Paperspace notebook.

GPT-J and GPT-J-6B on IPUs

GPT-J is an open-source alternative from EleutherAI to OpenAI's GPT-3. Available for anyone to download, GPT-J can be successfully fine-tuned to perform just as well as large models on a range of NLP tasks including question answering, sentiment analysis, and named entity recognition.

Try running GPT-J for yourself on Paperspace with Graphcore's IPU (Intelligence Processing Unit), a completely new kind of massively parallel processor to accelerate machine intelligence. Access advanced, cost-efficient IPU compute on demand in the cloud on Paperspace to build, fine-tune and deploy AI models such as GPT-J.

GPT-J notebooks powered by IPUs

Notebook	Framework	Type	Try for Free
Textual Entailment on IPU using GPT-J - Fine-tuning	Hugging Face	Fine-tuning
Text generation with GPT-J 6B	Hugging Face	Inference
Faster Text Generation with GPT-J using 4-bit Weight Quantization on IPUs	Hugging Face	Inference

In the Textual Entailment on IPU using GPT-J - Fine-tuning notebook, we show how to fine-tune a pre-trained GPT-J model running on a 16-IPU system on Paperspace. We will explain how you can fine-tune GPT-J for Text Entailment on the GLUE MNLI dataset to reach SOTA performance, whilst being much more cost-effective than its larger cousins.

In the Text generation with GPT-J 6B notebook, we demonstrate how easy it is to run GPT-J on the Graphcore IPU using this implementation of the model and 🤗 Hub checkpoints of the model weights.

In the Faster Text Generation with GPT-J using 4-bit Weight Quantization on IPUs notebook, we show how to use group quantisation to compress model parameters to 4 bits with no fine-tuning, using 4x less memory and speeding up inference on GPT-J by ~1.5x.

GPT-J resources

• Hugging Face Model
• Blog
• How-to walkthrough blog
• Original Paper

To take your GPT-J usage on IPUs further, or speak to an expert, please feel free to contact us.

IPU community

Join our growing community and interact with AI experts, IPU developers and researchers. Hear the latest IPU news and get access to our newest models.

License

The contents of this repository are made available according to the terms of the MIT license. See the included LICENSE file for details.