fourthbrain / building-with-instruction-tuned-llms-a-step-by-step-guide

Resources relating to the DLAI event: https://www.youtube.com/watch?v=eTieetk2dSw

👋 Welcome to the Support Repository for the DeepLearningAI Event: Building with Instruction-Tuned LLMs: A Step-by-Step Guide

Here are a collection of resources you can use to help fine-tune your LLMs, as well as create a few simple LLM powered applications!

🗣️ Slides:

Here are the slides used for the event: Building with Instruction-Tuned LLMs

🪡 Fine-tuning Approaches:

Instruct-tuning OpenLM's OpenLLaMA on the Dolly-15k Dataset Notebooks:

Notebook	Purpose	Link
Instruct-tuning Leveraging QLoRA	Supervised fine-tuning!	Here
Instruct-tuning Leveraging Lit-LLaMA	Using Lightning-AI's Lit-LLaMA frame for Supervised fine-tuning	Here
Natural Language to SQL fine-tuning using Lit-LLaMA	Using Lightning-AI's Lit-LLaMA frame for Supervised fine-tuning on the Natural Language to SQL task	Here

MarketMail Using BLOOMz Resources:

Notebook	Purpose	Link
BLOOMz-LoRA Unsupervised Fine-tuning Notebook	Fine-tuning BLOOMz with an unsupervised approach using Sythetic Data!	Here
Creating Synthetic Data with GPT-3.5-turbo	Generate Data for Your Model!	Here

🏗️ Converting Models into Applications:

Notebook	Purpose	Link
Open-source LangChain Example	Leveraging LangChain to build a Hugging Face 🤗 Powered Application	Here
Open AI LangChain Example	Building an Open AI Powered Application	Here

🖥️ Demos:

Demo	Info	Link
Instruct-tuned Chatbot Leveraging QLoRA	This demo is currently powered by the Guanaco Model - will be updated once our instruct-tuned model finishes training!	Here
TalkToMyDoc	Query the first Hitch Hiker's Guide book!	Here

building-with-instruction-tuned-llms-a-step-by-step-guide's Issues

CUDA error: device-side assert triggered; CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.

I'm re-running the Supervised_Instruct_tuning_OpenLLaMA_... notebook on Colab Pro with A100 GPU. I got the following error during supervised_finetuning_trainer.train() step (a quick Stack Overflow search suggests shape mismatch as the likely cause, but I did not change anything in the code):

You're using a LlamaTokenizerFast tokenizer. Please note that with a fast tokenizer, using the __call__ method is faster than using a method to encode the text followed by a call to the pad method to get a padded encoding.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <cell line: 1>:1 │
│ │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:1696 in train │
│ │
│ 1693 │ │ inner_training_loop = find_executable_batch_size( │
│ 1694 │ │ │ self._inner_training_loop, self._train_batch_size, args.auto_find_batch_size │
│ 1695 │ │ ) │
│ ❱ 1696 │ │ return inner_training_loop( │
│ 1697 │ │ │ args=args, │
│ 1698 │ │ │ resume_from_checkpoint=resume_from_checkpoint, │
│ 1699 │ │ │ trial=trial, │
│ │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:1973 in _inner_training_loop │
│ │
│ 1970 │ │ │ │ │ with model.no_sync(): │
│ 1971 │ │ │ │ │ │ tr_loss_step = self.training_step(model, inputs) │
│ 1972 │ │ │ │ else: │
│ ❱ 1973 │ │ │ │ │ tr_loss_step = self.training_step(model, inputs) │
│ 1974 │ │ │ │ │
│ 1975 │ │ │ │ if ( │
│ 1976 │ │ │ │ │ args.logging_nan_inf_filter │
│ │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:2787 in training_step │
│ │
│ 2784 │ │ │ return loss_mb.reduce_mean().detach().to(self.args.device) │
│ 2785 │ │ │
│ 2786 │ │ with self.compute_loss_context_manager(): │
│ ❱ 2787 │ │ │ loss = self.compute_loss(model, inputs) │
│ 2788 │ │ │
│ 2789 │ │ if self.args.n_gpu > 1: │
│ 2790 │ │ │ loss = loss.mean() # mean() to average on multi-gpu parallel training │
│ │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:2819 in compute_loss │
│ │
│ 2816 │ │ │ labels = inputs.pop("labels") │
│ 2817 │ │ else: │
│ 2818 │ │ │ labels = None │
│ ❱ 2819 │ │ outputs = model(**inputs) │
│ 2820 │ │ # Save past state if it exists │
│ 2821 │ │ # TODO: this needs to be fixed and made cleaner later. │
│ 2822 │ │ if self.args.past_index >= 0: │
│ │
│ /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:1501 in _call_impl │
│ │
│ 1498 │ │ if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks │
│ 1499 │ │ │ │ or _global_backward_pre_hooks or _global_backward_hooks │
│ 1500 │ │ │ │ or _global_forward_hooks or _global_forward_pre_hooks): │
│ ❱ 1501 │ │ │ return forward_call(*args, **kwargs) │
│ 1502 │ │ # Do not call functions when jit is used │
│ 1503 │ │ full_backward_hooks, non_full_backward_hooks = [], [] │
│ 1504 │ │ backward_pre_hooks = [] │
│ │
│ /usr/local/lib/python3.10/dist-packages/peft/peft_model.py:827 in forward │
│ │
│ 824 │ │ │ │ │ **kwargs, │
│ 825 │ │ │ │ ) │
│ 826 │ │ │ │
│ ❱ 827 │ │ │ return self.base_model( │
│ 828 │ │ │ │ input_ids=input_ids, │
│ 829 │ │ │ │ attention_mask=attention_mask, │
│ 830 │ │ │ │ inputs_embeds=inputs_embeds, │
│ │
│ /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:1501 in _call_impl │
│ │
│ 1498 │ │ if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks │
│ 1499 │ │ │ │ or _global_backward_pre_hooks or _global_backward_hooks │
│ 1500 │ │ │ │ or _global_forward_hooks or _global_forward_pre_hooks): │
│ ❱ 1501 │ │ │ return forward_call(*args, **kwargs) │
│ 1502 │ │ # Do not call functions when jit is used │
│ 1503 │ │ full_backward_hooks, non_full_backward_hooks = [], [] │
│ 1504 │ │ backward_pre_hooks = [] │
│ │
│ /usr/local/lib/python3.10/dist-packages/accelerate/hooks.py:165 in new_forward │
│ │
│ 162 │ │ │ with torch.no_grad(): │
│ 163 │ │ │ │ output = old_forward(*args, **kwargs) │
│ 164 │ │ else: │
│ ❱ 165 │ │ │ output = old_forward(*args, **kwargs) │
│ 166 │ │ return module._hf_hook.post_forward(module, output) │
│ 167 │ │
│ 168 │ module.forward = new_forward │
│ │
│ /usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py:687 in │
│ forward │
│ │
│ 684 │ │ return_dict = return_dict if return_dict is not None else self.config.use_return │
│ 685 │ │ │
│ 686 │ │ # decoder outputs consists of (dec_features, layer_state, dec_hidden, dec_attn) │
│ ❱ 687 │ │ outputs = self.model( │
│ 688 │ │ │ input_ids=input_ids, │
│ 689 │ │ │ attention_mask=attention_mask, │
│ 690 │ │ │ position_ids=position_ids, │
│ │
│ /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:1501 in _call_impl │
│ │
│ 1498 │ │ if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks │
│ 1499 │ │ │ │ or _global_backward_pre_hooks or _global_backward_hooks │
│ 1500 │ │ │ │ or _global_forward_hooks or _global_forward_pre_hooks): │
│ ❱ 1501 │ │ │ return forward_call(*args, **kwargs) │
│ 1502 │ │ # Do not call functions when jit is used │
│ 1503 │ │ full_backward_hooks, non_full_backward_hooks = [], [] │
│ 1504 │ │ backward_pre_hooks = [] │
│ │
│ /usr/local/lib/python3.10/dist-packages/accelerate/hooks.py:165 in new_forward │
│ │
│ 162 │ │ │ with torch.no_grad(): │
│ 163 │ │ │ │ output = old_forward(*args, **kwargs) │
│ 164 │ │ else: │
│ ❱ 165 │ │ │ output = old_forward(*args, **kwargs) │
│ 166 │ │ return module._hf_hook.post_forward(module, output) │
│ 167 │ │
│ 168 │ module.forward = new_forward │
│ │
│ /usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py:536 in │
│ forward │
│ │
│ 533 │ │ │ attention_mask = torch.ones( │
│ 534 │ │ │ │ (batch_size, seq_length_with_past), dtype=torch.bool, device=inputs_embe │
│ 535 │ │ │ ) │
│ ❱ 536 │ │ attention_mask = self._prepare_decoder_attention_mask( │
│ 537 │ │ │ attention_mask, (batch_size, seq_length), inputs_embeds, past_key_values_len │
│ 538 │ │ ) │
│ 539 │
│ │
│ /usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py:464 in │
│ _prepare_decoder_attention_mask │
│ │
│ 461 │ │ # [bsz, seq_len] -> [bsz, 1, tgt_seq_len, src_seq_len] │
│ 462 │ │ combined_attention_mask = None │
│ 463 │ │ if input_shape[-1] > 1: │
│ ❱ 464 │ │ │ combined_attention_mask = _make_causal_mask( │
│ 465 │ │ │ │ input_shape, │
│ 466 │ │ │ │ inputs_embeds.dtype, │
│ 467 │ │ │ │ device=inputs_embeds.device, │
│ │
│ /usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py:49 in │
│ make_causal_mask │
│ │
│ 46 │ Make causal mask used for bi-directional self-attention. │
│ 47 │ """ │
│ 48 │ bsz, tgt_len = input_ids_shape │
│ ❱ 49 │ mask = torch.full((tgt_len, tgt_len), torch.tensor(torch.finfo(dtype).min, device=de │
│ 50 │ mask_cond = torch.arange(mask.size(-1), device=device) │
│ 51 │ mask.masked_fill(mask_cond < (mask_cond + 1).view(mask.size(-1), 1), 0) │
│ 52 │ mask = mask.to(dtype) │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be
incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

Please provide access to MarketMail NB

Hi,

can you please provide access to the NB - https://colab.research.google.com/drive/1ARmlaZZaKyAg6HTi57psFLPeh0hDRcPX?usp=sharing

Getting the error - "There was an error loading this notebook. Ensure that the file is accessible and try again.
The caller does not have permission"

Getting issue while running the training cell in this notebook: BLOOMz-LoRA Unsupervised Fine-tuning Notebook

RuntimeError: unscale_() has already been called on this optimizer since the last update().

Bug report: uploading model to Huggingface failed

Hi, after I finished training and trying to upload model to HF using command

base_model.push_to_hub("zbruceli/openLLaMA_QLora", private=True)

The following error occurred:

NotImplementedError: You are calling save_pretrained on a 4-bit converted model. This is currently not supported

Are there something I need to change when I create the model/repo on HF? Thank you!

Uploading the tokenizer works fine, so it is not a HF token issue.

TalkToMyDoc Error

BloomForSequenceClassification' does not have a lm_head ... can this technique still apply?

re the notebook :✉️ MarketMail AI ✉️ Fine tuning BLOOMZ (Completed Version).ipynb

https://colab.research.google.com/drive/1ARmlaZZaKyAg6HTi57psFLPeh0hDRcPX?usp=sharing

i tried to modify the example to use BloomForSequenceClassification instead of AutoModelForCausalLM but the "Post-processing on the model":
model.lm_head = CastOutputToFloat(model.lm_head)
fails because BloomForSequenceClassification does not have an attribute lm_head.

This is true, so i change code to try and affect the last layer of BloomForSequenceClassification:
model.ln_f = CastOutputToFloat(model.ln_f)
This also fails: AttributeError: 'BloomForSequenceClassification' object has no attribute 'ln_f'

This leaves me wondering can thus gradient accumulation work for BloomForSequenceClassification? Or only for AutoModelForCausalLM? Alternatively, does anyone know if AutoModelForCausalLM can be used for fine tuning a classification task equally well as BloomForSequenceClassification?

ValueError: num_samples should be a positive integer value, but got num_samples=0

Notebook: Instruct-tuning Leveraging QLoRA

While executing:
supervised_finetuning_trainer.train()