- Huggingface Blog about efficient training on a single GPU
- Padding large-language-models
- How to generate text: using different decoding methods for language generation with Transformers
- Llama.cpp Tutorial
- Introduction to Llama
- Getting Started with Axolotl for Fine-Tuning LLMs
- The Novoice's LLM Training Guide
- Transformers from scratch
- SFT Training Example using TRL
- Perplexity evaluation metric
- Finetuning LLMs with LoRA and QLoRA by Sebastian Raschka
- Zephyr 7B Guide
- W&B's Training LLM by Thomas Capelle
- Pytorch blog on finetuning LLM
- Minimalistic implementation of LoRA with guidelines
- Maxime Labonne's Fine-tune Llama 2 on Google Colab.ipynb
- Llama from scratch by Brian Kitano
- Fine-Tune Llama2 | Step by Step Guide to Customizing Your Own LLM: Great Short introduction on custom SFT data creation and SFT training using TRL
- Learn RAG from scratch: Learn how to implement RAG (Retrieval Augmented Generation) from scratch using Langchains
- Aligning LLMs with Direct Preference Optimization
- Maxime Labonne LLM course
- W&B Training and Fine-tuning Large Language Models (LLMs)
- W&B Building LLM-Powered Apps
- LLM Datahub contain datasets for LLM Training
- LLaMA model (and others) inference in pure C/C++
- Karpathy's llama C implementation
- LLM evaluation using colabl notebook (AutoEval)
- Llama recipes by Meta
- Huggingface Alignment Handbook
- LLM foundry by Databricks
- Open-Platypus: SFT dataset for improving LLM logical reasoning skills and was used to train the Platypus2 models.
- guanaco-llama2-1k: 1000 example subset of
timdettmers/openassistant-guanaco
dataset in Llama 2's prompt format. Good for learning purposes. - Ultrachat 200K: Filtered version of the UltraChat dataset used to train Zephyr-7B-β model
- HF's datasets collections: Curated SFT datasets.