edumunozsala / llama-2-7b-4bit-python-coder

Fine-tune and quantize Llama-2-like models to generate Python code using QLoRA, Axolot,..

Hello, my name is Eduardo Muñoz Sala 👋

---

I'm an experienced Software Engineer, Data Scientist and a Machine Learning Engineer and practitioner in a continuous learning path. I love building projects in ML, Generative AI and testing tools for MLOps architectures. Great skills in Data Analysis, Project Management and leading SW Development teams

• 📚 I’m always studying courses on Coursera, Deeplearning.ai, Udemy, etc... improving my skills on ML, Generative AI and SW engineering
• 💥 I love testing new tools and platforms about Vector databases, Data management and MLOps
• 🔨 Great interest in NLP in spanish: train and fine-tune models and test for improvements
• ✍ I’m looking to write more articles on Medium

Detailed board on my Studies & Personal Projects on Data Science and Machine Learning

Certifications

• 📝 Microsoft Certified: Azure Data Scientist Associate
• 📝 Servicenow Certified System Administrator
• 📝 Microsoft Certified: Azure AI Fundamentals
• 📝 Microsoft Certified: Azure Data Fundamentals
• 📝 AWS Certified Machine Learning Specialty
• 📝 Deeplearningai Specializations: Generative AI with LLM, Practical Data Science on AWS, NLP, GANs, Deep Learning, Tensorflow
• 📝 Machine Learning Engineer Nanodegree on Udacity
• 📝 Data engineer, Big Data and Machine Learning in Google Cloud Platform
• 📝 Microsoft Professional Program in Artificial Intelligence
• 📝 Microsoft Professional Program in Data Science
• 📝 Applied Data Science in Python Specialization Michigan University

Connect with me:

Languages and Tools:

---

💻 Latest Projects

• 📰 Fine-Tuning a Llama-2 7B Model for Python Code Generation. Blog Code
• 📰 Embeddings with Sentence Transformers and Pinecone for Question Answering in Spanish. Blog Code
• 📰 And end-to-end AWS Sagemaker Pipeline to prepare data, train, evaluate and conditional deployment of a Keras Text Classification model. Code
• 📰 Training a PyTorch Text Summarization Model on Vertex AI using a pretrained Huggingface model. Code
• 📰 Train and Evaluate a Sentence Transformer model for news in spanish in AWS Sagemaker. Code
• 📰 Train and Deploy on AWS Sagemaker a Sentiment Classifier on IMDb Reviews in Spanish. Code
• 📰 Fine-tune a RoBERTa Encoder-Decoder model trained on MLM for Text Generation. Blog Code
• 📰 Create a Tokenizer and Train a Huggingface RoBERTa Model from Scratch. Blog Code
• 📰 Apply Machine Learning in an Easy and Fast Way with AWS AutoPilot: Diving Deep on AutoML. Blog Code
• 📰 Fine-tune a Huggingface T5 model for Text Summarization using AWS Sagemaker and Weights and Biases. Code
• 📰 Text Summarization Guide: Development of EDA for text data, Encoder-Decoder with Attention, Transformer using Tensorflow 2, AWS SageMaker and Weight&Biases. Blog Code
• A Guide to the Encoder-Decoder Model and the Attention Mechanism: build a encoder decoder architecture using LSTM units with the Luong's Attention in Tf2- Blog Code
• Character-level text generator with Pytorch and Amazon SageMaker - Blog Code
• 📜 Intro to NLP and Text Classification: Compilation of notebooks introducing some relevants topics and concepts to get started in Natural Language Processing. - Blog Code
• Malaria Predictor based on Image Classification: An image classifier to predict if a cell is infected or not with malaria. CNNs, data augmentation and transfer learning using Azure Machine Learning Services - Code
• 🌲 Forest Cover Type Predictor: EDA to analyze the problem and develope a XGBoost model and an stacked ensemble model to predict the cover type in Roosevelt National Forest. - Code
• 🔈 Using Machine Learning to identify accents in spectrograms of speech: Capstone Project in the Microsoft Professional Program in Artificial Intelligence, CNN to identify accents in spectrograms using Azure Machine Learning Services - Blog Code
• 🏠 Predicting Mortgage Approvals: A capstone project in the Microsoft Professional Program in Data Science, an Exploratory Data Analysis on applications and machine learning model in Azure Machine Learning Studio. - Blog Code

---

📰 Latest Blog Posts

• Fine-Tuning a Llama-2 7B Model for Python Code Generation
• Embeddings with Sentence Transformers and Pinecone for Question Answering in Spanish
• Fine-tune a RoBERTa Encoder-Decoder model trained on MLM for Text Generation
• Create a Tokenizer and Train a Huggingface RoBERTa Model from Scratch
• Apply Machine Learning in an Easy and Fast Way with AWS AutoPilot: Diving Deep on AutoML
• Text Summarization Guide: Exploratory Data Analysis on Text Data
• Steps to Start Training your Custom Tensorflow Model in AWS SageMaker
• Attention is all you need: Discovering the Transformer paper
• A Guide to the Encoder-Decoder Model and the Attention Mechanism
• Intro to RNN: Character-Level Text Generation With PyTorch
• Using Machine Learning to identify Accents in Spectrograms of Speech. Part 2
• Predicting mortgage approvals: Data analysis and prediction with Azure ML Studio

llama-2-7b-4bit-python-coder's Issues

Cant run the code on Colab

hi

thz for so comprehensive guide which it is really helpful for me to understanding the state-of-art in the field. However, I could not run the code which you shared on Colab.

There are the Erorrs :
#1 on # Define the training arguments box: ValueError: Your setup doesn't support bf16/gpu. You need torch>=1.10, using Ampere GPU with cuda>=11.0
so I charged it from "fp16 = False, bf16 = True" to "fp16 = True, bf16 = False" on the Setting Global Parameters cell. is it ok to charge it?

#2 after # train cell called OutOfMemoryError since "CUDA out of memory. Tried to allocate 2.00 GiB (GPU 0; 14.75 GiB total capacity; 10.23 GiB already allocated; 790.81 MiB free; 12.92 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF"
this time I have no ideas to fix :(