Comments (4)
Yes, it is definitely possible to retrofit a different base model with QLoRa, using the guidelines from our repository. The process involves modifying the code that loads and converts the base model's weights into the QLoRa format, and then training the resulting QLoRa-compatible model on your data.
from efficient-tuning-llms.
Yes, there are many models open-sourced on Huggingface support QLoRA, such as OPT
, 'LLAMA', 'BLOOM', 'GPT-NeoX'...
from efficient-tuning-llms.
That's remarkable! I do have another question though. Are your code adaptable with other base models? Specifically, there's a base model on Huggingface that I'm interested in, but it's not currently compatible with QLoRa. Would it be possible to retrofit this specific model with QLoRa, using the guidelines from your repo?
from efficient-tuning-llms.
Based on the goals I've mentioned earlier, which file should I refer to and modify? Is it qlora_finetune.py or qlora_int4_finetune.py? Because I noticed that in finetune-baichuan-7b.sh, qlora_finetune.py is being used directly.
Also, regarding the adaptation for other models, I've been encountering OOM errors on a 3090-24GB setup with the CpmBee-1B model. I'm not sure why this is happening. So, what should I specifically look out for in your code when it comes to using them to adapt QLoRA to other base models? maybe in get_accelerate_model?
The finetune process in CpmBee is like the following, I am not sure how to make it in your code:
from transformers import AutoTokenizer, AutoModelForCausalLM
from accelerate import Accelerator
from torch.utils.data import Dataset, DataLoader
accelerator = Accelerator()
trainset = Dataset() # Make sure trainset.__getitem__() can get data with correct format like {"input": "...", "<ans>": ""}
# for details, you can read https://github.com/OpenBMB/CPM-Bee/tree/main/tutorials/basic_task_finetune
train_loader = DataLoader(trainset, batch_size=1)
tokenizer = AutoTokenizer.from_pretrained("openbmb/cpm-bee-10b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("openbmb/cpm-bee-10b", trust_remote_code=True).cuda()
optimizer = torch.optim.Adam(model.parameters())
model, optimizer, train_loader = accelerator.prepare(
model, optimizer, train_loader
)
for iter, data in enumerate(train_loader):
optimizer.zero_grad()
# change the data to a trainable format
input_encoded = tokenizer.prepare_for_finetune(data, max_length=512).to(model.device)
outputs = model(**input_encoded)
loss = outputs.loss
accelerator.backward(loss)
optimizer.step()
from efficient-tuning-llms.
Related Issues (20)
- 该项目与qlora的差别
- 不同样式的样本对应什么样的情形,如何根据自己的需求选择样本的样式
- 参数设置
- 中文文档里没写对baichuan-13B的支持,但英文写了 HOT 1
- 微调训练失败 HOT 4
- 多卡似乎不能将每张卡跑满,请问如何才能让每张卡的计算负载跑满呢 HOT 13
- 单机多卡并行训练报错 HOT 3
- Baichuan7B使用lora微调后测试时总会再次输出query HOT 1
- 下载了百川7b模型后,直接在gradio_webserver.py里推理,生成内容乱码问题 HOT 3
- ValueError: Undefined dataset tatsu-lab/alpaca HOT 3
- llama2-13B和llama2-70b微调所需要的显卡配置 HOT 1
- 如何使用自己的数据集 HOT 1
- zero3保存的模型无法加载 HOT 2
- About llama-2-70B fine-tuning HOT 2
- [问题]有关训练可视化 HOT 2
- 总是这个错误怎么解决 HOT 1
- llama-2-13b的模型用单卡跑lora就会报错 HOT 1
- QLORA微调alpaca_data.json报错 'padding_value' (position 3) must be float, not NoneType HOT 2
- 微调后的Llama-2-7b,在模型加载时出错 HOT 1
- 批量推理时结果异常
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from efficient-tuning-llms.