hanguo97 / lq-lora Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
Is it possible to release different bit model weights without any training? I.e. newly initialized LoRA weights and Q.
I couldn't find the training time in your paper.
From your training script, it seems you use 1 and 4 GPU for Llama 7b and 70b, respectively. May I ask what is the training time for Llama 7b and 70b on c4? What is your GPU type, A100 80G or 40G version?
Hi,
Great project! Thanks for releasing it! Would you mind adding a license (ie MIT) so it can be used in production?
Thank you!
Could you offer me the full GLUE scores, i.e. scores for different tasks, for comparison?
I can only find the average GLUE score in Table 2. However, I only want to do the experiments on some tasks for the first step and compare my results to your results. It would be great if you could offer the full scores of all methods in Table 2. Thank you in advance.
Hi Guo,
Hopes everything goes well with you.
I tried to run all the .sh scripts and here is the question I could not solve right now. Could you please help me out of this?
Currently running a script named glue_lora.sh. There are 4 tasks in this script, task1-4. The file output by task 1 and task 2 is "output_glue_dense_20230924_ranks64_qnli".
And task 3&4 are based on the output files of task 1 and 2 (model training weights, etc.) roberta-large.ilp.ranks-64.data-True.pth to run. But the current problem is: The files produced by task 1&2 do not contain pth file, or the codes does not include the part that converted the dense output into pth files. It leads to the error of FileNotfoundError: lErrno 2] No such file or directory: '/home/ubuntu/lg-lora/llama-2/ilp data fp32/roberta-large.ilp.ranks-64.data-True.pth. Therefore, I guess we may need to add some functions to run_glue.py to store pth file correctly.
Thanks for answering!
https://github.com/HanGuo97/lq-lora/blob/main/models/allocation_utils.py#L277 I got this error while using backend: scipy.
i0 is 20 and i1 is 19. In this case, how can I solve this issue?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.