akjindal53244 / arithmo Goto Github PK
View Code? Open in Web Editor NEWSmall and Efficient Mathematical Reasoning LLMs
License: Apache License 2.0
Small and Efficient Mathematical Reasoning LLMs
License: Apache License 2.0
Hi,
Could you please add a new baseline——MuggleMATH in the Comparing Arithmo-Mistral-7B with other LLM models section on the github webpage?MuggleMATH mainly investigates the scaling law and generalization for data augmentation for LLM math reasoning and gets comparable results in GSM8K. The address of paper is https://arxiv.org/abs/2310.05506 .
Thanks a lot!
I appreciate your dataset for math reasoning, But can you provide me more details for how you construct your test data (11k size in listed in the huggingface)? https://huggingface.co/datasets/akjindal53244/Arithmo-Data
would be good to release the training process as well :)
Hello,
I truly admire your work on fine-tuning LLMs for mathematical reasoning and I have a few questions about the data preprocessing. I would appreciate some insights into the following aspects:
1. Data Deduplication Impact
2. Effects of Random Lower-Casing
3. Diversity of PoT Prompts
I am also looking forward to any papers or further documentation you might release on this project, as I believe they would be incredibly informative.
Thank you for your dedication to this project and for taking the time to address my inquiries. Your work is truly inspiring.
Best regards,
lyf-00
Hi,
when I am trying to use your model for inference on my data, I get 'CUDA is out of memory' error.
when i try to quantize the model using bitsandbytes using your query_model.py, I get the following error while importing bitsandbytes:
File "/home/.conda/envs/designtodoc/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1355, in_get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.trainer because of the following error (look up to see its traceback):
[Errno 13] Permission denied: '/fs/applications/jupyterhub/gpu.jupyterhub.rng-dl01/srv/jupyterhub'
File "/home/prdsfwjehm/semanticgraph/few-shot-learning-mistral.py", line 7, in
model = AutoModelForCausalLM.from_pretrained("akjindal53244/Arithmo-Mistral-7B", device_map={"": "cpu"})
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/prdsfwjehm/.conda/envs/semanticgraph/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 527, in from_pretrained
config, kwargs = AutoConfig.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/prdsfwjehm/.conda/envs/semanticgraph/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py",line 1039, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/prdsfwjehm/.conda/envs/semanticgraph/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py",line 734, in getitem
raise KeyError(key)
KeyError: 'mistral'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.