Comments (3)
so the max_length
is used to truncate the prompt+completions rather than remove them... I suppose if you need to remove them can't do that in the data-preparation stage?
from trl.
That's what I mean. You have to setup your own way of filtering out the samples beforehand, when it would make more sense to have a flag to drop instead of truncate. The code is already checking sample length to decide if it needs to truncate, so the code could just be done there to drop the samples instead.
from trl.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
from trl.
Related Issues (20)
- Dataset format for `ORPOTrainer` HOT 4
- [Bug][OnlineDPO] Far too many checkpoints are saved
- PPOV2 Trainner use Deepspeed Zero3 Offload CPU: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cpu! HOT 1
- Rewards actual_end index in PPOv2 trainer HOT 1
- How to speed up PPOTrainer .generate()? HOT 1
- Please add TPO trainer to the trl
- Problem in dpo_llama2.py? NCCL timeout?
- Configuration argument overriding by deprecated default values
- SFTTrainer.evaluate() predictions and computing metrics
- When eval_packing=False, training failed with KeyError: 'eval_loss' .
- Cannot import name 'DPOConfig' from 'trl' HOT 2
- Minor KTO mis-estimation of KL term on odd batch sizes
- Redundant special tokens when applying chat template (?)
- DataCollatorForCompletionOnlyLM not working with MPS HOT 3
- How to dpo llava?
- How to Add PEFT to PPO Trainer or PPO Config
- Error in AutoModelForCausalLMWithValueHead.load_pretrained("path/to/my/ppo_model")
- Adding Space caused " This instance will be ignored in loss calculation. Note, if this happens often, ..." HOT 1
- losses = losses * self.args.rpo_alpha + policy_nll_loss TypeError: only integer tensors of a single element can be converted to an index HOT 1
- how to use kto_pair loss in the latest version ? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from trl.