Comments (3)
shibing624
commented on July 27, 2024
reward model 是个排序模型，是个打分模型，你可以看下代码实现，loss是用kl散度计算的。
另外RLHH是啥？
from medicalgpt.
yangliuIOC
commented on July 27, 2024
我的意思是，reward model 是hf训练出来的， 而不是sft 训练出来的。
from medicalgpt.
shibing624
commented on July 27, 2024
reward model 可以用sft后的model开始训练，也可以用robert等其他预训练模型开始训练。没写过rm是sft训练出来的。
from medicalgpt.
Related Issues (20)
- 使用deepspeed 全参数sft后，inference 回答的都为空，有解决办法吗 HOT 2
- Regarding RLHF and DPO training data HOT 2
- UserWarning: None of the inputs have requires_grad=True. Gradients will be None HOT 2
- reward_modeling咨询 HOT 1
- orpo脚本NoneType问题 HOT 6
- 训练reward_modeling.py HOT 1
- 几步的训练怎么都是独立的，rm都没用sft的adapter HOT 1
- 对chat模型进行二次预训练后，自问自答 HOT 1
- 关于提前结束训练 HOT 4
- dpo_training.py eal存在空的情况 HOT 2
- AMD 执行 run_pt.sh失败 HOT 1
- 有没有人能分享下自己微调后的模型id，我懒得弄，只想吃现成的 HOT 1
- vocab扩展后的模型合并问题 HOT 1
- ppo训练时出现问题：UserWarning: KL divergence is starting to become negative: -233.50 HOT 2
- DPO训练，报错：“IndexError: Invalid key: 0 is out of bounds for size 0” HOT 2
- 运行pretraining.py时报错：RuntimeError: CUDA error: device-side assert triggered HOT 4
- 医学大模型全流程体验 HOT 2
- 关于llama3的权重转换 HOT 1
- ValueError: Please specify target_modules in peft_config HOT 1
- PPO和SFT阶段数据集 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from medicalgpt.