apollohuang1 / cpm-live

Live Training for Open-source Big Models

CPM-Live

Live Training for Open-source Big Models

MFTable • Website • Plan • Discussion • 简体中文

What's New

• 2023/01/13 MFTable, powered by CPM-Bee (under training), is released! Process texts in a surprisingly simple and efficient way!
• 2022/10/12 CPM-Ant+, a bilingual model, is released! In addition to generating Chinese/English text, you can now use our model for QA, summarization and translation tasks!
• 2022/09/16 CPM-Ant is released!
• 2022/05/29 The training of CPM-Live has launched today! See training dynamics.
• 2022/05/25 The training plan for CPM-Live is now published. Look forward to the training!

Milestones

• CPM-Bee (2022/10/13- ) [Plan]
• CPM-Ant+ (2022/08/05-2022/10/12) [Code][Model]
• CPM-Ant (2022/05/29-2022/08/05) [Code][Model][Website][Blog][Plan]

Training Plan

Considering the scale of data and computing resources, CPM-Live will start with a 10B model training.

During training we will do:

• Real-time: Display model training metrics
• Every day: Release the model training log
• Every week: Deal with discussions and feedback from the community
• Irregularly: Release checkpoints during model training which everyone can download

During training you can:

• Raise your model proposal: Have better ideas on model architecture, training methods, or data sources? You can put forward your model proposal in the community. If the proposal receives more support and is practically feasible, we will add it to the model we are training, so that CPM-Live can learn continuously and progress with the help of everyone.

• Develop your application: You can submit your initial ideas, prototypes, development code, or finished apps, which are based on CPM-Live, to the community. We will exhibit the most popular apps on the website.

• Chat on the forum: You can talk about anything related to big models in our forums, such as academic research, engineering implementation, tool use, application design, etc. No matter whether you are experienced or not, we believe everyone can benefit from positive and open discussions.

• Download the resource: Once the model training is complete, you are free to download the model parameters under an open use license. CPM-Live uses an open license that includes permission for commercialization. With model compression and inference acceleration tools, you can experience the power of big models on your own PC!

Community

Our community is based on GitHub Discussions.

Read the first post and start your exploration on CPM-Live!