- 👨🦰 I’m currently a M.Phil candidate of Peking University.
- 👦 Before that, I received the (Honours) B.E., HUST.
- ❤️🔥 Now, I am intersted in Multi-modal Learning especially MLLM.
- 💥 In 2023 summer, I take part in OSPP(Open Source Promotion Plan) Summer Camp
, with the honor of contributing for MMPretrain to build prompt-based classifier.
- 💥 2023.10: I implement MiniGPT4Qwen, which is a toy model aligning MiniGPT4 with Qwen-Chat LLM model. I just use 18.8k high quality instruction-tuning data(bi-lingual, selected from minigpt4 and llava). Just fine-tuning the projection layer (3M trainable parameters), this model support Chinese and English! MiniGPT4Qwen
- 💥 2024.2: I extend MiniGPT4Qwen to MPP-Qwen14B(Multimodal Pipeline Parallel), scaling up both the LLM(to Qwen-14B-Chat) and pretrain-data(using LLaVA-pretrain-data). I also unfreeze the whole LLM during SFT-stage. All training is conducted on 3090/4090 GPUs. To prevent poverty (24GB of VRAM) from limiting imagination, I implemented an MLLM version based on deepspeed Pipeline Parallel. Pre-training can be completed in 22 hours on 2x4090s, while SFT requires training on 6x4090s (because it needs to fully activate the LLM), but due to the small amount of data, it only takes several hours.MPP-Qwen14B
- 💥 2024.6: MPP-Qwen-Next
is released! Support {video/image/multi-image} {single/multi-turn} conversations. All training periods are conducted on 8 RTX3090(24GB) GPUs. MPP-Qwen-Next.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent