- Train small LLM; 2. Use its outputs on the training data as labels for training large LLM, where their argmax agrees with the training data.
snimu / llm-small-to-large Goto Github PK
View Code? Open in Web Editor NEW1. Train small LLM; 2. Use its outputs on the training data as labels for training large LLM, where their argmax agrees with the training data.
License: Apache License 2.0