git
ask
hub
Privacy
Terms
Sign in with GitHub
yedaotian9/lite-opd
↗
Stars ·
37
Language ·
Python
License ·
MIT
Simplify and Visualize This Repo
Scan the safety of this repo
Self Host this repo
Check out more work of Developer: yedaotian9
Ask anything about this repo to start.
Set up Lite-OPD to distill a Qwen 2.5 1.5B student from a Qwen 2.5 7B teacher on one H100, and show me the exact command.
Walk me through the four steps of a Lite-OPD training step and point to the line in the single-file loop where the KL divergence is computed.
Swap the loss function in Lite-OPD from KL divergence to a reverse-KL variant and explain which lines to edit.
Reproduce the Lite-OPD vs ms-swift benchmark on a pair of H20 GPUs and report tokens-per-second per step.
Modify Lite-OPD to log per-position teacher entropy during training and write a small matplotlib plot script.
Full explanation on explaingit →
📎
Send
By chatting or signing in you agree to the
Terms
and chat-message logging (revocable in
History
).