git
ask
hub
Privacy
Terms
Sign in with GitHub
brainchip-inc/do-transformers-need-3-projections
↗
Stars ·
43
Language ·
Python
License ·
MIT
Simplify and Visualize This Repo
Scan the safety of this repo
Self Host this repo
Check out more work of Developer: brainchip-inc
Ask anything about this repo to start.
I want to reproduce the Q-K=V experiment from this repo at the 300M parameter scale. What hardware do I need and which training script should I run first?
Explain the difference between the four attention projection variants tested in this paper and show me which config file to edit to switch between them.
I want to apply the Q-K=V projection sharing technique to my own transformer architecture. Walk me through the code changes in the training script that implement this.
Run the profiler script from this repo and explain what each column in the output table corresponds to in the paper theoretical analysis.
Full explanation on explaingit →
📎
Send
By chatting or signing in you agree to the
Terms
and chat-message logging (revocable in
History
).