git
ask
hub
Privacy
Terms
Sign in with GitHub
david19p/custom-llm-kernel-2080
↗
Stars ·
2
Language ·
C++
License ·
MIT
Simplify and Visualize This Repo
Scan the safety of this repo
Self Host this repo
Check out more work of Developer: david19p
Ask anything about this repo to start.
Clone custom-llm-kernel-2080 with submodules and build it with cmake for CUDA arch 75 on Linux with GCC.
Download the recommended Qwen3.5-9B 4-bit target plus the DFlash draft and PFlash drafter from the Hugging Face links and wire them into the quick-start launch command.
Point my existing OpenAI chat client at the local /v1/chat/completions endpoint and stream a code completion to confirm SSE works.
Reproduce the HumanEval benchmark on my RTX 2080 and compare baseline tokens per second against DFlash with DDTree verification.
Add notes to the README about which roadmap items, like tool-calling parity and grammar masking, are still missing.
Full explanation on explaingit →
📎
Send
By chatting or signing in you agree to the
Terms
and chat-message logging (revocable in
History
).