git
ask
hub
Privacy
Terms
Sign in with GitHub
pingsizhong1/arctic-inference-llm
↗
Stars ·
19
Language ·
Python
License ·
Apache-2.0
Simplify and Visualize This Repo
Scan the safety of this repo
Self Host this repo
Check out more work of Developer: pingsizhong1
Ask anything about this repo to start.
Explain what Arctic Inference does as a plugin on top of vLLM.
Walk me through installing this and running the offline Python example script from the README.
Show me the difference between Shift Parallelism and speculative decoding as used in this project.
Help me understand the throughput and latency benchmark claims in this repo's README.
Full explanation on explaingit →
📎
Send
By chatting or signing in you agree to the
Terms
and chat-message logging (revocable in
History
).