gitaskhub

CascadeExit: Adaptive early-exit speculative decoding for LLM inference acceleration. 1.76x speedup on Llama-3.2-3B with 0.51% parameter overhead via confidence-calibrated cascade routing

Language · Jupyter Notebook
License · MIT
Ask anything about this repo to start.

By chatting or signing in you agree to the Terms and chat-message logging (revocable in History).