git
ask
hub
Privacy
Terms
Sign in with GitHub
tlrmchlsmth/lmcache-tests
↗
License ·
Apache-2.0
Simplify and Visualize This Repo
Scan the safety of this repo
Self Host this repo
Check out more work of Developer: tlrmchlsmth
Ask anything about this repo to start.
Set up lmcache-tests to benchmark a Llama 3.1 8B model with and without LMCache enabled, using the default workload, and generate a PDF report comparing time-to-first-token and throughput.
Run only the Redis backend tests from lmcache-tests against my vLLM deployment and show me the CSV output with per-request latency and GPU memory metrics.
Create a new test scenario in lmcache-tests that simulates a customer support chatbot with 5-turn conversations at 20 requests per second, comparing cached vs uncached performance.
Filter lmcache-tests to run only chunked prefill scenarios and explain what the results tell me about whether caching helps with variable-length workloads.
Full explanation on explaingit →
📎
Send
By chatting or signing in you agree to the
Terms
and chat-message logging (revocable in
History
).