The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with 100+ tok/s single-request decode with support of FP8 weight
By chatting or signing in you agree to the Terms and chat-message logging (revocable in History).