git
ask
hub
Privacy
Terms
Sign in with GitHub
openai/simple-evals
↗
Stars ·
4,570
Language ·
Python
License ·
MIT
Simplify and Visualize This Repo
Scan the safety of this repo
Self Host this repo
Check out more work of Developer: openai
Ask anything about this repo to start.
Using the openai/simple-evals library, write code to run the MMLU benchmark on gpt-4o and print accuracy scores broken down by subject category.
How do I add a new model provider to simple-evals so I can benchmark a local LLaMA model on the same GPQA tasks OpenAI uses?
Show me how to run the HumanEval benchmark from simple-evals on Claude claude-sonnet-4-6 using the Anthropic API.
Using simple-evals, compare GPT-4o and a custom fine-tuned model on SimpleQA factual accuracy and output a results table.
Explain the difference between the zero-shot prompting approach in simple-evals and older few-shot evaluation methods, and show me the actual prompt template used for MATH.
Full explanation on explaingit →
📎
Send
By chatting or signing in you agree to the
Terms
and chat-message logging (revocable in
History
).