bench.sllim.ai · coming soon
What a business workload costs on each model, and whether it still does the job.
A public benchmark built from the jobs companies actually send to language models: support triage, document extraction, code review, claims intake, answers from internal documents. Each result will say which configuration cleared the quality floor, how sure we are, and what the monthly bill would be with and without caching and batch pricing.
How it will work
- Scenarios come before results. Every suite states what we assume about a kind of business, with a source for each assumption or a plain label saying it is our guess. Nothing was run while the scenarios were written.
- A pass rate comes with its bound. Results report an exact lower bound on the pass rate, not just the average, and say whether the sample is a pilot, a screen, or a certification.
- Sealed cases stay sealed. The cases that certify a result are written by people, are never shown, and every read of them is logged.
- The bill is the whole bill. Input, output, reasoning, cache reads and writes, batch discounts, long-context tiers, and retries, priced from the vendors' own pages on a stated date.
- Results that go against us are published too. When no cheaper configuration clears the floor, that is the result.
Results are not public yet. Until they are, nothing on this site is a measurement. Questions: sllim.ai/contact.