FAQ
Frequently asked questions about BLXBench.
General
What is BLXBench?
BLXBench is an open benchmark for AI models that evaluates speed, security, reasoning, and coding capabilities.
How is BLXBench different from other benchmarks?
- Open source — All tests are freely available
- Reproducible — Anyone can run the same tests
- No paid placements — Results are based purely on performance
- Community-driven — New tests can be contributed
Which models/providers are supported?
blxbench talks to adapters (OpenRouter, OpenAI, Hugging Face, Together, Portkey, Cloudflare — see Configuration). You pick a model id accepted by that endpoint (e.g. OpenRouter-style vendor/model). New adapters can be added in packages/benchmark-core/adapters/.
Using BLXBench
What is the npm package name?
@bitslix/blxbench (Bitslix on npm). The command after install is blxbench. See Installation.
How do I run a benchmark?
See Quick Start.
Do I need an API key?
Yes — you need credentials for the adapter you run against (e.g. OPENROUTER_API_KEY for opr).
A BLXBench API key is separate: use it for headless --submit uploads to the web app, together with a pass tier that includes submission quota. Interactive TUI login uses /auth login and the browser device flow.
How long does a benchmark take?
Depends on the model and test count:
- Quick run (5 tests): ~5 minutes
- Full run (~100 tests): ~30-60 minutes
Does BLXBench cost money?
blxbench and local benchmark runs are free to use. You pay your model provider for API usage, and public leaderboard submission requires a BLXBench pass tier that includes submission quota.
Results
How are scores calculated?
See Leaderboard.
Why did my model's score change?
Scores reflect the latest test results. Changes can occur when:
- New tests are added
- Model versions update
- Your run is superseded by newer submissions
Can I dispute a result?
Contact Support with details about the disputed result.
Troubleshooting
Playwright not found
Run:
blxbench --headless --install-chromiumRate limit errors
Use --ratelimit flag:
blxbench --headless --ratelimit 30API key errors
Make sure your API key is set:
export OPENROUTER_API_KEY=sk-or-...Cannot upload results
Ensure you have a BLXBench account, an API key, and a pass tier that includes leaderboard submission (Runner alone is local-only). See Account. Check the blxbench warning if --submit was skipped because BLXBENCH_API_KEY was missing.