Quick Start

You need the blxbench command on your PATH. Install the published package @bitslix/blxbench first (see Installation).

Step 1: Create Your .env

Create a .env file in the directory where you run blxbench. OpenRouter is the default provider alias (opr):

OPENROUTER_API_KEY=

SUMMARY_PROVIDER=openrouter
SUMMARY_MODEL=qwen/qwen3-235b-a22b-2507

VALIDATION_MODEL=openai/gpt-5.4-mini

Fill OPENROUTER_API_KEY with your OpenRouter key. SUMMARY_PROVIDER and SUMMARY_MODEL enable AI-generated run summaries, while VALIDATION_MODEL is used to validate coding_ui fixtures. Without VALIDATION_MODEL, those validation checks are skipped.

Step 2: Run a Benchmark (headless)

Outside an interactive terminal, or when you pass --headless, blxbench runs without the TUI. There is no run subcommand — pass flags directly:

blxbench --headless --provider opr --models openai/gpt-5.4-mini

(--provider defaults to opr if omitted.) This runs the suite against the given model id(s).

Step 3: Filter Tests (Optional)

Run only specific categories or levels:

# Only speed tests
blxbench --headless --provider opr --models openai/gpt-5.4-mini --category speed

# Only easy difficulty
blxbench --headless --provider opr --models openai/gpt-5.4-mini --level easy

Step 4: View Results

After the run completes, results are saved to ./results/:

# View HTML report
open results/report.html

# View JSON results
cat results/report.json

Step 5: Upload to Leaderboard (Optional)

To submit a report after the run:

export BLXBENCH_API_KEY=your-key
blxbench --headless --provider opr --models openai/gpt-5.4-mini --submit

Uploads require an account, a BLXBench API key, and a paid pass tier that includes submission quota (see Account and Pass / pricing).

Common Options

Next Steps

• blxbench Reference — All available commands
• Understanding Results — How to read the results