Evaluation Methodology
This suite evaluates how AI models and agent harnesses generate production-grade single-file web interfaces. All results are deterministic, reproducible, and scored for web accessibility compliance.
1. Strict Single-File Constraint
Every evaluation prompt given to an AI model explicitly mandates a single self-contained index.html file. No build steps, no unbundled React or Vue compilers, and no backend servers.
- Tailwind CSS via official script CDN (
cdn.tailwindcss.com) - Icon packs via CDN (Lucide Icons, FontAwesome 6)
- Standalone CDN graphics libraries (Chart.js, D3.js, Google Fonts)
- Vanilla JavaScript for inline interactive state management
2. How Results Are Produced
Each result is a single self-contained HTML file generated by a model and recorded under a versioned folder tree (Git is the database). Desktop and mobile screenshots plus accessibility scores (axe-core) are attached where available; use Open on any sandbox to view that result in a dedicated browser tab without the suite panel around it.
- Zero-shot generation: Outputs are captured on the first generation with no human retries, edits, or agent loops.
- No skills, plugins, or extra tooling layered on top of the bare prompt.
3. Leaderboard Ordering
The leaderboard's Adjusted Score column ranks models
by a sample-size-adjusted score so that a single scored run cannot outrank a well-tested
model. The adjusted score is (n·avg + 4·μ)/(n + 4),
where n is the number of scored runs for that
model, avg is its raw average, and
μ is the global mean across all scored runs.
Each row shows the adjusted score with the raw average and run count in the microtext
beneath, so a lucky one-run score is clearly pulled toward the suite average.