WebArena
Benchmark for autonomous web agents on realistic self-hosted sites
About
WebArena is a benchmark for evaluating autonomous web agents, introduced by Zhou et al. at CMU in 2023. It provides 812 tasks across four fully functional, self-hosted website domains (e-commerce shopping, a Reddit-style forum, GitLab, and a CMS) plus map and Wikipedia tools. Evaluation is execution-based, with programmatic validators checking functional correctness; the original GPT-4-based agent managed only 14.41% end-to-end success against a 78.24% human baseline.
It has become the standard web-agent benchmark, underlying VisualWebArena, WebChoreArena, and the BrowserGym ecosystem. The repo is open source under Apache-2.0 with a Gym-style interface supporting accessibility-tree and screenshot observation modes, human trajectory recordings for about 170 tasks, and a public leaderboard.
What it can do
- 812 realistic web tasks across 4 self-hosted domains
- Execution-based evaluation with programmatic validators
- Accessibility-tree and screenshot observation modes
- Gym-style environment interface
- Human trajectory recordings for about 170 tasks
- Public leaderboard
Setup
- Clone the repo: git clone https://github.com/web-arena-x/webarena
- Follow the README to launch the self-hosted site containers with Docker
- Install the agent harness dependencies
- Point your web agent at the local environments and run the task suite
Prerequisites
Docker for the self-hosted sites, Python 3.8+, and a web agent to evaluate.
Use cases
Works with
Trust score — 7/10
Scored 7/10 (TIER verified, metrics-v1). Evidence: 1,630 GitHub stars (api.github.com, 2026-10-16); Apache-2.0; peer-reviewed paper (Zhou et al., arXiv 2307.13854); the most-cited web-agent benchmark, underlying VisualWebArena and BrowserGym. Strengths: reproducible self-hosted design, strong research adoption. Docked for: (1) no commits since 2025-11-26, so maintenance is quiet; (2) heavyweight setup (Docker, self-hosted sites) limits casual use. License: Apache-2.0.
★ 1,630 GitHub stars
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.0/10
- Maintenance · recency, releases, issue load
- 5.0/10
- Openness · license + docs
- 9.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · 1,630 GitHub stars via api.github.com/repos/web-arena-x/webarena (2026-10-16)
- · Last push 2025-11-26T21:16:00Z - quiet for ~11 months
- · Peer-reviewed: Zhou et al., arXiv 2307.13854
- · Foundation for VisualWebArena, WebChoreArena, BrowserGym; public leaderboard at webarena.dev
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.