SWE-smith

SWE-bench proved that agents can fix real bugs, but its ~2,300 hand-filtered instances are far too few to train on. SWE-smith fixes the data problem by manufacturing it: given any Python codebase, it automatically injects bugs — LM-generated errant rewrites, procedural AST modifications, undone PRs, and bug combinations — then runs the test suite to keep only the bugs that actually break tests. The result is 50,000+ task instances across 128 real repositories, each with a complete test setup and LM-crafted issue text.

This is the toolkit behind a wave of SWE-agent training work: the paper's authors fine-tuned Qwen 2.5 Coder on SWE-smith data (with Claude-generated expert trajectories) to a new open-weight state of the art on SWE-bench Verified, and Meta's SWE-RL and DeepSWE pipelines both consume SWE-smith data. It is MIT-licensed and maintained by the SWE-bench team.

The honest caveat: synthesized bugs are not real bugs — models trained on them can overfit to bug-shaped problems and underperform on genuine issue reports. It is also Python-only and compute-hungry (environment setup per repo). Use it for what it is: the best available way to get training-scale SWE data, not a substitute for real-world evaluation.

1. Clone https://github.com/SWE-bench/SWE-smith and install dependencies (Python, Docker for execution environments).
2. Point the pipeline at a Python repository; SWE-smith sets up the SWE-agent execution environment automatically.
3. Choose bug-generation strategies (LM rewrite, procedural AST edits, PR mirrors, combinations) and difficulty.
4. Generate task instances with tests and issue text; export the dataset or expert trajectories for training.