What if an RL environment could be synthesized like data?
We turned 100 seed domain and company types into 2,000 executable environments, then trained Qwen3 models across 748 of them with standard GRPO. The distribution of worlds—not a larger simulator prompt—became the training asset.
Aggregate research result · not a score or lift claim for a world generated on this page
Environment factoryone pipeline · widening distribution
Seed types100domain × company
Generated2,000executable worlds
Trained on748worlds · GRPO
Executable by constructionDocker image · MCP server · SQLite state · deterministic verifier
What the pipeline produced
One world is software, not a conversation.
Every environment ships as a Docker image with runnable MCP tools, a seeded SQLite database, generated tasks, and executable grading. Each stage is sandbox-tested and self-corrected until the code runs.
37.2MCP tools / worldCompilable, code-based tools
20.7SQLite tables / worldStateful operational data
~1.3×corrections / stageSandbox-tested and repaired until runnable
1,000turn task horizonIdentical replay trajectories across mock Salesforce + Office Suite MCPs
0% / 100%pass bands removedNo-signal tasks filtered from the curriculum
$57per worldFor 100 long-horizon task samples
Held-out results
More worlds. Better tool use.
Standard GRPO
Benchmark
Model
Base
After
Δ
BFCLv3Qwen3-8B
Qwen3-8B
53.8
68.1
+14.3
BFCLv3Qwen3-14B
Qwen3-14B
61.3
72.4
+11.1
MCP-UniverseQwen3-8B
Qwen3-8B
6.7
11.2
+4.5
MCP-UniverseQwen3-14B
Qwen3-14B
8.4
14.5
+6.1
τ²-Bench Pass@1Qwen3-8B
Qwen3-8B
26.4
35.7
+9.3
Aggregate held-out evaluations from the 748-world training study. Scores are percentage points; generating a sandbox world does not itself train a model or guarantee improvement.
Scaling thesis
Environments are the new data.
Scale the executable situations a model can experience—not just the text it can imitate.
Try the environment factory
Build a world your agent can fail in.
Turn a business workflow brief into inspectable software: deterministic fictional data, callable tools, multi-step tasks, and deterministic verifiers.
No API key, company data, or upload required. Run a verified example instantly; a fresh build takes several minutes and creates synthetic fictional records.
Synthetic-data trust contract
No account, upload, or API key required. Generated company operations use fictional records; shared proofs may replay labeled public sources. Your prompt and generated world are stored, so do not enter confidential, client, or production data.
Operational records are fictional and isolated. Generated company and customer rows are not imported from your systems. A shared proof can bundle a source-receipted replay of public records; each such asset is labeled and is not a live vendor account. Public examples open read-only and do not silently create a private copy.
A visible seed ID identifies the generated baseline. Each world records its generation seed on the world page. Its saved seed database is the immutable baseline restored before every task run.
Task runs reset to the declared seed first. Every task run restores the world's SQLite state to its seed before executing, so runs are reproducible episodes. Free-form chat is the one deliberately stateful surface.
No automatic expiry — use non-confidential prompts. Generated worlds persist. Anonymous users cannot self-delete them today; authenticated API owners can delete. Temporary per-world API sessions are cleaned up after 24 hours.
Get proof before you buildRun a verified example with no setup, or use its brief to build a fresh fictional company. Verified means executable state assertions passed; it does not mean live integrations or customer data.
Litigation companyMarketing, sales, sourcing, matters, discovery, and filings
Insurance claimsIntake, fraud review, reserves, approval, and payout
Commerce operationsOrders, inventory exceptions, returns, and support
Stateful dataFictional operations plus explicitly labeled public-source replays
Executable toolsLocal fixture, bounded replay, local twin, or remote call — labeled per tool
Graded tasksMulti-step work with pinned outcomes
Hard verifiersState diffs, not an LLM judge
Published research · source locked
The benchmark case for executable worlds.
Agent-World turns real system sources into state, tested tools, grounded long-horizon tasks, and executable rewards. These are Dong et al.’s reported results for that recipe—not scores from a world created here.
Agent-World-14B · two evolution rounds
MCP-Mark
29.538.1
Round 0 → round 2 · +8.6 points
The diagnosis arena finds failed interaction patterns, synthesizes targeted worlds, then continues RL.
Source lock: arXiv v1 reports 29.5, not the circulated 25.3 starting score.
Paper evidence, product receipt, and this world stay separate.
Blobfish’s current recorded reproduction is partial: Qwen3-14B has not been run, MCP-Mark improvement has not been demonstrated, and a Sandbox-created world has not yet been isolated as the causal training corpus.
One published world produced one cross-world verifier pass.
We trained the same small model on a six-world control and on those exact bytes plus 11 verified traces from Gold Harbor, then scored both arms on the same 14 old-world tasks with executable state verifiers.
Live production bundle ingested255 tools · 194/194 oracle traces replayed on fresh state
0/14 → 1/14 on the fixed subsetQwen2.5-0.5B · eight epochs · same task-scoped evaluator
≠Directional mechanism onlyOne seed, 14 tasks; rows 44→55 and assistant turns 239→382. The paper-scale 18.4→38.5 result is still unproven by Blobfish.
The passing episode was on the original hotel world; the added litigation world scored 0/58. That is cross-world transfer, but environment count, data volume, and optimizer work co-vary, so causality is not isolated.
Blobfish Sandbox