Generate, run, and verify synthetic RL worlds over HTTP.

Authentication

Anonymous access works for exploration (10 gen/day, 60 calls/hour). For production use, create a free API key via POST /api/v1/auth/keys and pass it as X-API-Key header or Authorization: Bearer bf_... header. Free tier: 10 gen/day, 100 calls/hour. Worlds created with a key are tracked and retrievable via ?mine=true.

LLM-Optional

Generation is prompt-driven and LLM-optional: with a provider key the domain model is richer, otherwise a deterministic heuristic pipeline still produces a runnable world.

Persistence

Worlds persist per server instance. Set BLOBFISH_SANDBOX_DIR to a durable mount for cross-instance access. Worlds are also downloadable as Harbor archives.

Verification

Every task is scored by executable VCode over real before/after SQLite state. No LLM judge in the reward path. Verifiers use deterministic state-diff assertions.

Honesty Policy

The API honestly reports its capabilities. Benchmarks with no in-repo harness return null. RunPod returns not_configured when unavailable. GRPO regressions are disclosed.

Rate Limits

Anonymous: 10 gen/day, 60 API calls/hour. Free key: 10 gen/day, 100 calls/hour. Pro: 100/1000. Enterprise: 1000/10000. Exceeded? 429 with Retry-After header.

Error Responses

All errors return JSON with an error field. Common: 404 world not found,429 rate limit exceeded,400 missing required fields,500 generation timeout or VCode execution error.

Cookbook: Attach your production agent over HTTP

Create a key, fork an isolated session for the mirrored sandbox world, then point any MCP-over-HTTP client at the returned mcp_url. Pass both the API key andX-Blobfish-Session on every call so repeated or parallel runs do not share state. For assessment and CI runs, also pass X-Blobfish-Task,X-Blobfish-Run, and X-Request-Id so the report can attribute every tool call to a task.

• MCP over HTTP: use the returned mcp_url.
• Blobfish JSON transport: use the returned tool_calls_url.
• OpenAI-compatible agent endpoint: use blobfish eval --policy agent.
• Vendor/internal REST base-URL swapping: use tools/vendor_rest_shim with a route map.

# 1. Create an API key
curl -sX POST https://blobfish.ai/api/v1/auth/keys \
  -H 'Content-Type: application/json' \
  -d '{"name":"ci-agent","email":"agent@example.com"}'

# 2. Create a session for one sandbox world
curl -sX POST https://blobfish.ai/api/v1/sandbox/worlds/sbx_9f3aa4bc6f684ff3/sessions \
  -H 'X-API-Key: bf_YOUR_KEY'
# → { "session_id": "sess_...", "mcp_url": ".../mcp", "tool_calls_url": ".../tool-calls" }

# 3. Use HTTP MCP against the session fork
curl -sX POST https://blobfish.ai/api/v1/sandbox/worlds/sbx_9f3aa4bc6f684ff3/mcp \
  -H 'X-API-Key: bf_YOUR_KEY' \
  -H 'X-Blobfish-Session: sess_...' \
  -H 'X-Blobfish-Task: task_...' \
  -H 'X-Blobfish-Run: ci-2026-07-08' \
  -H 'X-Blobfish-Run-Mode: ci' \
  -H 'X-Request-Id: req_001' \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}'

Reset between runs with POST /api/v1/sandbox/worlds/<world_id>/sessions/<session_id>/reset. For REST-only agents, run python tools/vendor_rest_shim/vendor_rest_shim.py --config route-map.jsonand point the vendor base URL at the shim.

Cookbook: Regression-test your agent in CI

Use the Blobfish CLI to gate PRs on agent quality. Three commands:

# 1. Export your world's tasks as an eval dataset
blobfish export-eval ./my_world --format jsonl
# → eval.jsonl: {task_id, split, difficulty, instruction, expected_calls, verifier_ref}

# 2. Grade your agent (or use oracle/random baselines)
blobfish eval ./my_world --policy agent --agent-endpoint http://localhost:3000/api/agent --run v1

# 3. Gate on regressions (exit 1 = fail the build)
blobfish ci-gate ./my_world \
  --policy agent \
  --agent-endpoint http://localhost:3000/api/agent \
  --hosted-world-url https://blobfish.ai/api/v1/sandbox/worlds/sbx_9f3aa4bc6f684ff3 \
  --api-key bf_YOUR_KEY \
  --min-pass-rate 0.7 \
  --no-regressions \
  --baseline baseline.json \
  --write-baseline current-baseline.json \
  --langfuse-dataset blobfish-pr \
  --langfuse-run "$GITHUB_SHA"

Optional: push eval datasets to Langfuse with --format langfuse (requiresLANGFUSE_PUBLIC_KEY / LANGFUSE_SECRET_KEY). Omit --hosted-world-url to run the same agent loop against the local CLI harness instead of hosted sandbox sessions. Use the Blobfish download package workflow for moving a generated world into the hosted sandbox, and app-mirror for cloning your production app into a testable mock world.

Cookbook: Cheaper-model ROI POC

Use validated customer traces for SFT first, then serve the adapter, run A/B eval, and compute cost from measured usage. Training result fields stay nulluntil a real eval measures them.

blobfish distill-data \
  --traces ./traces.jsonl \
  --world ./my_world \
  --out ./sft.jsonl \
  --validate replay \
  --report-out ./distill_report.json

blobfish train \
  --world ./my_world \
  --data ./sft.jsonl \
  --base Qwen/Qwen3-8B \
  --method sft \
  --target local-mlx \
  --out ./output/train

blobfish serve-adapter \
  --adapter ./output/train/adapter \
  --base Qwen/Qwen3-8B \
  --target local \
  --registry-url https://model-registry.example.com

blobfish eval-ab \
  --world ./my_world \
  --baseline-run frontier \
  --agent-endpoint http://127.0.0.1:8000/v1/chat/completions

blobfish cost-report \
  --world ./my_world \
  --baseline-run frontier \
  --eval-run agent-... \
  --frontier-price 5.00 \
  --tuned-serving-cost 0.30 \
  --training-cost 25.00 \
  --usage-jsonl ./usage.jsonl