16.5 — Project: Agent Platform (Autonomous Coding Agent)

This project builds the coding-agent platform designed in Part 11.7 into a real, runnable system. Read Part 11.7 first.

1. Customer Scenario

A developer-tools company (Part 11.7's scenario) needs to let customer teams deploy AI agents that investigate a codebase, propose fixes, run tests, and open pull requests — genuinely open-ended technical work, requiring real agent autonomy (Part 3.7) rather than a fixed workflow.

2. Requirements

Recap of Part 11.7: bounded agent autonomy for genuinely unpredictable investigation/fix tasks; strong sandboxing for arbitrary code execution; PR-only output (never direct commits); full trajectory tracing for human PR reviewers.

3. Architecture

Part 11.7's architecture: a bounded agent loop (Part 3.7) with code-aware retrieval, sandboxed test execution (Part 9.6's strongest isolation tier), and a PR-creation tool as the sole output mechanism.

4. Technology Selection

• create_agent with middleware (Part 4.1) built on LangGraph, given this is the clearest "agent justified" case in this entire handbook.
• Firecracker microVMs or an equivalent strong-isolation sandbox (Part 9.6) for test execution — the highest-risk code-execution scenario in this book, warranting the strongest available isolation tier.
• Code-aware chunking (Part 3.5, respecting function/class boundaries) for repository retrieval.

5. Folder Structure

agent-platform/
├── src/
│   ├── agent/
│   │   ├── loop.py                 # bounded agent loop (Part 3.7), max_iterations enforced
│   │   └── prompts.py              # investigation/planning prompts (Part 3.1)
│   ├── retrieval/
│   │   └── code_chunker.py         # code-aware chunking (Part 3.5)
│   ├── sandbox/
│   │   ├── microvm_executor.py     # Part 9.6's strongest isolation tier
│   │   └── resource_limits.py      # CPU/memory/time caps per execution
│   ├── tools/
│   │   ├── run_tests.py            # sandboxed test execution
│   │   └── create_pull_request.py  # THE ONLY write path — never direct commit
│   └── tracing/
│       └── trajectory_logger.py    # full reasoning trace for PR reviewers (Part 6.1)
├── tests/
│   ├── unit/
│   ├── security/
│   │   └── test_sandbox_escape_resistance.py  # adversarial sandbox testing (Part 8.5)
│   └── eval/
│       └── trajectory_evaluation.py  # Part 8.1's trajectory, not just outcome, evaluation
├── infra/
│   └── microvm-provisioning/       # per-run disposable execution environment provisioning
└── requirements.txt

6. Implementation (key excerpt)

The PR-only output enforcement — the single most important design decision in this entire project, directly implementing Part 11.7/9.2's structural human-in-the-loop principle:

class CreatePullRequestTool:
    """The agent's ONLY write-capable tool — structurally impossible to
    bypass into a direct commit, regardless of agent reasoning."""

    async def create_pull_request(self, repo: str, branch_name: str, changes: dict, description: str) -> dict:
        """Creates a pull request. There is deliberately NO tool in this
        codebase that commits directly to a protected branch."""
        if branch_name in PROTECTED_BRANCHES:
            raise PermissionError("cannot target a protected branch directly — this tool only creates PRs")

        pr = await git_provider.create_pull_request(
            repo=repo, branch_name=branch_name, changes=changes,
            description=description, base_branch=DEFAULT_BASE_BRANCH,
        )
        return {"pr_url": pr.url, "status": "created_for_review"}

7. Testing

test_sandbox_escape_resistance.py runs a deliberate red-team suite (Part 8.5) attempting known sandbox-escape patterns against the microVM executor — this is exactly the kind of proactive, adversarial testing Part 8.5 argued normal evaluation traffic wouldn't surface.

8. Evaluation

Trajectory evaluation (Part 8.1), not just outcome evaluation — checking whether the agent's investigation and fix process was sound, not just whether tests eventually passed, given Part 8.1's "correct by luck" warning applies directly to a coding agent that could pass tests via an accidentally-correct but poorly-reasoned change.

9. Security

This project sits at Part 9.2's highest excessive-agency-risk tier — strongest available sandboxing (Part 9.6), a structurally non-bypassable PR-only output path, and per-repository/per-customer tool-access scoping (Part 9.3/9.6) preventing any cross-customer code or credential exposure.

10. Observability

Full trajectory tracing (Part 6.1) is a functional product requirement here — the human reviewer's ability to understand the agent's reasoning behind each proposed change is core to the product's usability, not optional debugging infrastructure.

11. Deployment

Each agent run gets its own disposable, isolated microVM execution environment (Part 9.6), provisioned per-run and torn down after — never a shared, persistent execution environment across runs or customers.

12. Scaling

Agent runs parallelize naturally across customers/tasks (Part 5.6) — the binding scaling constraint is likely the sandboxed execution environment provisioning capacity, not the agent's own LLM-calling logic.

13. Cost Considerations

Agent runs are meaningfully more expensive per-task than a single call (Part 3.7/7.10) given potentially many iterations — cost-per-completed-task should directly inform this platform's own pricing model (Part 15).

14. Failure Scenarios

Agent stuck in an unproductive loop → bounded by the hard iteration cap (Part 3.7), opening a PR with partial progress and an explicit note rather than looping indefinitely. Sandboxed execution attempts something genuinely malicious → fully contained within the microVM's isolation boundary (Part 9.6) — this is precisely the failure mode the strongest isolation tier exists to contain.

15. Improvements

A hybrid future evolution: a fixed, templated workflow for common, well-understood bug-fix patterns (cheaper, more predictable) alongside the full agent for genuinely novel tasks (Part 11.7, section 14) — pursued once real usage data reveals which task types are actually predictable enough to warrant it.

16. Business Metrics

PR acceptance rate (how often a human reviewer merges the agent's proposed change with no or minor modification) as the primary quality/value metric, alongside average developer time saved per completed task (Part 15's time-savings ROI pattern).

---

Key takeaways

• This project is the clearest instance in the entire handbook of Part 3.7's "agent justified" criterion — genuinely unpredictable, discoverable-only-during-execution task structure that no fixed workflow could capture.
• The PR-only output tool is a structural, code-enforced safety boundary, not a convention the agent is merely instructed to follow — exactly the distinction Part 9.1's core lesson insists on.
• Trajectory evaluation, not outcome evaluation alone, is essential here given how easily a coding agent could pass tests via an accidentally-correct but unsound process.

Recommended next chapter

06-enterprise-knowledge-assistant.md