Overview
Middleware is the safety layer between the LLM and the real world. While the LLM proposes actions, middleware deterministically enforces rules, catches errors, and enforces budgets. Open SWE's middleware system is one of its most important architectural decisions — and one we are extending significantly for xCoder.
Hook Types
Agent middleware operates at multiple interception points:
| Hook | When It Runs | Use Cases |
|---|---|---|
| before_model | Before each LLM call | Inject messages, check budgets, enforce step limits |
| wrap_model_call | Around LLM invocation | Fallback models, retry logic, response sanitization |
| wrap_tool_call | Around tool execution | Error handling, sandbox recovery, input sanitization |
| before_tool_call | Before tool execution | Phase gates, policy enforcement, allow/deny decisions |
| after_tool_call | After tool execution | Activity logging, cost tracking, result observation |
| after_agent | After agent loop completes | Notifications, cleanup, circuit breaker checks |
Open SWE Middleware
Open SWE ships with a curated set of middleware:
- ToolErrorMiddleware — Catches exceptions and converts them to error ToolMessages so the LLM can self-correct.
- ModelFallbackMiddleware — Retries with a fallback model on transient errors (5xx, 429, timeouts).
- SandboxCircuitBreakerMiddleware — Stops runs after repeated sandbox failures to prevent infinite loops.
- SanitizeToolInputsMiddleware — Cleans tool inputs to prevent injection attacks.
- check_message_queue_before_model — Injects follow-up messages (Slack/Linear) before the next model call.
- notify_step_limit_reached — Posts to Slack when the agent hits its step limit.
xCoder Extensions
xCoder adds several middleware stacks that Open SWE does not have:
- PhaseGate — Blocks tool calls that are not allowed in the current FlowEngine phase. E.g., you cannot call
git commitduring thespecphase. - PolicyGate — Enforces team policies: branch naming conventions, required tests, commit message formats, PR templates.
- ActivityLogger — Durable logging of every tool call, model response, and phase transition to PostgreSQL.
- CostTracker — Per-session cost tracking and budget enforcement with real-time alerts.
- ApprovalGate — Requires human approval for high-impact operations (production deploys, schema changes, auth modifications).
- SecurityScanner — Scans tool inputs and outputs for secrets, SQL injection, and other security issues.
Implementation Patterns
Middleware can be implemented in multiple languages depending on performance requirements:
| Language | Best For | Example |
|---|---|---|
| TypeScript | Business logic, orchestration, async I/O | PhaseGate, ActivityLogger |
| Rust (WASM) | Hot-path policy enforcement, deterministic checks | PolicyGate, SecurityScanner |
| Rust (native) | High-throughput tool execution, sandbox control | CostTracker (stream processing) |
Middleware Ordering
Order matters. xCoder uses this default stack (outermost first):
- SecurityScanner (catches malicious input first)
- ApprovalGate (human approval for dangerous ops)
- PhaseGate (enforce flow discipline)
- PolicyGate (team policy enforcement)
- CostTracker (budget enforcement)
- ModelFallbackMiddleware (model resilience)
- ToolErrorMiddleware (error normalization)
- ActivityLogger (audit trail)
- SandboxCircuitBreaker (stop dead sandboxes)
Design principle
Middleware should be composable and order-independent where possible. Each middleware is a pure function of (state, request) → (result, modified_state). This makes testing deterministic and enables middleware reuse across different agent configurations.