Agent Harness Best Practices: 10 Rules for Production Agents

The model is rarely the hardest part anymore. Frontier models are capable enough that, for many agent workloads, switching between them can be as simple as changing a configuration field. What separates an agent that works in a demo from one that still works reliably three months into production is everything around the model: the loop, the context, the tools, the sandbox, the approvals, and the controls that stop it from doing something you can't undo.

That layer is the agent harness, and the decisions you make there can have a bigger impact on reliability and cost than the model choice itself. We learned that while building and running TrueForge, our open-source agent harness. These are the 10 agent harness best practices we'd recommend if you're building production agents today.

Why the Harness Decides Your Bill

Start with a number, because it reframes everything below.

Accuracy is effectively tied. The model sets the ceiling; the harness doesn't make the underlying model smarter.

What the harness does determine is how much work it takes to reach that ceiling.

In this benchmark, TrueForge solved roughly the same number of tasks as Claude Managed Agents while using about 62% fewer tokens and costing about 28% less per run. Compared with deepagents, the token difference was even larger.

That's the part of agent infrastructure that is easy to overlook. Two agents can use the same model, have access to the same tools, and attempt the same task, yet produce very different execution costs.

The difference comes from what happens around the model: how much context it carries, which tools it loads, how it handles large responses, when it compacts state, and how efficiently it executes the loop.

Read the full benchmark methodology →

Almost all of the cost gap in this benchmark comes back to the first four practices.

TrueForge: Open-Source Agent Harness

TrueForge is TrueFoundry's open-source, vendor-neutral agent harness for teams that want more control over how their agents run in production. Instead of tying the runtime to a single model provider, TrueForge lets you bring your own models, MCP servers, and infrastructure while handling the agent loop, tool execution, context management, approvals, and sandboxing.

Where a self-hosted agent like Hermes leaves much of the production infrastructure to the developer, TrueForge provides the surrounding control plane needed to operate agents across teams and environments. This includes centralized MCP access and credentials, human-in-the-loop approvals, sandboxed execution, observability, and governance.

You can run it locally with npx @truefoundry/trueforge or deploy it for a team with Docker Compose or Helm.

TrueForge also separates the agent runtime from the model layer. This means teams can switch models or route workloads to different providers without rebuilding the agent itself.Combined with TrueFoundry's AI Gateway, teams can also apply model routing and cost controls to avoid using expensive frontier models for tasks that don't require them. The result is a middle ground between a fully managed runtime and building the entire agent infrastructure yourself: an open-source agent harness with the operational controls needed to run production agents at scale.‍

TrueFoundry also provides a second lever for reducing model costs through AI Gateway's Auto Routing. Instead of sending every request to a frontier model, Auto Routing classifies requests by complexity and routes them to different model tiers. In TrueFoundry's benchmark across 550 prompts, routing between Haiku, Sonnet, and Opus reduced cost by 69% while retaining 98% of baseline quality, and mean latency dropped from 7.6 seconds to 4.0 seconds. On production-shaped traffic, the overall cost reduction reached 80%.

These two layers address different parts of the cost equation. TrueForge reduces the overhead of running the agent itself, while model routing lets you avoid using an expensive model when the task doesn't require one. For teams running agents at scale, having both controls can matter more than the price of the model alone.

TrueForge is open source and MIT licensed, while TrueFoundry's AI Gateway adds the operational layer teams need when moving from individual agents to production: centralized model access, budgets, guardrails, credential management, and unified traces.

If you're comparing Claude Managed Agents with a more configurable architecture, TrueFoundry is worth considering when model flexibility, infrastructure control, and agent cost optimization matter alongside the agent runtime itself.

An Open-Source Alternative to Claude Managed Agents, Up to 75% Cheaper

Run production-ready AI agents on your own infrastructure with TrueForge, using your choice of models, tools, and sandbox providers.

View on GitHub

Docs

Context Engineering Best Practices

This is where the money is. Context engineering is the discipline of deciding what the model sees on each turn, and on a long-running agent it compounds turn after turn.

1. Treat the context window as a budget, not a buffer

What goes wrong: you connect five MCP servers, every tool schema loads at startup, and the agent burns tens of thousands of tokens on definitions for tools it will never call. Every subsequent turn carries them.

What to do: load tool schemas on demand rather than upfront. Good agent context management starts before the first model call, not after the window fills.

In TrueForge: deferred tool loading gives the agent a lightweight index and fetches the full schema only when it reaches for that tool.

2. Compact, don't truncate

What goes wrong: the window fills, the harness drops the oldest content, and the agent quietly forgets the constraint you gave it on turn three. It doesn't error. It just gets worse, and the failure is hard to spot because the output still looks plausible.

What to do: summarize old history into a structured record and keep the summary. Set the threshold below the wall so compaction happens on your terms rather than under pressure.

In TrueForge: compaction fires at 80% of the model's context length and replaces old history with a structured summary. Truncation is fine for a short support agent and actively harmful on a multi-hour task.

One caveat worth knowing: compaction carries its own risk if policy and constraints live only in the conversation. We wrote about how compaction erodes governance and why enforcement belongs outside the context window.

3. Offload large tool responses instead of inlining them

What goes wrong: a tool returns a 200KB JSON payload, all of it lands in the context window, and the agent needed three fields from it.

What to do: write the payload somewhere the agent can reach and hand it a reference instead of the contents.

In TrueForge: large tool response offloading writes the response to a sandbox file and replaces it with a path and a preview. Code Mode goes further, letting the agent chain several tool calls inside one sandbox script so only the printed result enters context. Five round trips become one summary.

4. Give subagents clean context and take back only the result

What goes wrong: a subagent inherits the full parent thread, does its work, and returns its entire reasoning trace into the parent context. You've now paid for the same history twice and polluted the main thread.

What to do: delegation should shrink the parent's context, not grow it.

In TrueForge: subagents run with their own clean context and hand back only the result.

Try it: npx @truefoundry/trueforge runs a harness with all four of these on by default, locally, in about a minute. Star it on GitHub.

Execution and Safety

5. Make the sandbox a tool, not a wrapper

What goes wrong: the harness wraps every session in a container, whether or not the agent ever runs code. You pay container time for conversations that were only ever text, and one server holds far fewer concurrent agents.

What to do: provision the sandbox when the agent actually needs to execute something. Agent sandboxing is about blast radius, and you get the same isolation without paying for it on every turn.

In TrueForge: the sandbox is a tool, started only when the agent needs to run code, so one server handles many agents and turns that never touch code stay cheap. Sandbox providers are bring-your-own.

The deeper argument for letting models execute without letting them roam is worth reading if you're designing this from scratch.

6. Gate the actions you can't undo, and only those

What goes wrong: teams approach approvals at one of two extremes. Either nothing is gated and an agent eventually deletes something, or everything is gated and a human clicks approve forty times an hour until they stop reading.

What to do: gate on irreversibility. Sending an email, writing to production, spending money, deleting anything. Reads and drafts run free. And don't implement a compliance requirement by describing it in the system prompt and hoping, because a prompt is a suggestion where guardrails that inspect every tool call are a control.

In TrueForge: the core server pauses the run and waits for approval on sensitive actions before the call goes out, rather than after.

Where that gate lives matters as much as whether it exists, which we covered in designing MCP tool approvals at the gateway boundary.

7. Make sessions survive reconnects

What goes wrong: a laptop sleeps, a load balancer recycles a connection, and forty minutes of agent work is gone. On tasks that run for an hour, this stops being an edge case.

What to do: persist session state so a dropped connection resumes rather than restarts.

In TrueForge: sessions persist across reconnects and restarts. Resumability is one of those properties nobody asks for in the design review and everybody needs by week two.

Operations and Governance

8. Instrument the loop, not just the model

What goes wrong: you have token counts and latency per model call, and no idea which step of a 60-step run went sideways. Debugging becomes archaeology.

What to do: emit structured events for every step, so a run reads as a timeline rather than a wall of logs. Check that the loop itself is observable before you're debugging it in anger.

In TrueForge: every step streams as it happens, and the agent events runtime contract is published so your own tooling can consume it.

The framing in loop engineering is useful here: the loop is the middleware, so treat it like infrastructure.

9. Keep the model swappable

What goes wrong: the agent is written against one provider's SDK and quirks. A cheaper or better model ships and switching means a rewrite, so you don't switch.

What to do: talk to models through OpenAI-compatible interfaces and keep provider specifics out of your agent logic.

In TrueForge: models, MCP servers and sandbox providers are all bring-your-own through open interfaces. This is not hypothetical value. In the same benchmark, running TrueForge on GLM-5.2 instead of Opus 4.8 solved the same ~11 of 14 tasks at $2.90 per run against $8.50. That saving is only available if switching is a config change.

10. Centralize governance once you're past a handful of agents

What goes wrong: every team holds its own model keys and MCP credentials. Nobody can cap spend, PII masking is inconsistent, and answering an audit question means stitching together logs from nine services.

What to do: put a gateway between your agents and everything they call.

In TrueForge: deliberately not the harness's job. Self-hosted, you bring your own keys and manage them yourself, which works until you're running a lot of agents. At that point point it at TrueFoundry's AI Gateway, which puts 1,000+ LLMs behind one OpenAI-compatible API at roughly 3 to 4 ms of added latency and 350+ RPS on a single vCPU, with RBAC, budgets, guardrails, credential rotation and OpenTelemetry traces into Grafana, Datadog or Prometheus.

Running more than a few agents? See how the AI Gateway governs them centrally, or book a walkthrough if you want it mapped to your setup.

How TrueForge Handles These

Nine of ten are in the harness. The tenth is deliberately not, because a harness that also wants to be your control plane tends to be bad at both.

Four Anti-Patterns

Rebuilding a loop that already exists. If you are writing plan-act-observe from scratch, check whether a harness already ships it. Agent harness design is worth your time only where your problem is genuinely unusual.

Prompting your way to a control. Compliance steps, spend caps and approval gates belong in code or at a gateway. A model that can be asked nicely can be talked out of it.

Sandboxing everything. Isolation on every turn costs real money and buys nothing on turns that never execute code.

Measuring the model instead of the run. Per-call token counts look fine while total cost per completed task quietly triples. Measure the run.

FAQ

Q: What are agent harness best practices?

A: The ones that matter most are context practices: load tool schemas on demand, compact history rather than truncating it, offload large tool responses to files, and keep subagent context separate from the parent. After that, provision sandboxes only when code runs, gate irreversible actions, persist sessions across reconnects, and emit structured events for every step. In our benchmark these decisions accounted for a more than 4x difference in tokens per run on identical tasks.

Q: How do you stop an agent running out of context on a long task?

A: Compaction, not truncation. Summarize older history into a structured record at a threshold, around 80% of the model's context length, so the agent keeps the thread instead of silently losing it. Pair that with deferred tool loading and response offloading so the window fills more slowly in the first place.

Q: Should you sandbox every agent session?

A: No. Sandbox the code execution, not the session. Holding a container open for a whole conversation costs money on every turn that never runs code, and it limits how many agents one server can handle. Provisioning on demand gives you the same isolation at a fraction of the cost.

Q: Can I run an agent harness in my own VPC or on-prem?

A: Yes, with any self-hosted option. TrueForge runs from a single npx command locally, or via Docker Compose and Helm for a team deployment with Postgres, Redis, replicas and OIDC login. TrueFoundry's managed platform also runs self-hosted, on-prem, air-gapped or hybrid, so no data leaves your domain.

Q: Does TrueFoundry support MCP and agents from other frameworks?

A: Yes. The platform includes an MCP Gateway, Agent Gateway and a registry with tool-level access control. Agents built on LangGraph, CrewAI, AutoGen or a custom framework can be governed through the same layer, so these practices don't require you to adopt one stack.

Related reading

• Agent Harness vs Agent Framework: What's Actually Different?: the category distinction behind all of this
• Context Engineering: Designing What Your AI Agent Sees: the deeper version of practices 1 to 4
• Agent Sandboxes, Explained: why the sandbox is a tool, not a wrapper
• Loop Engineering: The Loop Is the New Middleware: treating the runtime as infrastructure
• TrueForge vs Claude Managed Agents: the full Enterprise-Bench methodology and results

Conclusion

Agent harness best practices come down to one habit: decide deliberately what the model sees and what it is allowed to do, then measure the whole run rather than the individual call. Teams that do this end up with agents that cost a quarter as much and fail in ways they can diagnose.

You don't have to take our word for the numbers. TrueForge is MIT-licensed and the benchmark methodology is published, so you can run it against your own tasks.

npx @truefoundry/trueforge to try it in about a minute, read the docs, or book a walkthrough if you'd rather see it applied to your stack.

‍