Sandboxed Code Agents: Let Models Execute Without Letting Them Roam

Built for Speed: ~10ms Latency, Even Under Load
Blazingly fast way to build, track and deploy your models!
- Handles 350+ RPS on just 1 vCPU — no tuning needed
- Production-ready with full enterprise support
Giving an AI model code execution is one of the highest-leverage upgrades you can make. Instead of reading 100,000 rows into context, the agent can write code that computes over them. Instead of asking a model to mentally transform files, it can run a parser. Instead of returning a vague suggestion, it can build an artifact and test it.
It is also the moment an AI system stops being “just text.” Code can read files, open sockets, consume CPU, fork processes, inspect environment variables, and mutate whatever the runtime can reach. The correct production pattern is not “trust the model more.” It is give the model a smaller computer.
1. The Sandbox Boundary
agent
↓
ephemeral workspace
├─ filesystem: scoped
├─ CPU / memory: bounded
├─ runtime: code + shell
├─ network: deny / allowlist
├─ secrets: none by default
└─ artifacts: explicit export
↓
governed tool boundary
↓
approval / policy
↓
external side effect
Filesystem
Give the run a workspace, not the host filesystem. Mount only necessary inputs and make sensitive host paths unreachable. Artifact export should be explicit so “the agent created a file” does not mean “the agent can write anywhere.”
Network
Network access should be treated as a capability. Many analysis tasks need no outbound network at all. Others may need an allowlist. Broad egress turns arbitrary code execution into a data-exfiltration path.
Secrets
Do not inject long-lived provider or business credentials into the sandbox environment by default. If the agent needs an external capability, prefer a governed tool that authenticates outside the sandbox and exposes only the narrow operation required.
Time and resources
Set wall-clock, CPU, memory, process, and output-size limits. Without them, an accidental infinite loop or unbounded data expansion becomes an infrastructure incident instead of a failed agent step.
Artifact handoff
Separate creation from consequence. The sandbox can produce a patch, report, image, SQL file, or deployment plan. A later policy/approval step decides whether that artifact is published or applied.
Threat-model the sandbox by capability
“Sandboxed” is not a binary property. Enumerate the capabilities the process receives: read/write paths, outbound destinations, package installation, subprocess creation, kernel features, environment variables, mounted credentials, and maximum runtime. The security posture is the intersection of those capabilities, not the container label.
For example, a workspace with no host mounts but unrestricted internet and cloud credentials can still exfiltrate data or mutate infrastructure. A workspace with no network but a writable production volume can still cause damage. The useful design artifact is a capability matrix that shows exactly what each class of agent job may do.
Prefer artifact promotion over direct mutation
Many useful code-agent tasks do not require live production access. Let the agent create a patch, notebook, report, migration plan, deployment manifest, or transformed dataset in the sandbox. Then promote that artifact through a separate reviewed path. This creates a natural checkpoint where deterministic tests, policy, and human review can run before consequence.
The pattern is especially strong for software work: generate patch → run tests in sandbox → emit diff and test evidence → approve → apply through a controlled repository or deployment tool. The agent gets rich autonomy over computation without receiving equivalent autonomy over production.
Package installation is code execution too
An agent that can freely install packages can execute arbitrary third-party install scripts and expand its network surface. Production sandboxes should use curated base images, pinned dependency sets, allowlisted package sources, or an explicit package-install policy. The convenience of “pip install whatever you need” is appropriate for some low-risk ephemeral work, but it should be a conscious choice.
Capture enough evidence to reproduce the run
Store the sandbox image/runtime version, input artifact identifiers, executed commands or code cells, exit statuses, produced artifact hashes, and any governed external calls. Full byte-for-byte replay may not always be practical, but an operator should be able to explain what ran and which outputs crossed the sandbox boundary.
2. Where TrueFoundry Fits: Sandbox Inside the Harness, Authority Outside It
TrueFoundry's Agent Harness treats sandboxed execution as part of the runtime around the model, alongside context management, approvals, tool execution, and tracing. That makes code execution useful for both automation and context control: bulky intermediate data can be processed in the workspace rather than copied through repeated model turns.
External actions remain better expressed through the MCP Gateway, where tool credentials, access control, pre/post guardrails, and approvals are centralized. Agent Registry supplies the identity/owner context around the agent, and end-to-end traces correlate sandbox work with model and tool steps.

The architectural point is separation: the sandbox is where the agent may compute; the gateway is where the agent earns authority to affect other systems.
3. Boundaries, Stated Plainly
A sandbox reduces blast radius; it does not make arbitrary code trustworthy. Kernel/container escapes, dependency risk, excessive privileges, and network exposure still matter. High-assurance environments may require stronger isolation than a standard container.
Also, not every agent needs code execution. Adding a shell to a workflow that only calls three well-defined APIs increases attack surface without adding much capability. Use the smallest execution surface that solves the problem.
References
- TrueFoundry: What Is an Agent Harness?.
- TrueFoundry: JIT Context.
- TrueFoundry MCP Gateway.
- TrueFoundry Guardrails.
- TrueFoundry Agent Registry.
Sandboxing is a defense-in-depth control, not a guarantee of safety. Isolation strength should match the sensitivity of the code, data, network, and downstream systems involved.
TrueFoundry AI Gateway delivers ~3–4 ms latency, handles 350+ RPS on 1 vCPU, scales horizontally with ease, and is production-ready, while LiteLLM suffers from high latency, struggles beyond moderate RPS, lacks built-in scaling, and is best for light or prototype workloads.













.webp)
.webp)


.png)

.png)















