What the checklist covers:
31
Evaluation criteria for retention, deletion, and residency across your LLM stack
5
Control areas, from provider defaults to continuous deletion verification
12
Must-have controls, flagged so you know what to fix first
5
Providers covered: OpenAI, Anthropic, Azure OpenAI, Amazon Bedrock, and Google Vertex AI
Inside the checklist
Download the checklist to get:
A working framework your platform, security, and privacy teams can use to evaluate an AI gateway or audit the retention story across every model provider you use.
Download the Checklist✓
The full 31-point LLM data retention framework
Every criterion grouped by control area, with the question to ask and a must-have, should-have, or nice-to-have priority.
✓
A three-step rollout plan
Catalog your providers and contract terms, configure overrides at the gateway and provider level, then verify deletion on a schedule.
✓
A regulatory retention map
How retention schedules line up with GDPR Article 17, HIPAA record rules, and SEC 17a-4 / FINRA where they apply.
✓
A vendor evidence-pack request list
The architecture diagrams, retention configs, deletion-event logs, and subprocessor disclosures to ask any vendor for.
Why it matters
Your Data Retention Story Is Only as Strong as Your Weakest LLM Provider
Every LLM provider sets its own retention windows, training-data policy, and deletion guarantees, and those defaults change over time. Some offer zero-data-retention modes, but only to eligible accounts and only on certain endpoints. Others retain data for abuse monitoring unless you opt out. Those choices decide your exposure to subpoenas, breach notifications, and right-to-be-forgotten requests.
Most enterprises now use several providers at once, which multiplies the problem. A request can be routed to one provider, fail over to another, write a cached copy, land in gateway logs, and be exported to a SIEM, all within a second. Each of those copies has its own retention clock.
One prompt, many copies. The checklist helps you decide how long each copy lives and how you prove it's gone.
Teams usually discover the gaps during a privacy review or the first erasure request. Zero-retention settings often don't cover stateful features like assistants, threads, files, vector stores, and batch jobs. Fallback routing quietly sends traffic to a provider nobody reviewed. Gateway logs, semantic caches, and embeddings keep copies of prompts long after the provider has deleted theirs. And nobody can produce evidence that deletion actually happened.
What good LLM data retention looks like
Every provider is cataloged with its default retention, training policy, and contract terms, and configured for zero or minimized retention wherever the account is eligible. Routing and fallback are restricted to an approved-provider allow-list, and data residency is enforced at runtime, not just at deployment.
Gateway logs live in storage you own, with shorter TTLs for sensitive classes like PHI, PCI, and EU PII. Caches expire automatically. Erasure requests reach logs, backups, exports, and derived data, and are reconciled with litigation holds and regulated retention by policy. Retention changes and deletion events are logged, so you can hand auditors evidence instead of assurances.
Control retention at the AI gateway
You can't change how a provider stores data, but you can control which providers see it, what you keep, and for how long. An AI gateway is the one point every request passes through, which makes it the right place to enforce provider allow-lists, residency, log TTLs, and deletion. For full retention ownership, self-hosted models keep the entire lifecycle inside your infrastructure.
The LLM Data Retention Checklist turns that standard into 31 criteria you can run against every provider and your gateway. Download it to close the gaps before an erasure request or audit finds them.
Preview
The five control areas in the LLM data retention checklist
A sample of the criteria in each area. The full checklist includes all 31, with priorities and evaluation questions.
Area 1 · 7 checks
Provider defaults and opt-outs
Zero or minimized retention and training opt-out are set where eligible
Stateful features that retain data under zero-retention modes are inventoried
+5 more checks
Area 2 · 5 checks
Gateway-level retention controls
Gateway logs are stored in your own cloud storage
Prompt and response bodies auto-delete after a configurable TTL
+3 more checks
Area 3 · 8 checks
Right to erasure and deletion
All data tied to a data subject can be located and deleted per GDPR Article 17
Erasure, litigation holds, and regulated retention are reconciled by policy
+6 more checks
Area 4 · 4 checks
Data residency and sovereignty
Residency is enforced at runtime across inference, agents, tools, and observability
Region-specific endpoints are enforced by geography
+2 more checks
Area 5 · 7 checks
Audit, evidence, and verification
Retention configuration changes are logged with who, what, and when
Each subprocessor's retention terms are visible to customers
+5 more checks
How to use · 3 steps
Rolling it out
Catalog every LLM, embeddings, and tool provider with its default retention and contract terms
Configure TTLs, opt-outs, and zero-retention modes at gateway and provider level
Verify deletion with receipts, audit logs, and provider attestations
4.7
★★★★★
Gartner Peer Insights
Rated 4.7 on Gartner Peer Insights
Platform teams run TrueFoundry's AI Gateway for LLM, MCP, and agent traffic in production, with logging that stays in their own cloud, retention by team and data class, runtime data residency enforcement, and self-hosted deployment for full retention ownership.
96%
of reviewers likely to recommend
4.9
ease of deployment and admin
3–4 ms
added gateway latency
