> ## Documentation Index
> Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Deploy AI Gateway
> Learn how to deploy TrueFoundry's AI Gateway on your own infrastructure with detailed compute requirements and installation instructions.
## Key Components
To install the complete control plane on your own infrastructure, you need to install the following components:
Shipped as a single Helm chart called `truefoundry` that includes both the control plane and AI Gateway.
Managed or self-hosted PostgreSQL (version 13 or higher).
S3, GCS, Azure Blob, or any other S3-compatible storage for request logs.
## Compute Requirements
TrueFoundry ships as a helm chart ([https://github.com/truefoundry/infra-charts/tree/main/charts/truefoundry](https://github.com/truefoundry/infra-charts/tree/main/charts/truefoundry)) that has configurable options to either deploy both Deployment and AI Gateway feature or just choose the one of them according to your needs. The compute requirements change based on the set of features and the scale of the number of users and requests.
Here are a few scenarios that you can choose from based on your needs.
The small tier is recommended for development purposes. Here all the components are deployed on Kubernetes and in non HA mode (single replica). This is suitable if you are just testing out the different features of TrueFoundry.
This setup brings up 1 replica of the services and is not highly-available. It
can enable you to test the features but we do not recommend this for
production mode.
| Component | Resources | Min Nodes | Approx. Cost (AWS) |
| :-------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------- | :---------------------------------------------- |
| Helm-Chart
*(AI Gateway Control Plane components)* | CPU: 6 vCPU
Memory: 12 GB
Storage: 60 GB
*Persistent Volumes (Block Storage) on Kubernetes* | 2
*Pods should be spread over min 2 nodes* | \~ \$220 pm (EC2 and EC2 others) |
| Helm-Chart
*(AI Gateway component only)* | CPU: 1 vCPU
Memory: 512 Mi
Storage: — | 1
*Pods should be spread over min 1 node* | \~ \$35 pm (EC2 and EC2 others) |
| Postgres
(Deployed on Kubernetes) | CPU: 0.5 vCPU
Memory: 0.5 GB
Storage: 5 GB
*Persistent Volumes (Block Storage) on Kubernetes* | — | \~ \$15 pm (RDS compute and storage) |
| Blob Storage
(S3 Compatible) | Storage: 20 GB | — | \~ \$3 pm (S3 storage) |
Resource requirements are the same across AWS, GCP, and Azure.
Costs shown are approximate AWS prices
The medium tier is configured for production and will suffice teams of 10-500 members.
**The AI Gateway is configured for a minimum 3 replicas (1 vCPU 1GB each) which can handle around 500 requests/second to LLMs.**
It's configured to be horizontally scalable and autoscale when the load increases. The Block Storage and S3 are used to store LLM request logs. The size is dependent on the size and number of requests and should be set as per the expected usage.
| Component | Resources | Min Nodes | Approx. Cost (AWS) |
| :-------------------------------------------------------- | :------------------------------------------------------- | :----------------------------------------------------------- | :---------------------------------------------- |
| Helm-Chart
*(AI Gateway Control Plane components)* | CPU: 14 vCPU
Memory: 32 GB
Storage: 250 GB | 3
*Pods should be spread over min 3 nodes* | \~ \$600 pm (EC2 and EC2 others) |
| Helm-Chart
*(AI Gateway component only)* | CPU: 3 vCPU
Memory: 3 GB
Storage: — | 3
*Pods should be spread over min 3 nodes* | \~ \$105 pm (EC2 and EC2 others) |
| Postgres
(Managed Database) | CPU: 2 vCPU
Memory: 4 GB
Storage: 30 GB | — | \~ \$60 pm (RDS compute and storage) |
| Blob Storage
(S3 Compatible) | Storage: 500 GB | — | \~ \$15 pm (S3 storage) |
Resource requirements are the same across AWS, GCP, and Azure. Costs shown are approximate AWS prices
The large tier is configured for production and will suffice organizations of 500-50000 members.
**The AI Gateway is configured for a minimum 10 replicas (1 vCPU 1GB each) which can handle around 2000 requests/second to LLMs.**
It's configured to be horizontally scalable and autoscale when the load increases. The Block Storage and S3 are used to store LLM request logs. The size is dependent on the size and number of requests and should be set as per the expected usage.
| Component | Resources | Min Nodes | Approx. Cost (AWS) |
| :-------------------------------------------------------- | :------------------------------------------------------- | :------------------------------------------------------------- | :---------------------------------------------- |
| Helm-Chart
*(AI Gateway Control Plane components)* | CPU: 32 vCPU
Memory: 64 GB
Storage: 400 GB | 10
*Pods should be spread over min 10 nodes* | \~ \$1400 pm (EC2 and EC2 others) |
| Helm-Chart
*(AI Gateway component only)* | CPU: 10 vCPU
Memory: 10 GB
Storage: — | 10
*Pods should be spread over min 10 nodes* | \~ \$350 pm (EC2 and EC2 others) |
| Postgres
(Managed Database) | CPU: 2 vCPU
Memory: 4 GB
Storage: 30 GB | — | \~ \$60 pm (RDS compute and storage) |
| Blob Storage
(S3 Compatible) | Storage: 1000 GB | — | \~ \$30 pm (S3 storage) |
Resource requirements are the same across AWS, GCP, and Azure. Costs shown are approximate AWS prices
## Prerequisites for Installation
Make sure the following are set up in your environment before starting the installation.
| Requirement | Details |
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Kubernetes Cluster** | Kubernetes version `1.27` or higher. |
| **Dynamic Storage Provisioning** | Support for dynamic PVC provisioning (e.g., AWS EBS, Azure Disk, GCP Persistent Disk). |
| **Ingress** | An ingress controller (e.g., NGINX Ingress Controller) or Istio service mesh to expose the control plane dashboard and AI Gateway. |
| **Domain & TLS Certificate** | A domain to map the ingress of the control plane dashboard and AI Gateway, along with a TLS certificate for the domain. This domain is referred to as the **Control Plane URL** in our documentation. |
| **Egress Access** | Outbound HTTPS access to the TrueFoundry endpoints listed below. |
| **PostgreSQL Database** | PostgreSQL version `>= 13`. Managed PostgreSQL (AWS RDS, Google Cloud SQL, Azure Database for PostgreSQL) is recommended for production. For PostgreSQL `17+`, disable SSL (AWS: set `force_ssl=0`; Azure: set `require_secure_transport=false` in the parameter group). For instance sizing, refer to the [Compute Requirements](#compute-requirements) section. |
| **Blob Storage** | S3, GCS, Azure Blob Storage, or any other S3-compatible storage to store AI Gateway request logs. Setup instructions are included in the guide below. |
**Egress URLs from your cluster to TrueFoundry**
* [`https://auth.truefoundry.com`](https://auth.truefoundry.com) — Central authentication server for licensing and authentication.
* [`https://login.truefoundry.com`](https://login.truefoundry.com) — Login UI for the central authentication server.
* [`https://catalogue.truefoundry.com`](https://catalogue.truefoundry.com) — Central repository for fetching catalogues for latest models, their public cost, MCP servers, etc.
* [`https://analytics.truefoundry.com`](https://analytics.truefoundry.com) — Analytics server for sending usage analytics to TrueFoundry.
If you do not have a managed PostgreSQL database and just want to test, set `devMode` to `true` in the values file to spin up a local PostgreSQL database on Kubernetes.
## Things required from TrueFoundry team
The following will be provided by the TrueFoundry team. Make sure your organization is [registered on TrueFoundry](https://truefoundry.com/register) before requesting them.
| Item | Description |
| ------------------------------- | ------------------------------------------------------------- |
| **Tenant Name** | Unique identifier for your control plane. |
| **License Key** (`TFY_API_KEY`) | Authentication key used to license your installation. |
| **Image Pull Secret** | Docker config JSON used to pull TrueFoundry container images. |
One Tenant Name and License Key must only be used to set up one Control Plane. Switching to a new tenant name and license key later will lead to complete data loss of the existing control plane.
## Contents
| Section | Description |
| ------------------------------------------------------------------------------- | ------------------------------------------------------------- |
| [Overview](/docs/platform/deploy-control-plane-and-gateway-plane) | Components, compute requirements, prerequisites |
| [AWS installation](/docs/platform/deploy-control-plane-install-aws) | Install on AWS with S3 and IAM |
| [GCP installation](/docs/platform/deploy-control-plane-install-gcp) | Install on GCP with GCS |
| [Azure installation](/docs/platform/deploy-control-plane-install-azure) | Install on Azure Blob Storage |
| [OpenShift installation](/docs/platform/deploy-control-plane-install-openshift) | OpenShift-specific Helm values |
| [On-prem installation](/docs/platform/deploy-control-plane-install-onprem) | Minio, self-hosted Postgres, generic setup |
| [FAQs](/docs/platform/deploy-control-plane-faq) | Common questions, troubleshooting, and advanced configuration |