Key Components
To install the complete control plane on your own infrastructure, you need to install the following components:TrueFoundry Control Plane + Gateway
Shipped as a single Helm chart called
truefoundry that includes both the control plane and AI Gateway.PostgreSQL Database
Managed or self-hosted PostgreSQL (version 13 or higher).
Blob Storage
S3, GCS, Azure Blob, or any other S3-compatible storage for request logs.
Compute Requirements
TrueFoundry ships as a helm chart (https://github.com/truefoundry/infra-charts/tree/main/charts/truefoundry) that has configurable options to either deploy both Deployment and AI Gateway feature or just choose the one of them according to your needs. The compute requirements change based on the set of features and the scale of the number of users and requests. Here are a few scenarios that you can choose from based on your needs.Small (Dev)
Small (Dev)
The small tier is recommended for development purposes. Here all the components are deployed on Kubernetes and in non HA mode (single replica). This is suitable if you are just testing out the different features of TrueFoundry.
Resource requirements are the same across AWS, GCP, and Azure.
Costs shown are approximate AWS prices
Medium (Recommended)
Medium (Recommended)
The medium tier is configured for production and will suffice teams of 10-500 members.
The AI Gateway is configured for a minimum 3 replicas (1 vCPU 1GB each) which can handle around 500 requests/second to LLMs.It’s configured to be horizontally scalable and autoscale when the load increases. The Block Storage and S3 are used to store LLM request logs. The size is dependent on the size and number of requests and should be set as per the expected usage.
Resource requirements are the same across AWS, GCP, and Azure. Costs shown are approximate AWS prices
Large (Prod)
Large (Prod)
The large tier is configured for production and will suffice organizations of 500-50000 members.
The AI Gateway is configured for a minimum 10 replicas (1 vCPU 1GB each) which can handle around 2000 requests/second to LLMs.It’s configured to be horizontally scalable and autoscale when the load increases. The Block Storage and S3 are used to store LLM request logs. The size is dependent on the size and number of requests and should be set as per the expected usage.
Resource requirements are the same across AWS, GCP, and Azure. Costs shown are approximate AWS prices
Prerequisites for Installation
Make sure the following are set up in your environment before starting the installation.Egress URLs from your cluster to TrueFoundry
https://auth.truefoundry.com— Central authentication server for licensing and authentication.https://login.truefoundry.com— Login UI for the central authentication server.https://catalogue.truefoundry.com— Central repository for fetching catalogues for latest models, their public cost, MCP servers, etc.https://analytics.truefoundry.com— Analytics server for sending usage analytics to TrueFoundry.
If you do not have a managed PostgreSQL database and just want to test, set
devMode to true in the values file to spin up a local PostgreSQL database on Kubernetes.Things required from TrueFoundry team
The following will be provided by the TrueFoundry team. Make sure your organization is registered on TrueFoundry before requesting them.One Tenant Name and License Key must only be used to set up one Control Plane. Switching to a new tenant name and license key later will lead to complete data loss of the existing control plane.