Install on AWS - TrueFoundry Docs

Deploy the TrueFoundry control plane and AI Gateway. See the overview for compute requirements and prerequisites.

Installation Instructions

Create S3 Bucket

Create a S3 Bucket with following config:

Make sure the bucket has lifecycle configuration to abort multipart upload set for 7 days.
Make sure CORS is applied on the bucket with the below configuration:

[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["GET", "POST", "PUT"],
    "AllowedOrigins": ["*"],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3000
  }
]

Setup Control Plane Platform IAM Role

Creating AWS IAM Role for Control Plane

Control Plane IAM Role needs to have permission to access the S3 bucket created in the previous step.

Create a new IAM role for Control Plane with a suitable name like tfy-control-plane-platform-deps
Add the following trust policy to the Control Plane IAM Role:

{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Principal": {
              "Federated": "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/oidc.eks.<AWS_REGION>.amazonaws.com/id/<OIDC_ID>"
          },
          "Action": "sts:AssumeRoleWithWebIdentity",
          "Condition": {
              "StringEquals": {
                  "oidc.eks.<AWS_REGION>.amazonaws.com/id/<OIDC_ID>:sub": [
                      "system:serviceaccount:truefoundry:truefoundry"
                  ]
              }
          }
      }
  ]
}

In place of <ACCOUNT_ID>, <AWS_REGION>, and <OIDC_ID> you can also give the values from your EKS cluster. You can find the OIDC_ID from the EKS cluster. Also, here we are assuming that the service account is truefoundry and the namespace is truefoundry, you can change it as per your needs.

Create a IAM Policy to allow access to the S3 Bucket with following config:

{
  "Statement": [
    {
      "Sid": "S3",
      "Effect": "Allow",
      "Action": ["s3:*"],
      "Resource": [
        "arn:aws:s3:::<YOUR_S3_BUCKET_NAME>",
        "arn:aws:s3:::<YOUR_S3_BUCKET_NAME>/*"
      ]
    }
  ],
  "Version": "2012-10-17"
}

Attach the IAM Policy to the Control Plane Platform IAM Role. You can also attach the IAM policy to access AWS bedrock models from the link here.

If you want to restrict the S3 permissions to a minimal set instead of using s3:*, see the FAQ.

If you are integrating with AWS bedrock models from a different AWS account, see the FAQ.

Create Postgres RDS Database

Create a Postgres RDS instance of size db.t3.medium with storage size of 30GB.

Important Configuration Notes: - For PostgreSQL 17: Disable SSL by setting force_ssl parameter to 0 in the parameter group - Security Group: Ensure your RDS security group has inbound rules allowing traffic from EKS node security groups

In case you want to setup PostgreSQL on Kubernetes and not use RDS for testing purposes, skip this step and set devMode to true in the values file below

Create Kubernetes Secrets

We will create two secrets in this step:

Store the License Key and DB Credentials
Store the Image Pull Secret

Create Kubernetes Secret for License Key and DB Credentials

We need to create a Kubernetes secret containing the licence key and db credentials.

If you are using PostgreSQL on Kubernetes in the dev mode, the values will be as follows:DB_HOST: <HELM_RELEASE_NAME>-postgresql.<NAMESPACE>.svc.cluster.local // eg. truefoundry-postgresql.truefoundry.svc.cluster.localDB_NAME: truefoundryDB_USERNAME: postgres # In order to use custom username, please update the same at postgresql.auth.usernameDB_PASSWORD: randompassword # You can change this to any value here.

truefoundry-creds.yaml

apiVersion: v1
kind: Secret
metadata:
  name: truefoundry-creds
type: Opaque
stringData:
  TFY_API_KEY: <TFY_API_KEY> # Provided by TrueFoundry team
  DB_HOST: <DB_HOST>
  DB_NAME: <DB_NAME>
  DB_USERNAME: <DB_USERNAME>
  DB_PASSWORD: <DB_PASSWORD>

Apply the secret to the Kubernetes cluster (Assuming you are installing the control plane in the truefoundry namespace)

kubectl apply -f truefoundry-creds.yaml -n truefoundry

Create Kubernetes Secret for Image Pull Secret

We need to create a Image Pull Secret to enable pulling the truefoundry images from the private registry.

truefoundry-image-pull-secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: truefoundry-image-pull-secret
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: <IMAGE_PULL_SECRET> # Provided by TrueFoundry team

Apply the secret to the Kubernetes cluster (Assuming you are installing the control plane in the truefoundry namespace)

kubectl apply -f truefoundry-image-pull-secret.yaml -n truefoundry

Create HelmChart Values file

Create a values file as given below and replace the following values:

Control Plane URL: URL that you will map to the control plane dashboard (e.g., https://truefoundry.example.com)
Tenant Name: Tenant name provided by TrueFoundry team
AWS S3 Bucket Name: Name of the S3 bucket you created in the previous step (e.g., my-truefoundry-bucket)
AWS Region: Region of the S3 bucket you created in the previous step (e.g., us-west-2)
Control Plane IAM Role ARN: ARN of the IAM role you created in the previous step (e.g., arn:aws:iam::123456789012:role/tfy-control-plane-platform-deps)

truefoundry-values.yaml

global:
  # Domain to map the platform to
  controlPlaneURL: https://example.com

  # Ask TrueFoundry team to provide these
  tenantName: <TENANT_NAME>

  # Choose the resource tier as per your needs
  resourceTier: medium # or small or large

  # This is the reference to the secrets we created in the previous step
  existingTruefoundryCredsSecret: "truefoundry-creds"
  imagePullSecrets:
    - name: "truefoundry-image-pull-secret"
  ## Add if you have restricted public registry access
  # image:
  #   pullSecretNames:
  #   - "truefoundry-image-pull-secret"

  config:
    defaultCloudProvider: "aws"
    storageConfiguration:
      awsS3BucketName: "<AWS_S3_BUCKET_NAME>"
      awsRegion: "<AWS_REGION>"

  serviceAccount:
    annotations:
      eks.amazonaws.com/role-arn: <CONTROL_PLANE_IAM_ROLE_ARN>

  ingress:
    hosts:
      - example.com
    enabled: true
    annotations: {}
    ingressClassName: nginx # Replace with your ingress class name

# In case, you want to spin up PostgreSQL on kubernetes, enable this
# Please add creds and host details in the secret `truefoundry-creds` in the previous step
devMode:
  enabled: false
tags:
  llmGateway: true
  llmGatewayRequestLogging: true

# Disable few dependencies for only LLM Gateway setup
tfyBuild:
  enabled: false
sfyManifestService:
  enabled: false
tfyController:
  enabled: false
tfyConfigs:
  enabled: false

Install Helm chart

helm upgrade --install truefoundry oci://tfy.jfrog.io/tfy-helm/truefoundry -n truefoundry --create-namespace -f truefoundry-values.yaml

Platform

Documentation Index

​Installation Instructions

Installation Instructions