Deploy Gateway Plane

​

Prerequisites for Installation

1. Kubernetes Cluster: K8s cluster 1.27+.
2. Domain to map the ingress of the AI Gateway along with certificate for the domain. (also any ingress controller or istio gateway you are using)
   This Domain will be referred as Gateway Endpoint Host in our documentation.
3. TrueFoundry SaaS Account: Sign up and create a new tenant on TrueFoundry SaaS by logging in to TrueFoundry. You will receive an invitation on your email to verify your email and login.

4. Egress access to TrueFoundry SaaS:
   • Control Plane: https://<CONTROL_PLANE_URL> (for example, https://example-company.truefoundry.cloud)
5. Image Pull Secret: Image pull secret from TrueFoundry team to enable pulling the truefoundry images from the private registry.

​

Installation Instructions

1

Create Gateway Endpoint in TrueFoundry SaaS

Create an entry for Gateway Endpoints by logging into the TrueFoundry SaaS platform.

1. Navigate to Settings -> Gateway Endpoints
2. Create a new gateway endpoint with a unique name
3. Enter the URL where you will be deploying your gateway

curl -X 'PUT' --location 'https://<CONTROL_PLANE_URL>/api/svc/v1/llm-gateway/installations' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <PERSONAL_ACCESS_TOKEN>' \
-d '{
  "manifest": {
    "type": "gateway-installation",
    "name": "<GATEWAY_INSTALLATION_NAME>",
    "url": "<GATEWAY_URL>",
    "enabled": true,
    "isDefault": false
  }
}'

Parameter	Description
CONTROL_PLANE_URL	Your control plane URL, e.g. example-company.truefoundry.cloud (without https://)
PERSONAL_ACCESS_TOKEN	A Personal Access Token created from Settings -> Access -> Personal Access Tokens. The user must have the tenant-admin role.
GATEWAY_INSTALLATION_NAME	The AI Gateway installation name from step 1
GATEWAY_URL	The AI Gateway URL that you are planning to use

The AI Gateway installation name you provide here will be used in the next step to generate the GatewayToken.

2

Generate the AI GatewayToken

The GatewayToken is used by the AI Gateway service running in your infrastructure to authenticate with the TrueFoundry control-plane. To generate it, you need to call a control-plane API using a Personal Access Token from a user with the tenant-admin role. This is a one-time call — once you have the GatewayToken, you do not need to call this API again.

curl --location 'https://<CONTROL_PLANE_URL>/api/svc/v1/llm-gateway/installations/token?name=<GATEWAY_INSTALLATION_NAME>' \
--header 'Authorization: Bearer <PERSONAL_ACCESS_TOKEN>'

Parameter	Description
CONTROL_PLANE_URL	Your control plane URL, e.g. example-company.truefoundry.cloud (without https://)
PERSONAL_ACCESS_TOKEN	A Personal Access Token created from Settings -> Access -> Personal Access Tokens. The user must have the tenant-admin role.
GATEWAY_INSTALLATION_NAME	The AI Gateway installation name from step 1

Copy and save the GatewayToken from the response in a secure location. It will not be shown again.

Once you have the GatewayToken, proceed to create the Kubernetes secrets needed for the AI Gateway helm chart installation.

3

Create Kubernetes Secrets

We will create two secrets in this step:

1. Store the GatewayToken
2. Store the Image Pull Secret

Create Kubernetes Secret for GatewayToken

We need to create a Kubernetes secret containing the GatewayToken obtained in the previous step. The helm chart expects this token under the key name TFY_API_KEY.

truefoundry-creds.yaml

apiVersion: v1
kind: Secret
metadata:
  name: truefoundry-creds
type: Opaque
stringData:
  TFY_API_KEY: <GATEWAY_TOKEN> # GatewayToken obtained from step 2

Apply the secret to the Kubernetes cluster (Assuming you are installing the AI Gateway plane in the truefoundry namespace)

kubectl apply -f truefoundry-creds.yaml -n truefoundry

Create Kubernetes Secret for Image Pull Secret

We need to create a Image Pull Secret to enable pulling the truefoundry images from the private registry.

truefoundry-image-pull-secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: truefoundry-image-pull-secret
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: <IMAGE_PULL_SECRET> # Provided by TrueFoundry team

Apply the secret to the Kubernetes cluster (Assuming you are installing the AI Gateway plane in the truefoundry namespace)

kubectl apply -f truefoundry-image-pull-secret.yaml -n truefoundry

4

Create Helm Chart Values file

Create a values file as given below and replace the following values:

• CONTROL_PLANE_URL: Your control plane URL in the format company-name.truefoundry.cloud (without the https:// prefix). For example, if your company name is example-company, then use example-company.truefoundry.cloud.
• TENANT_NAME: The tenant name (same as the company name you used to create your account on TrueFoundry SaaS)
• GATEWAY_ENDPOINT_HOST: The domain where you will expose the AI Gateway endpoint (e.g., gateway.example.com)

truefoundry-values.yaml

global:
  # This is the reference to the secrets we created in the previous step
  imagePullSecrets:
    - name: "truefoundry-image-pull-secret"

  # Choose the resource tier as per your needs
  resourceTier: medium # or small or large
  controlPlaneURL: <CONTROL_PLANE_URL> # eg. https://example-company.truefoundry.cloud
  tenantName: <TENANT_NAME>

ingress:
  enabled: true
  annotations: {}
  ingressClassName: nginx
  tls: []
  hosts:
    - <GATEWAY_ENDPOINT_HOST>


# Optional: Istio configuration (if using Istio instead of standard ingress)
# istio:
#   virtualservice:
#     hosts:
#       - <GATEWAY_ENDPOINT_HOST>
#     enabled: true
#     retries:
#       enabled: true
#       retryOn: gateway-error
#     gateways:
#       - istio-system/tfy-wildcard
#     annotations: {}

You can find full list of supported values here.

5

Install the Helm Chart

Install the TrueFoundry Helm chart using the values file created in the previous step:

helm upgrade --install truefoundry oci://tfy.jfrog.io/tfy-helm/tfy-llm-gateway -n truefoundry --create-namespace -f truefoundry-values.yaml

6

Verify the Deployment

Verify that all pods are running:

kubectl get pods -n truefoundry

You can verify the deployment by checking the health of the AI Gateway endpoint by running the following command:

curl -X GET "https://<GATEWAY_ENDPOINT_HOST>/health"

A successful response indicates that the AI Gateway plane is deployed and running correctly.

​

Accessing the AI Gateway

​

FAQ

httpRoute:
  enabled: true
  parentRefs:
    - name: my-gateway # Name of your Gateway resource
      namespace: gateway-system # Namespace where the AI Gateway is deployed
      sectionName: https # Listener section on the AI Gateway (e.g. http or https)
  hostnames:
    - "ai-gateway.example.com"

helm upgrade --install tfy-llm-gateway oci://tfy.jfrog.io/tfy-helm/tfy-llm-gateway \
  -n truefoundry --create-namespace \
  -f truefoundry-values.yaml

env:
  DEFAULT_GATEWAY_METADATA: '{"custom_key1":"custom_value1","custom_key2":"custom_value2"}'

helm upgrade --install tfy-llm-gateway oci://tfy.jfrog.io/tfy-helm/tfy-llm-gateway \
  -n truefoundry --create-namespace \
  -f truefoundry-values.yaml