Documentation Index
Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt
Use this file to discover all available pages before exploring further.
How to enable SSL for PostgreSQL connections?
How to enable SSL for PostgreSQL connections?
DB_SSL_MODE environment variable in your truefoundry-values.yaml.Supported DB_SSL_MODE values:| Mode | Encryption | Certificate Validation | Use Case |
|---|---|---|---|
disable | No | No | Local development or trusted networks |
no-verify | Yes | No | Managed databases with self-signed or unverified certs |
require | Yes | Yes (system CA store) | When you have a valid CA certificate and want full verification |
verify-ca | Yes | Yes (custom CA) | Same as require but explicitly checks CA |
verify-full | Yes | Yes (CA + hostname) | Strictest mode, validates CA and hostname |
| Variable | Purpose | Required |
|---|---|---|
DB_SSL_CA_PATH | Path to the server CA certificate file | For require, verify-ca, or verify-full modes |
DB_SSL_CERT_PATH | Path to the client certificate file (for mTLS) | Only for mTLS (GCP Cloud SQL, Azure Database for PostgreSQL) |
DB_SSL_KEY_PATH | Path to the client private key file (for mTLS) | Only for mTLS (GCP Cloud SQL, Azure Database for PostgreSQL) |
DB_SSL_CA_PATH), while GCP Cloud SQL and Azure Database for PostgreSQL may require all three certificate paths when client certificate authentication (mTLS) is enabled. Refer to the cloud-specific control plane documentation for detailed examples.no-verify)This is the simplest option for managed databases. It encrypts the connection but skips server certificate validation.require)This mode encrypts the connection and validates the server certificate. You must provide the appropriate certificate files for your database provider. The example below shows the full configuration with all three certificate paths (for GCP/Azure mTLS). For AWS RDS, only DB_SSL_CA_PATH is needed.Create a Kubernetes Secret containing your certificate files:truefoundry-values.yaml to mount the certificates and set the SSL paths:How to configure custom CA certificates?
How to configure custom CA certificates?
Method 1: Pass customCA as a multiline string
You can directly provide the CA certificate content as a multiline string in yourvalues.yaml:Method 2: Use an existing ConfigMap containing CA certificate(s)
If you already have your custom CA certificates in a Kubernetes ConfigMap, you can reference it directly. An initContainer will merge the custom CA with the system CAs.Create a ConfigMap with your custom CA certificate(s)
Reference the ConfigMap in your Helm values
truefoundry-values.yaml to reference the ConfigMap:Method 2b: Use an existing ConfigMap with overrideCAList
If you want the ConfigMap to replace the system CA bundle entirely instead of merging, set overrideCAList to true. In this mode, the ConfigMap is mounted directly at /etc/ssl/certs/ (no initContainer is used), so the ConfigMap must contain the full CA bundle (system + custom CAs).Prepare your CA certificate file
custom-ca.crt:/etc/ssl/certs/ca-certificates.crt with your custom CA included (system CAs + your custom CA).Create a ConfigMap from the complete ca-certificates.crt file
Reference the ConfigMap in your Helm values with overrideCAList
truefoundry-values.yaml to reference the ConfigMap with overrideCAList enabled:How to enable in-pod TLS termination on the proxy (control plane and gateway)?
How to enable in-pod TLS termination on the proxy (control plane and gateway)?
| Plane | Helm chart | Values path | Caddy listener |
|---|---|---|---|
| Control plane | truefoundry | global.proxy.tls | :8080 on tfy-proxy |
| Gateway | tfy-llm-gateway | proxy.tls | :8081 on the gateway proxy sidecar (app stays on :8787) |
Traffic flow (in-pod termination)
Prerequisites
- A Kubernetes TLS
Secretin the release namespace with PEM certificate and private key (standard keystls.crtandtls.key). - An ingress controller that can forward TLS without terminating it when using Ingress (see below).
- For self-signed or private CAs: also configure custom CA certificates so Node.js services trust outbound HTTPS, or use in-cluster HTTP URLs for internal API calls (recommended).
Control plane (truefoundry chart)
Create the TLS Secret
kubernetes.io/tls secret in the truefoundry namespace. Replace the paths with your certificate and key files:*.primary.example.com, issue a cert that covers your control-plane hostname (for example cp.primary.example.com).Enable proxy TLS in Helm values
truefoundry-values.yaml:Configure ingress for TLS passthrough
global.proxy.tls.enabled is true, Caddy expects HTTPS on the service port. Your ingress must forward the TLS connection without terminating it.ingress-nginx — enable passthrough on the controller (once per cluster) and annotate the control-plane Ingress:controller.extraArgs.enable-ssl-passthrough: "true".Istio / Gateway API — configure TLS mode PASSTHROUGH on the Gateway listener that fronts the control plane. TLS is not configured on the HTTPRoute itself.Verify the control plane
Update gateway `CONTROL_PLANE_URL` when `tags.llmGateway` is enabled and control-plane proxy TLS is on
global.proxy.tls.enabled is true, truefoundry-tfy-proxy listens with TLS on port 8080. In-cluster HTTP calls such as http://<release>-tfy-proxy:8080 will fail (for example ECONNRESET or certificate errors).If you deploy the gateway with the truefoundry chart (tags.llmGateway: true), override tfy-llm-gateway.env.CONTROL_PLANE_URL to your HTTPS control-plane URL (global.controlPlaneURL), not the internal http://...-tfy-proxy:8080 address:tfy-llm-gateway chart already sets CONTROL_PLANE_URL from global.controlPlaneURL by default. The override above is required when tags.llmGateway is true on the truefoundry chart, because the parent chart default uses http://{{ .Release.Name }}-tfy-proxy:8080.Gateway plane (tfy-llm-gateway chart)
Use this when deploying the gateway as its own Helm release (gateway plane only) or when overriding the tfy-llm-gateway subchart under the parent truefoundry chart.Enable proxy TLS and ingress passthrough
truefoundry-values.yaml for tfy-llm-gateway):truefoundry (tags.llmGateway: true) — nest under tfy-llm-gateway::Configure environment variables for startup
env based on whether the control-plane proxy has in-pod TLS enabled.When global.proxy.tls.enabled is true on the control plane (same cluster), set CONTROL_PLANE_URL to the public control-plane URL. Do not use http://<release>-tfy-proxy:8080 — that port expects HTTPS:controlPlaneURL uses a private or mkcert-signed certificate.When control-plane proxy TLS is disabled (default), you can use the internal proxy URL for CONTROL_PLANE_URL if the gateway and control plane share a release:truefoundry with your Helm release name if different. SERVICEFOUNDRY_SERVER_URL is used to fetch NATS credentials (/v1/x/llm-gateway/nats-creds); pointing it at servicefoundry-server avoids TLS issues on the proxy port.Verify the gateway
unable to verify the first certificate when fetching NATS credentials, see the custom CA section or the internal HTTP env overrides above.proxy.tls on the gateway sidecar secures traffic into the gateway pod from clients. On the control plane, global.proxy.tls makes port 8080 HTTPS on tfy-proxy. Gateway pods must use CONTROL_PLANE_URL: "{{ .Values.global.controlPlaneURL }}" (plus global.customCA for private CAs), or call servicefoundry-server / tfy-nats directly over HTTP — not http://...-tfy-proxy:8080.How to enable and access control plane monitoring (Grafana)?
How to enable and access control plane monitoring (Grafana)?
truefoundry-values.yaml:- Replace
<your-truefoundry-control-plane-url>with your actual control plane domain (e.g.,app.example.com) and<tenant-name>with your TrueFoundry tenant name provided during onboarding. - Only users with the admin role can access this endpoint.
- Make sure to include the trailing
/at the end of the URL. - If you already have Prometheus or VictoriaLogs in your cluster, you can point the monitoring stack to them using
externalServicesinstead of installing new instances.
How do you add default metadata to all requests passing via the gateway?
How do you add default metadata to all requests passing via the gateway?
DEFAULT_GATEWAY_METADATA environment variable on the gateway. The value should be a JSON string of key-value pairs.Add the following to your gateway configuration in values file of the gateway plane:How to expose additional metadata as Prometheus labels for gateway metrics?
How to expose additional metadata as Prometheus labels for gateway metrics?
customer_id, request_type, environment), set the LLM_GATEWAY_METADATA_LOGGING_KEYS environment variable on the gateway. The value is a JSON-encoded array of metadata keys.Each key listed here is exposed as a Prometheus label prefixed with ai_gateway_metadata_* — for example, customer_id becomes the label ai_gateway_metadata_customer_id. You can then use these labels for granular filtering and aggregation in Grafana.Add the following to your gateway configuration in values file of the gateway plane:ai_gateway_metadata_customer_id and ai_gateway_metadata_request_type labels.How to use HTTPRoute to route traffic using Kubernetes Gateway API?
How to use HTTPRoute to route traffic using Kubernetes Gateway API?
Ingress resources. Use HTTPRoute when your cluster uses a Gateway API-compatible controller (e.g. Envoy Gateway, Istio, NGINX Gateway Fabric, GKE Gateway).Control plane (truefoundry chart)Add the following to your truefoundry-values.yaml, setting parentRefs to point to your existing Gateway:- Only one routing method should be enabled at a time. Disable
global.ingress.enabledandglobal.virtualservice.enabledwhen usinghttpRoute. - The
sectionNamemust match a named listener on yourGatewayresource. Omit it if your Gateway has a single unnamed listener. - TLS termination is handled by the parent
Gateway— no TLS configuration is needed on theHTTPRouteitself.
How to restrict AWS S3 permissions to a minimal set?
How to restrict AWS S3 permissions to a minimal set?
s3:* for the S3 bucket IAM policy for simplicity. If your organization requires a least-privilege approach, you can replace s3:* with the following minimal set of permissions:How to configure security context for TrueFoundry components?
How to configure security context for TrueFoundry components?
runAsNonRoot: true), use a read-only root filesystem (readOnlyRootFilesystem: true), and drop all privileges (capabilities.drop: [ALL]).However, NATS (used internally for messaging) does not have these defaults applied automatically. If your cluster enforces Pod Security Standards (e.g. restricted profile) or you want a consistent security posture across all components, you need to explicitly add the security context for NATS by adding the following to your truefoundry-values.yaml:container.merge and podTemplate.merge) compared to other TrueFoundry components. This is because NATS uses its own Helm chart conventions for overriding pod and container specs.How to deploy on OpenShift with restricted Security Context Constraints (SCC)?
How to deploy on OpenShift with restricted Security Context Constraints (SCC)?
podSecurityContext and securityContext values (such as runAsUser, runAsNonRoot, fsGroup, etc.) on its components, which conflicts with the restricted or restricted-v2 SCC.To resolve this, disable both pod-level and container-level security contexts for all components by adding the following overrides to your truefoundry-values.yaml:enabled: false removes all explicit security context fields from the pod and container specs, allowing OpenShift’s SCC admission controller to assign user and group IDs as needed.