Custom CA Certificate Injection

If your organization signs TLS certificates with a private Certificate Authority (CA) instead of a public one, clients do not trust those certificates by default. TLS handshakes fail with errors such as x509: certificate signed by unknown authority or unable to verify the first certificate. To fix this, the CA bundle must be present in the trust store of every pod that opens the connection. TrueFoundry supports this in three scenarios.

Scenario	Who needs the CA	How to configure it
Control plane calls an endpoint served with a custom CA certificate	Control plane pods such as `servicefoundry-server` and the AI Gateway	`global.customCA` in the `truefoundry` chart
Compute plane connects to the control plane	`tfy-agent` pods	`global.customCA` in the `tfy-agent` chart
Workloads call an endpoint served with a custom CA certificate	Any pod in a workspace namespace	Kyverno policies from the `tfy-kyverno-config` chart

This configuration is only needed for self-signed or internal CA certificates. Endpoints with certificates issued by a public CA are already trusted.

Control plane calling endpoints signed by a custom CA

Control plane components do not talk to each other over TLS — traffic between them is plain pod-to-pod communication inside the cluster. The control plane needs a custom CA when TrueFoundry services make outbound calls to endpoints that present a certificate signed by your private CA, such as model endpoints, integration provider endpoints, private registries, or corporate proxies. Set the CA in your truefoundry chart values:

truefoundry-values.yaml

global:
  customCA:
    # Enable custom CA certificate injection via initContainers
    enabled: true
    # PEM-encoded CA certificate. An initContainer merges it with the system CAs.
    certificate: |
      -----BEGIN CERTIFICATE-----
      ... (your CA certificate) ...
      -----END CERTIFICATE-----

You can also reference an existing ConfigMap that holds the certificate under the key ca-certificates.crt:

truefoundry-values.yaml

global:
  customCA:
    enabled: true
    existingConfigMap:
      name: custom-ca-certificates
      # When false, an initContainer merges your CA with the system CA bundle.
      # When true, the ConfigMap is mounted directly at /etc/ssl/certs/ and
      # replaces the system bundle, so it must contain system CAs + your CAs.
      overrideCAList: false

For the full walkthrough, including how to build the ConfigMap and upgrade the release, see How to configure custom CA certificates.

Compute plane connecting to the control plane

tfy-agent runs on the compute plane and connects to your control plane URL. If that URL serves a certificate signed by a private CA, or the traffic passes through a corporate TLS proxy, the agent needs the CA bundle. Set the CA in your tfy-agent chart values. When enabled, the bundle is mounted into the tfyAgent, tfyAgentProxy, and sdsServer pods:

tfy-agent-values.yaml

global:
  customCA:
    enabled: true
    certificate: |
      -----BEGIN CERTIFICATE-----
      ... (your CA certificate) ...
      -----END CERTIFICATE-----
    # Or reference an existing ConfigMap with the key "ca-certificates.crt"
    # existingConfigMap:
    #   name: custom-ca-certificates
    #   overrideCAList: false

Install or upgrade tfy-agent after the control plane is reachable, so the agent can complete its first connection. See TFY Agent for the full agent configuration.

Workloads calling endpoints signed by a custom CA

Workloads deployed through TrueFoundry — services, jobs, notebooks, and LLM deployments — run from arbitrary container images in workspace namespaces. When these workloads call an endpoint that presents a certificate signed by your private CA, they need the CA bundle too, and no single Helm value covers images you do not build. TrueFoundry handles this with two Kyverno policies packaged in the tfy-kyverno-config chart:

Policy	What it does
ConfigMap sync	Clones the CA bundle ConfigMap from a source namespace such as `truefoundry` into every matching workspace namespace and keeps the copies in sync
Pod volume mounts	Mutates pods at admission to mount the CA bundle at `/etc/ssl/certs/ca-certificates.crt` and set the SSL environment variables

Together they remove the need to rebuild images, patch pod specs, or distribute certificates by hand.

Prerequisites

A compute plane cluster connected to your TrueFoundry control plane.
Cluster admin access, because both policies are cluster-scoped resources.
Kyverno installed on the cluster.
tfy-kyverno-config chart version 0.1.10 or later.
Your internal CA certificate in PEM format, referred to below as custom-ca.crt.

Build the combined CA bundle

The bundle you distribute must contain the public CA certificates and your internal CA. If you ship only your internal CA, workloads lose trust for every public endpoint.Extract a current public bundle from a standard image:

docker run --rm nginx:latest cat /etc/ssl/certs/ca-certificates.crt > public-ca-bundle.crt

Check that your internal CA certificate is valid:

openssl x509 -in custom-ca.crt -noout -text

Append it to the public bundle:

cat public-ca-bundle.crt custom-ca.crt > ca-certificates.crt

Create the CA bundle ConfigMap

Create the ConfigMap in a source namespace. The examples use the truefoundry namespace and the ConfigMap name ca-cert-bundle, which is the chart default. The key must be ca-certificates.crt.

kubectl create configmap ca-cert-bundle \
  --from-file=ca-certificates.crt=ca-certificates.crt \
  -n truefoundry

To update an existing ConfigMap, apply it instead:

kubectl create configmap ca-cert-bundle \
  --from-file=ca-certificates.crt=ca-certificates.crt \
  -n truefoundry --dry-run=client -o yaml | kubectl apply -f -

The sync policy watches this ConfigMap, so future certificate rotations only need this one update.

Install Kyverno

helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update kyverno
helm install kyverno kyverno/kyverno --namespace kyverno --create-namespace

If Kyverno is already installed on the cluster, skip this step and reuse the existing installation.

Configure tfy-kyverno-config

Create the chart values:

kyverno-config-values.yaml

syncConfigMaps:
  enabled: true
  includeNamespaces:
    - "tfy-*"
  items:
    - namespace: truefoundry
      name: ca-cert-bundle

podVolumeMounts:
  enabled: true
  includeNamespaces:
    - "tfy-*"
  configMapName: ca-cert-bundle
  mountInitContainers: true

tfy-* matches the namespaces that TrueFoundry creates for workspaces. Adjust the include list if your workspaces use a different naming pattern, and add excludeNamespaces for namespaces that must be skipped.

Deploy the chart

TrueFoundry platform
Helm CLI

Deploying from the platform records the chart version and change history, which makes later upgrades easier to track.

In the TrueFoundry dashboard, go to Deployments and click New.
Click Show advanced and select Helm.
Choose Public Helm Repository as the chart source and fill in:
- Helm repository URL: https://truefoundry.github.io/infra-charts
- Chart name: tfy-kyverno-config
- Version: the chart version you want to pin
Paste your values into the values editor and click Submit.

See Deploy Helm Charts for the full flow, including OCI registry and Git repository sources for air-gapped clusters.

These policies are cluster-scoped. Deploying Helm charts that create cluster-scoped objects requires cluster admin privileges. If your setup does not allow that, install the chart with the Helm CLI instead.

helm repo add truefoundry https://truefoundry.github.io/infra-charts
helm repo update truefoundry
helm install tfy-kyverno-config truefoundry/tfy-kyverno-config \
  -n kyverno --create-namespace -f kyverno-config-values.yaml

Verify the injection

Confirm both policies exist. You should see <release-name>-sync-configmaps and <release-name>-pod-volume-mounts:

kubectl get clusterpolicy

Confirm the ConfigMap was cloned into a workspace namespace:

kubectl get configmap ca-cert-bundle -n tfy-myworkspace

Deploy or restart a workload, then check the mount and the environment variables:

kubectl exec <pod> -n tfy-myworkspace -- ls -l /etc/ssl/certs/ca-certificates.crt
kubectl exec <pod> -n tfy-myworkspace -- env | grep -E 'SSL_CERT_FILE|REQUESTS_CA_BUNDLE|SSL_CERT_DIR'

Confirm the workload reaches an endpoint that uses your internal CA:

kubectl exec <pod> -n tfy-myworkspace -- curl -sS -o /dev/null -w '%{http_code}\n' https://internal.example.com

Policies created by the chart

The chart renders these manifests from your values. They are shown here so you can review what is applied to the cluster. ConfigMap sync:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: tfy-kyverno-config-sync-configmaps
spec:
  rules:
    - name: sync-configmap-ca-cert-bundle
      skipBackgroundRequests: false
      match:
        any:
          - resources:
              kinds:
                - Namespace
              names:
                - "tfy-*"
      generate:
        generateExisting: true
        synchronize: true
        apiVersion: v1
        kind: ConfigMap
        name: ca-cert-bundle
        namespace: "{{request.object.metadata.name}}"
        clone:
          namespace: truefoundry
          name: ca-cert-bundle

Pod volume mounts. The init container rule mirrors the container rule and is rendered only when mountInitContainers is enabled:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: tfy-kyverno-config-pod-volume-mounts
spec:
  background: false
  rules:
    - name: add-volumes
      match:
        resources:
          kinds:
            - Pod
          namespaces:
            - "tfy-*"
      mutate:
        patchStrategicMerge:
          spec:
            volumes:
              - name: custom-ca-cert
                configMap:
                  name: ca-cert-bundle
                  items:
                    - key: ca-certificates.crt
                      path: ca-certificates.crt

    - name: patch-all-containers
      match:
        resources:
          kinds:
            - Pod
          namespaces:
            - "tfy-*"
      mutate:
        foreach:
          - list: request.object.spec.containers
            patchStrategicMerge:
              spec:
                containers:
                  - name: "{{ element.name }}"
                    env:
                      - name: SSL_CERT_FILE
                        value: /etc/ssl/certs/ca-certificates.crt
                      - name: REQUESTS_CA_BUNDLE
                        value: /etc/ssl/certs/ca-certificates.crt
                      - name: SSL_CERT_DIR
                        value: /etc/ssl/certs
                    volumeMounts:
                      - name: custom-ca-cert
                        mountPath: /etc/ssl/certs/ca-certificates.crt
                        subPath: ca-certificates.crt
                        readOnly: true

Configuration reference

syncConfigMaps parameters

Parameter	Description	Default
`syncConfigMaps.enabled`	Enable ConfigMap syncing across namespaces	`false`
`syncConfigMaps.useClusterPolicy`	Render a `ClusterPolicy` (`kyverno.io/v1`) instead of a `GeneratingPolicy`. Required for wildcard namespace patterns	`true`
`syncConfigMaps.includeNamespaces`	Namespaces to sync into. When non-empty, only these are matched. Supports wildcards such as `tfy-*`	`[]`
`syncConfigMaps.excludeNamespaces`	Namespaces to skip	`[]`
`syncConfigMaps.items`	ConfigMaps to sync. Each entry takes `namespace` (the source) and `name`	`[]`

The generated policy sets generateExisting: true and synchronize: true, so it populates namespaces that already exist as well as new ones, and propagates later changes to the source ConfigMap.

podVolumeMounts parameters

Parameter	Description	Default
`podVolumeMounts.enabled`	Enable volume injection into pods	`false`
`podVolumeMounts.useClusterPolicy`	Render a `ClusterPolicy` (`kyverno.io/v1`) instead of a `MutatingPolicy`. Required for wildcard namespaces, environment variables, `readOnly` mounts, and ConfigMap key selection	`true`
`podVolumeMounts.policyName`	Name of the generated `ClusterPolicy`	`<release-name>-pod-volume-mounts`
`podVolumeMounts.configMapName`	ConfigMap to mount for the built-in CA mount	`ca-cert-bundle`
`podVolumeMounts.mountInitContainers`	Also mount the volume into init containers	`false`
`podVolumeMounts.includeNamespaces`	Namespaces to mutate. When non-empty, only these are matched. Supports wildcards such as `tfy-*`	`[]`
`podVolumeMounts.excludeNamespaces`	Namespaces to skip	`[]`
`podVolumeMounts.objectSelector`	Label selector that restricts which pods are mutated	`{}`
`podVolumeMounts.mountDetails`	Replaces the chart’s default mount list	`[]`
`podVolumeMounts.additionalMountDetails`	Appends extra mounts to the default list	`[]`

Each entry in mountDetails and additionalMountDetails accepts:

Field	Required	Description
`type`	Yes	`configMap` or `secret`
`configMapName`	For ConfigMaps	Name of the ConfigMap to mount
`secretName`	For Secrets	Name of the Secret to mount
`volumeName`	No	Volume name. Defaults to the ConfigMap or Secret name
`mountPath`	Yes	Path inside the container
`subPath`	No	Sub-path within the volume
`readOnly`	No	Mount read-only
`configMapItems`	No	Key-to-path mapping for ConfigMap volumes
`envVars`	No	Environment variables to set on every matched container

For example, to mount a TLS Secret alongside the default CA bundle:

kyverno-config-values.yaml

podVolumeMounts:
  enabled: true
  includeNamespaces:
    - "tfy-*"
  additionalMountDetails:
    - type: secret
      secretName: internal-tls
      mountPath: /etc/tls
      readOnly: true

Default mount and environment variables

When mountDetails is empty, the chart mounts the CA bundle and sets the variables that most runtimes read:

Setting	Value
Mount path	`/etc/ssl/certs/ca-certificates.crt` with `subPath: ca-certificates.crt`, read-only
`SSL_CERT_FILE`	`/etc/ssl/certs/ca-certificates.crt`
`REQUESTS_CA_BUNDLE`	`/etc/ssl/certs/ca-certificates.crt`
`SSL_CERT_DIR`	`/etc/ssl/certs`

Setting mountDetails replaces these defaults, including the environment variables. To keep the CA mount and add more volumes, use additionalMountDetails.

Some runtimes keep their own trust store and ignore these variables. For Node.js applications, add NODE_EXTRA_CA_CERTS pointing at the same path. For Java applications, import the CA into the JVM truststore.

Troubleshooting

Symptom	Likely cause	Fix
ConfigMap missing in a workspace namespace	Namespace does not match `includeNamespaces`, or wildcards are used with `useClusterPolicy: false`	Correct the include list and keep `useClusterPolicy: true`, then check `kubectl describe clusterpolicy <release-name>-sync-configmaps`
Pods do not have the volume	Pods were created before the policy was installed	Restart the workload; the policy mutates pods at admission
Volume present but TLS still fails	Bundle is missing the internal CA, or the endpoint serves an incomplete chain	Verify with `openssl s_client -connect internal.example.com:443 -CAfile /etc/ssl/certs/ca-certificates.crt` inside the pod
Calls to public endpoints break	Bundle contains only the internal CA	Rebuild the combined bundle and update the source ConfigMap; the sync policy propagates the fix
Application ignores the mounted bundle	Runtime uses its own trust store	Set `NODE_EXTRA_CA_CERTS` for Node.js, or import the CA into the JVM truststore for Java

Use these commands to narrow down where the injection stopped. Check whether the sync policy ran and what it reported:

kubectl describe clusterpolicy <release-name>-sync-configmaps

Check whether a running pod received the volume:

kubectl get pod <pod> -n tfy-myworkspace -o yaml | grep -A 5 'custom-ca-cert'

Check the certificate chain that the endpoint serves against the mounted bundle:

kubectl exec <pod> -n tfy-myworkspace -- \
  openssl s_client -connect internal.example.com:443 -CAfile /etc/ssl/certs/ca-certificates.crt </dev/null

​Control plane calling endpoints signed by a custom CA

​Compute plane connecting to the control plane

​Workloads calling endpoints signed by a custom CA

​Prerequisites

​Policies created by the chart

​Configuration reference

​Troubleshooting

​Related pages

Control plane calling endpoints signed by a custom CA

Compute plane connecting to the control plane

Workloads calling endpoints signed by a custom CA

Prerequisites

Policies created by the chart

Configuration reference

Troubleshooting

Related pages