Skip to main content
Azure AI Foundry is a legacy integration. Use Microsoft Foundry for new accounts. Microsoft Foundry is generally available in TrueFoundry v0.175.0 and covers OpenAI models together with models from other publishers on one account. To move this account, follow Migrate to Microsoft Foundry. Hub and hub-based Azure resources cannot be migrated.

Adding Models

This section explains the steps to add Azure AI Foundry models and configure the required access controls.
1

Select Azure AI Foundry (Legacy) as the provider

From the TrueFoundry dashboard, go to AI Gateway > Models, click Add Model and choose To New Provider Account. Select Azure AI Foundry (Legacy) from the provider list.
Add Provider Account page listing the supported model providers

Choose a provider in Add Provider Account

2

Add Azure AI Foundry Account Details

Give a unique name to your Azure AI Foundry account.
Authentication (API Key or Certificate) is configured per-model in the next step, not at the account level.
Azure AI Foundry Configure Account form in the provider account setup drawer

Configure Account for Azure AI Foundry

3

Add Models from Azure AI Foundry

Click Add Model Manually to open the model form. For Azure AI Foundry, you add models based on your deployments in Azure. First, ensure you have deployed a model in your Azure AI Foundry project. You can follow Microsoft’s instructions here.Once deployed, navigate to your deployment in the Azure AI Foundry portal to find the Target URI (endpoint URL), Deployment Name, and API Key.
Azure portal showing deployed model Target URI, deployment name, and API key

Find Endpoint URL and Model Name in Azure AI Foundry

Fill in the model form with the following details:

Setting the Azure Endpoint

Copy the Target URI from your deployment in the Azure AI Foundry portal, but remove the API path and query parameters, gateway appends those automatically.
Model addition form for Azure AI Foundry with fields for model name, endpoint URL, and authentication

Adding an Azure AI Foundry model to TrueFoundry

Azure AI Foundry integration supports various AI models including OpenAI, Meta Llama, Mistral, DeepSeek, Cohere, and Anthropic Claude models deployed in your Azure account.
4

Save and grant access

Click Test Credentials to check the account against the selected models, or click Save without Testing. To keep the account in Git instead, click Apply using YAML and apply the generated spec with tfy apply.To let other users, teams or virtual accounts use or manage these models, open Manage Accounts, click ⋮ on the account and choose Access Control > Grant Access. See Access control.

Inference

After adding the models, you can perform inference using an OpenAI-compatible API via the Playground or by integrating with your own application.
Code Snippet and Try in Playgroud Buttons for each model

Test in Playground

Cohere Rerank

Cohere reranker models deployed on Azure AI Foundry are supported through the Rerank API. When adding the model, select Rerank under Model Types and set the Azure Endpoint to the Cohere provider path — the gateway appends /rerank automatically:
The Azure Deployment Name you configure is sent as the model field on the upstream Azure request, so it must match the deployment name in Azure AI Foundry.

Mistral OCR - Document Processing

Extract text from documents while maintaining structure and formatting (headers, paragraphs, lists, tables) using the Mistral OCR model via Azure AI Foundry. Returns markdown format and supports multiple formats including PDFs, images (png, jpeg, avif), and office documents (pptx, docx).
This endpoint cannot be used via the Mistral SDK. The Mistral SDK automatically appends /v1 to the base URL, which causes a URL mismatch (e.g. the request is sent to <base_url>/v1/ocr instead of <base_url>/ocr). Use direct HTTP requests (e.g. requests in Python) as shown below. See the open GitHub issue for details.