Rerank API (/rerank)

Provider capabilities

The table below summarizes gateway support for this endpoint by provider.

Legend:

✅ Supported by provider and TrueFoundry
Provided by provider, but not by TrueFoundry
Provider does not support this feature

Provider	Rerank
OpenAI
Azure OpenAI	✅
Anthropic
Bedrock	✅
Vertex
Cohere	✅
Gemini
Groq
Together-AI
xAI
DeepInfra

For every gateway endpoint and provider, see Supported APIs.

Using AWS Bedrock reranker models? The IAM user or role attached to your Bedrock model account needs the bedrock:Rerank permission in addition to bedrock:InvokeModel — reranking is served by a separate AWS endpoint (bedrock-agent-runtime), so invoke-only policies fail with AccessDeniedException. See AWS Bedrock — Required IAM Policy.

Reranking is a powerful technique that enhances the relevance of search results. It works by taking a query and a list of documents, then scoring each document based on how relevant it is to the query. This is especially useful in multi-step search pipelines where you:

Use a traditional or vector-based search system to retrieve candidate documents.

Use the Rerank API to reorder those candidates based on relevance.

Reranking helps boost precision in semantic search, chatbots, RAG systems, and knowledge retrieval systems.

The reranker takes:

A query (user input or search question)

A list of documents (initially retrieved search results)

Returns a ranked list with relevance scores

Code Snippet

import cohere

BASE_URL = "{GATEWAY_BASE_URL}"
API_KEY = "your-truefoundry-api-key"

# Configure Cohere client with TrueFoundry settings
client = cohere.ClientV2(
	api_key=API_KEY,
	base_url=BASE_URL
)

response = client.rerank(
    model="test-ss-cohere/rerank-multilingual-v3-0",
    query="Where is New Delhi?",
    documents=["New Delhi is capital of India","The sky is blue"],
    top_n=1,
)

print(response)

Expected Output

{
  "id": "a9279c7a-1e34-4bee-9892-db1d38ca9be0",
  "results": [
    {
      "index": 0,
      "relevance_score": 0.783542
    }
  ]
}

In this example, the document “New Delhi is the capital of India.” has a higher relevance score to the query compared to “The sky is blue.”, and is therefore ranked higher.

Get Started

LLM Gateway

MCP Registry and Gateway

Skills Registry

Prompt Registry

Guardrails and Security

Observability

Deployment

Admin Guide

Chat

Messages

Embeddings

Rerank

Responses

Image

Audio

Batch

Files

Fine-tuning

Moderations

Models

Provider capabilities

Code Snippet

Expected Output

​Provider capabilities

​Code Snippet

​Expected Output

Provider capabilities

Code Snippet

Expected Output