> ## Documentation Index > Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt > Use this file to discover all available pages before exploring further. # List prompts > List prompts with optional filtering by FQN, ML Repo, or name. ## OpenAPI ````yaml /openapi.json get /api/ml/v1/prompts openapi: 3.1.0 info: title: TrueFoundry API description: API for TrueFoundry version: 0.1.0 contact: {} servers: - url: https://{controlPlaneURL} variables: controlPlaneURL: default: app.truefoundry.com description: Control Plane URL security: [] tags: - name: Agent - name: Agent Skills - name: Applications - name: Metrics - name: Apply - name: Artifacts - name: Audit Logs - name: Clusters - name: Jobs - name: Logs - name: MCP Servers V2 - name: MLRepos - name: Model Deployments - name: Models - name: Personal Access Tokens - name: Prompts - name: Provider Integrations - name: SCIM v2 - name: Secret Groups - name: Secrets - name: Teams - name: Traces - name: Users - name: Virtual Accounts - name: Role Bindings - name: Workspaces paths: /api/ml/v1/prompts: get: tags: - Prompts summary: List prompts description: List prompts with optional filtering by FQN, ML Repo, or name. operationId: list_prompts_v1_prompts_get parameters: - description: >- Fully qualified name to filter prompts by (format: 'chat_prompt:{tenant_name}/{ml_repo_name}/{prompt_name}') required: false schema: type: string title: Fqn description: >- Fully qualified name to filter prompts by (format: 'chat_prompt:{tenant_name}/{ml_repo_name}/{prompt_name}') name: fqn in: query - description: ID of the ML Repo to filter prompts by required: false schema: type: string title: Ml Repo Id description: ID of the ML Repo to filter prompts by name: ml_repo_id in: query - description: Name of the prompt to filter by required: false schema: type: string title: Name description: Name of the prompt to filter by name: name in: query - description: Number of prompts to skip for pagination required: false schema: type: integer title: Offset description: Number of prompts to skip for pagination default: 0 name: offset in: query - description: Maximum number of prompts to return required: false schema: type: integer title: Limit description: Maximum number of prompts to return name: limit in: query - description: Whether to include prompts that have no versions required: false schema: type: boolean title: Include Empty Prompts description: Whether to include prompts that have no versions default: true name: include_empty_prompts in: query responses: '200': description: List of prompts matching the query with pagination information content: application/json: schema: $ref: '#/components/schemas/ListPromptsResponse' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' security: - jwt: [] components: schemas: ListPromptsResponse: properties: data: items: $ref: '#/components/schemas/Prompt' type: array title: Data description: List of prompts matching the query pagination: allOf: - $ref: '#/components/schemas/Pagination' title: Pagination description: Pagination information including total count, offset, and limit type: object required: - data - pagination title: ListPromptsResponse x-fern-type-name: ListPromptsResponse HTTPValidationError: properties: detail: items: $ref: '#/components/schemas/ValidationError' type: array title: Detail type: object title: HTTPValidationError x-fern-type-name: HTTPValidationError Prompt: properties: id: type: string title: Id description: Unique identifier for the artifact ml_repo_id: type: string title: Ml Repo Id description: ID of the ML Repo that this artifact belongs to type: type: string enum: - chat_prompt title: Type description: Type of the artifact, always 'chat_prompt' for Prompt entities default: chat_prompt name: type: string title: Name description: >- Name of the artifact (alphanumeric characters, hyphens, and underscores only, max 256 characters) fqn: type: string title: Fqn description: >- Fully qualified name of the artifact in the format '{artifact_type}:{tenant_name}/{ml_repo_name}/{artifact_name}' created_by_subject: allOf: - $ref: '#/components/schemas/Subject' title: Created By Subject description: Subject (user, team, or service account) that created this artifact created_at: type: string format: date-time title: Created At description: Timestamp when the artifact was created updated_at: type: string format: date-time title: Updated At description: Timestamp when the artifact was last updated latest_version: allOf: - $ref: '#/components/schemas/PromptVersion' title: Latest Version description: The most recent version of this prompt run_steps: items: type: integer type: array title: Run Steps description: List of run step numbers where this artifact was created or updated type: object required: - id - ml_repo_id - name - fqn - created_by_subject title: Prompt description: Generic artifact API DTO including run-step linkage for ML runs. x-fern-type-name: Prompt Pagination: type: object properties: total: type: integer description: Total number of items example: 100 offset: type: integer description: Number of items skipped example: 0 nullable: true limit: type: integer description: Number of items per page example: 10 nullable: true required: - total title: Pagination x-fern-type-name: Pagination ValidationError: properties: loc: items: anyOf: - type: string - type: integer type: array title: Location msg: type: string title: Message type: type: string title: Error Type type: object required: - loc - msg - type title: ValidationError x-fern-type-name: ValidationError Subject: type: object properties: subjectId: type: string description: Subject ID subjectType: description: Subject type allOf: - $ref: '#/components/schemas/SubjectType' subjectSlug: type: string description: Subject slug nullable: true subjectDisplayName: type: string description: Subject display name nullable: true subjectPatName: type: string description: Subject PAT name nullable: true subjectControllerName: type: string nullable: true subjectExternalIdentitySlug: type: string description: External identity slug (external_identity_id:sub:email) nullable: true required: - subjectId - subjectType title: Subject x-fern-type-name: Subject PromptVersion: properties: id: type: string title: Id description: Unique identifier for the artifact version fqn: type: string title: Fqn description: >- Fully qualified name of the artifact version in the format '{artifact_type}:{tenant_name}/{ml_repo_name}/{artifact_name}:{version}' created_by_subject: allOf: - $ref: '#/components/schemas/Subject' title: Created By Subject description: >- Subject (user, team, or service account) that created this artifact version created_at: type: string format: date-time title: Created At description: Timestamp when the artifact version was created updated_at: type: string format: date-time title: Updated At description: Timestamp when the artifact version was last updated manifest: allOf: - $ref: '#/components/schemas/ChatPromptManifest' title: Manifest description: Manifest containing metadata specific to the chat prompt version ml_repo_id: type: string title: Ml Repo Id description: ID of the ML Repo that this artifact version belongs to tags: items: type: string type: array title: Tags description: >- List of tags associated with this artifact version for filtering and organization version_alias: type: string title: Version Alias description: Optional human-readable version alias (e.g. v1.0.0) usage_code_snippet: type: string title: Usage Code Snippet description: Code snippet demonstrating how to use this artifact version usage_code_snippets: items: $ref: '#/components/schemas/UsageCodeSnippet' type: array title: Usage Code Snippets description: >- List of code snippets demonstrating how to use this prompt version in different languages prompt_id: type: string title: Prompt Id description: ID of the parent prompt that this version belongs to type: object required: - id - fqn - created_by_subject - manifest - ml_repo_id - prompt_id title: PromptVersion description: >- Tags, optional version alias, and SDK usage snippet (models, prompts, generic artifacts). x-fern-type-name: PromptVersion SubjectType: type: string enum: - user - team - serviceaccount - virtualaccount - external-identity - agent-identity - role description: >- Type of the authenticated subject. "user" for human users, "serviceaccount" (also known as virtual account) for programmatic access. title: SubjectType x-fern-type-name: SubjectType ChatPromptManifest: properties: name: type: string maxLength: 256 pattern: ^[A-Za-z0-9_\-]+$ title: Name description: >- Name of the prompt (alphanumeric characters, hyphens, and underscores only, max 256 characters) metadata: type: object title: Metadata description: >- Key value metadata. Should be valid JSON. For e.g. `{"business-unit": "sales", "quality": "good", "rating": 4.5}` ml_repo: type: string pattern: ^[a-zA-Z][a-zA-Z0-9\-]{1,98}[a-zA-Z0-9]$ title: Ml Repo description: >- Name of the ML Repo that this prompt belongs to (must start and end with alphanumeric, 2-100 characters) version: type: integer minimum: 1 title: Version description: Version of the entity type: type: string enum: - chat_prompt title: Type default: chat_prompt description: type: string maxLength: 512 title: Description version_alias: type: string maxLength: 128 pattern: ^v[a-zA-Z0-9.-]*([a-zA-Z0-9]+)$ title: Version Alias description: >- Version alias is alternate, ideally human readable, version string to reference an artifact version. It should start with `v` followed by alphanumeric and it can include `.` and `-` in between (e.g. `v1.0.0`, `v1-prod`, `v3-dev`, etc) messages: items: anyOf: - $ref: '#/components/schemas/SystemMessage' - $ref: '#/components/schemas/AssistantMessage' - $ref: '#/components/schemas/UserMessage' - $ref: '#/components/schemas/ToolMessage' - $ref: '#/components/schemas/DeveloperMessage' type: array title: Messages description: List of messages in the chat conversation, must be non-empty variables: additionalProperties: type: string type: object title: Variables description: >- Variables referenced in messages and that can be replaced when running generation model_configuration: $ref: '#/components/schemas/ModelConfiguration' tools: items: $ref: '#/components/schemas/ToolSchema' type: array title: Tools description: List of tools to be used in the chat prompt mcp_servers: items: anyOf: - $ref: '#/components/schemas/MCPServerWithFQN' - $ref: '#/components/schemas/MCPServerWithURL' type: array title: Mcp Servers description: A list of MCP servers FQNs or URLs and their tools guardrails: $ref: '#/components/schemas/Guardrails' response_format: oneOf: - $ref: '#/components/schemas/JsonObjectResponseFormat' - $ref: '#/components/schemas/JsonSchemaResponseFormat' title: Response Format description: Response format configuration for structured outputs discriminator: propertyName: type mapping: json_object: $ref: '#/components/schemas/JsonObjectResponseFormat' json_schema: $ref: '#/components/schemas/JsonSchemaResponseFormat' routing_config: oneOf: - $ref: '#/components/schemas/WeightBasedLoadBalancing' - $ref: '#/components/schemas/LatencyBasedLoadBalancing' - $ref: '#/components/schemas/PriorityBasedLoadBalancing' title: Routing Config description: Configuration for routing requests to different model targets discriminator: propertyName: type mapping: weight-based-routing: $ref: '#/components/schemas/WeightBasedLoadBalancing' latency-based-routing: $ref: '#/components/schemas/LatencyBasedLoadBalancing' priority-based-routing: $ref: '#/components/schemas/PriorityBasedLoadBalancing' cache_config: oneOf: - $ref: '#/components/schemas/SemanticCacheConfig' - $ref: '#/components/schemas/ExactMatchCacheConfig' title: Cache Config description: Cache configuration for the chat prompt discriminator: propertyName: type mapping: semantic: $ref: '#/components/schemas/SemanticCacheConfig' exact-match: $ref: '#/components/schemas/ExactMatchCacheConfig' tool_call_to_mcp_mapping: additionalProperties: $ref: '#/components/schemas/MCPServerToolDetails' type: object title: Tool Call To Mcp Mapping description: Mapping of tool calls to MCP server integration IDs and tool names logging_config: $ref: '#/components/schemas/LoggingConfig' sub_agents: items: $ref: '#/components/schemas/SubAgent' type: array title: Sub Agents description: Sub agents to be used in the chat prompt type: object required: - name - metadata - ml_repo - messages title: ChatPromptManifest description: Chat Prompt manifest. x-fern-type-name: ChatPromptManifest UsageCodeSnippet: properties: display_name: type: string title: Display Name description: >- Display label for the snippet (e.g. 'Python', 'Download via TFY CLI') language: type: string title: Language description: Language identifier (e.g. 'python', 'bash', 'javascript') libraries: items: type: string type: array title: Libraries description: Libraries, CLIs, or frameworks relevant to the snippet code: type: string title: Code description: The snippet body shown to users icon: type: string title: Icon description: Use a URL or an icon name like 'download'. type: object required: - display_name - language - code title: UsageCodeSnippet description: >- Example code or shell commands for using an ML entity (prompt, agent skill, etc.). x-fern-type-name: UsageCodeSnippet SystemMessage: description: System message for the chat type: object required: - role - content properties: role: description: Role of the message type: string enum: - system content: description: Text content for the system message oneOf: - type: string pattern: ^.[\s\S]*$ - $ref: '#/components/schemas/BlobStorageReference' name: description: Name of the system type: string title: SystemMessage x-fern-type-name: SystemMessage AssistantMessage: description: Assistant message turn type: object required: - role properties: role: description: Role of the message type: string enum: - assistant content: description: Text content for the assistant message oneOf: - type: string - type: array items: type: object oneOf: - $ref: '#/components/schemas/TextContentPart' - $ref: '#/components/schemas/RefusalContentPart' - $ref: '#/components/schemas/BlobStorageReference' tool_calls: description: List of tool calls made by the assistant type: array items: $ref: '#/components/schemas/ToolCall' name: description: Name of the assistant this message is from type: string title: AssistantMessage x-fern-type-name: AssistantMessage UserMessage: description: User message turn type: object required: - role - content properties: role: description: Role of the message type: string enum: - user content: description: Content of the user message. can be a mix of text and images oneOf: - type: string pattern: ^.[\s\S]*$ - type: array items: type: object oneOf: - $ref: '#/components/schemas/TextContentPart' - $ref: '#/components/schemas/ImageContentPart' - $ref: '#/components/schemas/BlobStorageReference' name: description: Name of the user this message is from type: string title: UserMessage x-fern-type-name: UserMessage ToolMessage: description: Tool message for the chat type: object required: - role - content - tool_call_id properties: role: description: Role of the message type: string enum: - tool content: description: Content of the tool call result oneOf: - type: string pattern: ^.[\s\S]*$ - $ref: '#/components/schemas/BlobStorageReference' tool_call_id: description: Unique identifier for the tool call type: string title: ToolMessage x-fern-type-name: ToolMessage DeveloperMessage: description: Developer message for the chat type: object required: - role - content properties: role: description: The role of the messages author, in this case developer. type: string enum: - developer content: description: The contents of the developer message. oneOf: - type: string pattern: ^.[\s\S]*$ - $ref: '#/components/schemas/BlobStorageReference' name: description: An optional name for the participant. type: string title: DeveloperMessage x-fern-type-name: DeveloperMessage ModelConfiguration: properties: provider: type: string title: Provider description: >- Name of the provider, must be one of the integration providers configured for the Gateway model: type: string pattern: ^[a-zA-Z](?:[a-zA-Z0-9\-.]{0,60}[a-zA-Z0-9])$ title: Model description: >- Name of the model to be used for generations. This model should be available in the provider parameters: allOf: - $ref: '#/components/schemas/Parameters' title: Parameters description: Parameters to pass to the model when generating extra_parameters: type: object title: Extra Parameters description: Arbitrary extra parameters to pass to the model when generating type: object required: - provider - model title: ModelConfiguration description: >- Add a model from Gateway along with parameters to be used for chat completion x-fern-type-name: ModelConfiguration ToolSchema: description: Schema defining a tool for the chat prompt type: object required: - type - function properties: type: description: Type of the tool type: string enum: - function function: $ref: '#/components/schemas/FunctionSchema' title: ToolSchema x-fern-type-name: ToolSchema MCPServerWithFQN: description: MCP server with FQN type: object required: - type - integration_fqn - enable_all_tools properties: type: description: The FQN of the MCP server type: string enum: - mcp-server-fqn integration_fqn: description: The FQN of the MCP server type: string enable_all_tools: description: Whether to enable all tools from the MCP server type: boolean default: false tools: description: The tools to enable from the MCP server type: array items: $ref: '#/components/schemas/MCPTool' title: MCPServerWithFQN x-fern-type-name: MCPServerWithFQN MCPServerWithURL: description: MCP server with URL type: object required: - type - url - enable_all_tools properties: type: description: The URL of the MCP server type: string enum: - mcp-server-url url: description: The URL of the MCP server type: string pattern: ^https?://.+ headers: description: The headers to send to the MCP server type: object additionalProperties: type: string enable_all_tools: description: Whether to enable all tools from the MCP server type: boolean default: false tools: description: The tools to enable from the MCP server type: array items: $ref: '#/components/schemas/MCPTool' title: MCPServerWithURL x-fern-type-name: MCPServerWithURL Guardrails: description: Guardrails type: object properties: input_guardrails: description: >- List of guardrail names to apply to user input messages before processing. type: array items: type: string output_guardrails: description: >- List of guardrail names to apply to AI response messages before returning to user. type: array items: type: string title: Guardrails x-fern-type-name: Guardrails JsonObjectResponseFormat: description: Simple JSON mode response format type: object required: - type properties: type: description: Type of response format type: string enum: - json_object title: JsonObjectResponseFormat x-fern-type-name: JsonObjectResponseFormat JsonSchemaResponseFormat: description: Structured JSON response format with schema validation type: object required: - type - json_schema properties: type: description: Type of response format type: string enum: - json_schema json_schema: $ref: '#/components/schemas/JsonSchema' title: JsonSchemaResponseFormat x-fern-type-name: JsonSchemaResponseFormat WeightBasedLoadBalancing: description: Weight-based Load Balancing type: object required: - type - load_balance_targets properties: type: description: Routing Type type: string enum: - weight-based-routing load_balance_targets: description: List of targets for load balancing with weights type: array items: $ref: '#/components/schemas/LoadBalanceTarget' sticky_routing: $ref: '#/components/schemas/StickyRouting' title: WeightBasedLoadBalancing x-fern-type-name: WeightBasedLoadBalancing LatencyBasedLoadBalancing: description: Latency-based Load Balancing type: object required: - type - load_balance_targets properties: type: description: Routing Type type: string enum: - latency-based-routing load_balance_targets: description: List of targets for latency-based load balancing type: array items: $ref: '#/components/schemas/LatencyBasedLoadBalanceTarget' title: LatencyBasedLoadBalancing x-fern-type-name: LatencyBasedLoadBalancing PriorityBasedLoadBalancing: description: Priority-based Load Balancing type: object required: - type - load_balance_targets properties: type: description: Routing Type type: string enum: - priority-based-routing load_balance_targets: description: List of targets for priority-based load balancing type: array items: $ref: '#/components/schemas/PriorityBasedLoadBalanceTarget' title: PriorityBasedLoadBalancing x-fern-type-name: PriorityBasedLoadBalancing SemanticCacheConfig: description: Semantic Cache Configuration type: object required: - type - similarity_threshold - ttl properties: type: description: Cache type type: string enum: - semantic similarity_threshold: description: Similarity threshold for semantic matching (between 0 and 1) type: number minimum: 0 exclusiveMinimum: true maximum: 1 namespace: description: Cache namespace (defaults to 'default' if not provided) type: string ttl: description: Time-to-live for cached entries in seconds (max 3 days) type: number minimum: 0 exclusiveMinimum: true maximum: 259200 default: 3600 title: SemanticCacheConfig x-fern-type-name: SemanticCacheConfig ExactMatchCacheConfig: description: Exact Match Cache Configuration type: object required: - type - ttl properties: type: description: Cache type type: string enum: - exact-match namespace: description: Cache namespace (defaults to 'default' if not provided) type: string ttl: description: Time-to-live for cached entries in seconds (max 3 days) type: number minimum: 0 exclusiveMinimum: true maximum: 259200 default: 3600 title: ExactMatchCacheConfig x-fern-type-name: ExactMatchCacheConfig MCPServerToolDetails: description: >- Mapping configuration for tool calls to MCP server integration and tool names type: object required: - mcp_server_integration_id - tool_name properties: mcp_server_integration_id: description: The integration ID of the MCP server type: string tool_name: description: The name of the tool in the MCP server type: string title: MCPServerToolDetails x-fern-type-name: MCPServerToolDetails LoggingConfig: description: Logging configuration for the chat prompt type: object required: - enabled properties: enabled: description: Whether logging is enabled for the chat prompt type: boolean default: true tracing_project_fqn: description: The FQN of the tracing project where the logs will be stored type: string title: LoggingConfig x-fern-type-name: LoggingConfig SubAgent: description: Sub agent configuration for the chat prompt type: object required: - name properties: name: description: Name of the sub agent type: string title: SubAgent x-fern-type-name: SubAgent BlobStorageReference: properties: type: type: string enum: - blob-storage title: Type path: type: string pattern: ^.{1,}$ title: Path description: Path to the content in blob storage mime_type: allOf: - $ref: '#/components/schemas/MimeType' description: MIME type of the content type: object required: - type - path - mime_type title: BlobStorageReference description: A reference to content offloaded to blob storage x-fern-type-name: BlobStorageReference TextContentPart: description: Text content for the message type: object required: - type - text properties: type: description: Type of the content part type: string enum: - text text: description: Text content for the message oneOf: - type: string pattern: ^.[\s\S]*$ - $ref: '#/components/schemas/BlobStorageReference' title: TextContentPart x-fern-type-name: TextContentPart RefusalContentPart: type: object required: - type - refusal properties: type: description: Type of the content part type: string refusal: description: Reason type: string title: RefusalContentPart x-fern-type-name: RefusalContentPart ToolCall: type: object required: - type - id - function properties: type: description: Type of the tool call type: string enum: - function id: description: Unique identifier for the tool call type: string function: $ref: '#/components/schemas/Function' title: ToolCall x-fern-type-name: ToolCall ImageContentPart: description: Image content for the message type: object required: - type - image_url properties: type: description: Type of the content part type: string enum: - image_url image_url: description: Image URL linking to the image type: object required: - url properties: url: description: Image URL linking to the image oneOf: - type: string pattern: "^\b((https?://)?(?:www\\.)?[-a-zA-Z0-9@:%._\\+~#=]{1,256}\\.[a-zA-Z0-9()]{1,6}(?:[-a-zA-Z0-9()@:%_\\+.~#?&/=]*)|data:image/[a-zA-Z]+;base64,[a-zA-Z0-9+/=\\s]+)$" - $ref: '#/components/schemas/BlobStorageReference' detail: description: Details type: string title: ImageContentPart x-fern-type-name: ImageContentPart Parameters: properties: max_tokens: type: integer title: Max Tokens temperature: type: number title: Temperature top_k: type: number title: Top K top_p: type: number title: Top P stop: anyOf: - type: array items: type: string - type: string title: Stop type: object title: Parameters description: Parameters to pass to the model when generating x-fern-type-name: Parameters FunctionSchema: description: Schema defining a function for tool calls type: object required: - name properties: name: description: Name of the function type: string description: description: Description of the function type: string parameters: description: Parameters schema for the function type: object strict: description: Indicates if the function should be called strictly type: boolean default: false title: FunctionSchema x-fern-type-name: FunctionSchema MCPTool: description: Tool configuration for MCP server type: object required: - name properties: name: description: The name of the tool as it appears in the MCP server type: string title: MCPTool x-fern-type-name: MCPTool JsonSchema: description: JSON schema for structured output type: object required: - name - schema properties: name: description: Name of the schema type: string schema: description: JSON schema object defining the expected structure type: object title: JsonSchema x-fern-type-name: JsonSchema LoadBalanceTarget: description: Weight-based Target type: object required: - target - weight properties: target: description: Target model or provider FQN type: string weight: description: >- Relative weight for routing decisions (higher values mean more traffic) type: integer retry_config: $ref: '#/components/schemas/RetryConfig' fallback_status_codes: description: >- Status Codes for which the request will fallback to other targets. If the status code is not present in fallback_status_codes, it fails immediately. type: array items: type: string default: - '401' - '403' - '404' - '408' - '429' - '500' - '502' - '503' fallback_candidate: description: >- Whether this target is a fallback candidate. If set to false, this model will not be considered as a fallback option for targets of this load-balance-rule type: boolean default: true override_params: description: Optional parameters to override in the request type: object headers_override: $ref: '#/components/schemas/HeadersOverride' metadata_match: description: >- Optional metadata key-value pairs that must match incoming request metadata headers for this target to be considered for routing. type: object additionalProperties: type: string title: LoadBalanceTarget x-fern-type-name: LoadBalanceTarget StickyRouting: description: Sticky Routing type: object required: - ttl_seconds - session_identifiers properties: ttl_seconds: description: Sticky mapping TTL in seconds type: integer minimum: 1800 maximum: 86400 session_identifiers: description: List of identifier selectors used to build sticky key type: array items: $ref: '#/components/schemas/StickySessionIdentifier' title: StickyRouting x-fern-type-name: StickyRouting LatencyBasedLoadBalanceTarget: description: Latency-based Target type: object required: - target properties: target: description: Target model or provider FQN type: string retry_config: $ref: '#/components/schemas/RetryConfig' fallback_status_codes: description: >- Status Codes for which the request will fallback to other targets. If the status code is not present in fallback_status_codes, it fails immediately. type: array items: type: string default: - '401' - '403' - '404' - '408' - '429' - '500' - '502' - '503' fallback_candidate: description: >- Whether this target is a fallback candidate. If set to false, this model will not be considered as a fallback option for targets of this load-balance-rule type: boolean default: true override_params: description: Optional parameters to override in the request type: object headers_override: $ref: '#/components/schemas/HeadersOverride' metadata_match: description: >- Optional metadata key-value pairs that must match incoming request metadata headers for this target to be considered for routing. type: object additionalProperties: type: string title: LatencyBasedLoadBalanceTarget x-fern-type-name: LatencyBasedLoadBalanceTarget PriorityBasedLoadBalanceTarget: type: object required: - target - priority properties: target: description: Target model or provider FQN type: string priority: description: >- Priority for the target, Lower the number, higher the priority (0 is the highest priority) type: integer minimum: 0 default: 0 sla_cutoff: $ref: '#/components/schemas/SLACutoff' retry_config: $ref: '#/components/schemas/RetryConfig' fallback_status_codes: description: >- Status Codes for which the request will fallback to other targets. If the status code is not present in fallback_status_codes, it fails immediately. type: array items: type: string default: - '401' - '403' - '404' - '408' - '429' - '500' - '502' - '503' fallback_candidate: description: >- Whether this target is a fallback candidate. If set to false, this model will not be considered as a fallback option for targets of this load-balance-rule type: boolean default: true override_params: description: Optional parameters to override in the request type: object headers_override: $ref: '#/components/schemas/HeadersOverride' metadata_match: description: >- Optional metadata key-value pairs that must match incoming request metadata headers for this target to be considered for routing. type: object additionalProperties: type: string title: PriorityBasedLoadBalanceTarget x-fern-type-name: PriorityBasedLoadBalanceTarget MimeType: type: string enum: - text/plain - application/json - image/png - image/jpeg - application/x-directory title: MimeType description: MIME type of the content x-fern-type-name: MimeType Function: type: object required: - name - arguments properties: name: description: Name of the tool/function being called type: string arguments: description: Arguments passed to the tool/function type: string title: Function x-fern-type-name: Function RetryConfig: type: object required: - attempts properties: attempts: description: Number of attempts to retry the request type: integer minimum: 0 delay: description: Delay between retries in milliseconds type: integer minimum: 0 exclusiveMinimum: true default: 100 on_status_codes: description: Status codes to retry on type: array items: type: string default: - '429' - '500' - '502' - '503' title: RetryConfig x-fern-type-name: RetryConfig HeadersOverride: description: Headers Override type: object properties: remove: description: List of headers to remove from upstream request type: array items: type: string set: description: Headers to add or overwrite in upstream request type: object additionalProperties: type: string title: HeadersOverride x-fern-type-name: HeadersOverride StickySessionIdentifier: description: Sticky Session Identifier type: object required: - source - key properties: source: description: Source from which session identifier value will be extracted type: string enum: - headers - metadata key: description: Key to read from selected source type: string pattern: ^[!#$%&'*+.^_`|~0-9A-Za-z-]{1,128}$ title: StickySessionIdentifier x-fern-type-name: StickySessionIdentifier SLACutoff: type: object properties: time_per_output_token_ms: description: >- Time per output token threshold in milliseconds. If time per output token exceeds this value, the target will be marked as unhealthy type: integer minimum: 0 exclusiveMinimum: true time_to_first_token_ms: description: >- Time to first token threshold in milliseconds. If time to first token exceeds this value, the target will be marked as unhealthy type: integer minimum: 0 exclusiveMinimum: true title: SLACutoff x-fern-type-name: SLACutoff securitySchemes: jwt: type: http scheme: bearer bearerFormat: JWT ````