What is MCP ? How Does it Work ?

Core Components of MCP

MCP interactions are driven by four key primitives: tools, resources, prompts, and the session layer. These components abstract away low-level complexity and provide a clean interface for models to reason, act, and interact with real-world systems in a modular and predictable way.

Tools

Tools are executable functions exposed by the server and callable by the model via the client. Each tool is defined with input parameters, output schema, and a description of its purpose. This metadata is used by the model to determine when and how to use a tool. Examples include creating support tickets, sending Slack messages, or initiating workflows. Tools may be synchronous or asynchronous, depending on the backend implementation. The model decides tool invocation timing based on the current context and task intent.

Resources

Resources are read-only data that servers expose to provide context to the model. These could be database records, documents, log entries, or search results. Clients request resources when the model needs to look up or retrieve data before taking an action. Servers define what is available, and the client handles formatting it for inclusion in the model’s prompt. Resources are critical for grounding agent responses in up-to-date and relevant information.

Prompts

Prompts are predefined templates or interaction flows registered by the server. They serve as structured scaffolds that can guide the model’s behavior in specific contexts. For instance, a prompt might define the layout for a bug report, escalation email, or onboarding guide. Prompts can be triggered by the user or invoked programmatically. Their role is to enforce consistency, reduce hallucination, and accelerate structured generation in high-stakes workflows.

Session Layer (JSON-RPC)

All communication between clients and servers happens over JSON-RPC 2.0. This protocol supports stateful, bidirectional messaging and is transport-flexible. It works over STDIO, HTTP/SSE, or WebSockets. The session layer enables request-response calls, event notifications, capability registration, and long-running operations. By adhering to JSON-RPC, MCP ensures strong typing, predictable communication patterns, and compatibility across languages and platforms. This session model is what makes dynamic tool discovery and runtime interaction possible.

Note: Tools, resources, prompts, and the session layer form the foundation of every MCP interaction. They allow models to reason, act, and retrieve context using structured, predictable primitives, enabling real-time integration with external systems without sacrificing safety or flexibility.

How does MCP work? 

MCP operates through a structured five-step flow that enables language models to discover, invoke, and interact with tools and data sources in real time. Here's how the protocol works under the hood:

Step 1: Host Initializes Clients

The host application starts the process by launching one or more MCP clients, each configured to connect to a specific MCP server. The host is also responsible for managing user authorization, enforcing access control policies, and maintaining context across multiple tool interactions.

Step 2: Clients Discover Server Capabilities

Each client initiates a handshake with its assigned server. During this exchange, the server advertises its supported tools, resources, and prompts, complete with descriptions, input parameters, and return types. The client then packages this metadata into the model’s context so the model knows what it can use.

Step 3: Model Makes Decisions

With capability data now embedded in its prompt, the model reasons over the available tools and current user input. If the task requires action or information retrieval, the model decides which tool or resource to invoke, formulates the appropriate parameters, and sends the request to the client.

Step 4: Client and Server Exchange Data

The client forwards the model’s request to the server using JSON-RPC. The server executes the function or fetches the data, then returns a structured response. The client delivers the result back to the model, which uses it to continue processing or make further decisions.

Step 5: Host Manages Feedback and Session Flow

The host application monitors the full loop, managing ongoing session state, logging tool usage, enforcing rate limits, and ensuring the agent stays within its scope. This step ensures safety, traceability, and observability of AI-driven actions.

Example: AI Agent Handling a Support Ticket

This diagram shows how an AI agent uses MCP to discover tools, invoke actions, and handle tickets across multiple servers in real time. It visualizes the full end-to-end flow from user input to system response.

Consider an AI support agent embedded in a helpdesk application. When the host application starts, it initializes two MCP clients: one connected to a Zendesk MCP server and another linked to an internal escalation system. Each client performs a capability handshake with its respective server. The Zendesk server advertises tools such as getTicketById and updateTicketStatus, while the escalation server exposes escalateTicket.

This metadata is passed into the model’s context. When a user types the instruction, “Escalate ticket #1289 if it’s marked urgent,” the model interprets the request and determines that it needs to fetch the ticket first. It calls getTicketById("1289") via the Zendesk client. The server responds with the ticket details, indicating the priority is “urgent.” Based on that, the model decides to escalate the issue and calls escalateTicket("1289") through the second client.

Both calls are executed over JSON-RPC. The clients forward the requests to their respective servers, which process the functions and return structured responses. These responses, ticket metadata, and escalation confirmation are routed back to the model and incorporated into its context. The host then finalizes the loop by logging the interactions, validating access policies, and updating the UI with the output: “Ticket #1289 escalated successfully.”

Why Do We Need MCP?

As AI agents become more capable, their biggest limitation isn’t reasoning — it’s access. Most language models operate in isolation, without any structured way to interact with external tools or data sources. Developers end up hardcoding API calls, building brittle wrappers, or managing one-off integrations that don’t scale well. This not only slows down development but also introduces security and maintainability issues.

MCP addresses this problem by standardizing how AI agents discover, reason over, and invoke tools. It defines a clean protocol that allows systems to expose their capabilities, such as APIs, databases, or file systems, in a structured, model-friendly way. Agents, in turn, can dynamically learn what actions are available, decide when to use them, and safely trigger them using JSON-RPC.

Real-World Use Case

Consider an AI assistant embedded in an enterprise HR platform. Using MCP, the assistant can connect to a payroll server, an internal knowledge base, and an employee directory, each as an MCP server. When a manager types, “Grant Rahul five days of leave and notify HR,” the model dynamically calls the appropriate tools exposed by these servers: one to update leave status, another to send a Slack message to HR, and a third to log the request in the internal system — all in real time, with zero custom code in the model.

This kind of orchestration would be fragile and expensive to build manually. With MCP, it becomes modular, repeatable, and secure. As organizations deploy more tools and services, MCP ensures AI agents can interact with them safely and intelligently, without needing new integrations for every change.

Why is MCP More Than Just Function Calling?

Function calling made it possible for language models to interact with tools in a structured way. You define a tool, its parameters, and its purpose, and the model can invoke it by returning a JSON object. This works well for simple use cases, especially when the number of tools is small and static. But function calling has critical limitations: it assumes you know all tools up front, that each model is tightly coupled to them, and that the infrastructure around them is either hardcoded or fragile.

MCP addresses these gaps by introducing a full-fledged protocol, not just a format. It formalizes the interaction between AI agents and external systems using a host–client–server architecture and structured communication over JSON-RPC. Rather than embedding tool definitions directly into the model’s prompt, MCP allows models to dynamically discover what tools are available at runtime, invoke them securely, and receive structured responses, all through a standard interface.

Real-World Example

Imagine an AI assistant embedded inside a retail operations dashboard. With function calling, the model would need to be manually configured with access to tools like getInventory, updatePrice, and notifyWarehouse. Every tool must be hardwired into the agent’s system prompt. But with MCP, you can connect to different servers — one for inventory, one for pricing, one for logistics, and the agent will discover all exposed tools automatically. No hardcoding, no redeploying.

Why It Matters

• Separation of concerns: Developers can build servers once and share them across models and teams
  
• Dynamic orchestration: Models can reason over available capabilities and adapt workflows on the fly
  

Function calling was a necessary first step. MCP builds on that foundation with the infrastructure, protocol, and flexibility needed for building robust, production-grade agent systems.

How Does The Truefoundry Gateway Implement The MCP Server? 

Unified MCP Server Registry

TrueFoundry’s MCP Gateway provides a single portal to discover and register all your MCP Servers, whether internal, on-premises, hybrid, or third-party, under organized “MCP Server Groups” that isolate environments (for example, dev versus prod) and enforce RBAC-driven approval flows for governance and visibility. Out-of-the-box, the Gateway includes prebuilt connectors for enterprise tools such as Slack, Confluence, Datadog, Sentry, and GitHub, enabling zero-code integration into LLM agent workflows. For custom needs, you can register any internal or proprietary API in minutes to make it instantly discoverable and usable by LLM-powered agents without modifying your SDK.

Secure Authentication and SDK Integration

The Gateway supports multiple authentication schemes, including no authentication, header-based tokens, and OAuth2 client credentials, with federated SSO via identity providers such as Okta and Azure AD. OAuth2 credentials are stored securely in TrueFoundry’s built-in secrets manager and injected automatically on proxy requests, centralizing credential management and reducing risk. 

‍

Developers can programmatically interact with registered MCP Servers using the official Python and TypeScript SDKs, which handle authorization end-to-end, or via the AI Gateway Playground, an intuitive interface where you add MCP Servers, select tools, and run natural language prompts that invoke remote services in real time. Both the Playground and SDK provide “API Code Snippet” buttons to generate ready-to-use integration boilerplate in your preferred language.

Observability and Load Balancing

TrueFoundry’s AI Gateway embeds enterprise-grade observability and load balancing. Requests are routed based on weight or latency to the healthiest endpoints, while all MCP invocations are trace-logged and audited in governance dashboards, ensuring compliance and performance monitoring. In a recent blog, the team describes how the Gateway acts as the central control plane for modern generative AI infrastructure, unifying LLMs, MCP Servers, and agent-to-agent protocols under one interface for low latency, high reliability, and seamless scalability. This approach empowers natural language orchestration across enterprise systems, enabling end-to-end automation—such as creating Jira issues from Slack alerts—without writing integration code.

Conclusion

MCP is not just another integration layer; it is the core infrastructure for the future of AI. As models evolve into agents that need to reason, act, and interact with tools, the demand for secure, modular, and scalable access becomes critical. Function calling helped start this journey, but it falls short at scale.

MCP solves this with a clean host, client, and server architecture, enabling dynamic tool discovery, structured invocation, and runtime flexibility. It turns scattered integrations into a unified, extensible ecosystem. Whether you're building agents, assistants, or automation pipelines, MCP provides the foundation for real-world utility. It brings structure and security to AI workflows, allowing teams to build confidently, without compromise.

FAQ’s

1. How does MCP differ from basic function calling? 

MCP extends function calling by introducing a full protocol with distinct host, client, and server roles. Instead of hardcoding tools into prompts, MCP lets models discover available actions at runtime, invoke them securely via JSON-RPC, and handle structured responses, all without redeploying or rewriting integration code.

2. What are the core components of MCP? 

MCP comprises three layers: the host, which enforces security policies, manages session state, and aggregates context; the client, which negotiates capabilities with servers, handles JSON-RPC messaging, and routes requests and responses; and the server, which wraps external tools or data sources, exposing actions, resources, and prompts through a standardized interface.

3. How do I secure an MCP deployment? 

Secure MCP by treating each server as an OAuth resource server, validating JSON Web Tokens for authentication, applying scoped permission checks, and rotating keys frequently. Add single sign-on integration, enforce end-to-end encryption for all traffic, implement rate limiting to prevent abuse, and centralize audit logs in a SIEM system for real-time monitoring and compliance reporting.

4. Which deployment model should I choose for MCP servers? 

Choose centralized deployment when you need simple management and cost savings in a single region. Opt for remote cloud to minimize latency and benefit from auto-scaling near critical workloads. Use a hybrid model to keep sensitive data on-premise while leveraging cloud resources for non-critical tasks, aligning with your compliance and performance needs.

‍