MCP in Production: Routing LLM Tool Calls Through an API Gateway

MCP in Production: Routing LLM Tool Calls Through an API Gateway Your LLM can now call tools. But who controls which tools it can call, how often, and with what credentials? Model Context Protocol (MCP) gives LLMs a standard way to discover and invoke external tools — databases, APIs, file systems, anything you expose as a tool server. The protocol is clean and simple: JSON-RPC 2.0 over HTTP. But "clean and simple" doesn't mean "production-ready." The moment you have multiple MCP servers, multiple LLM clients, and real users, you need the same infrastructure you'd need for any API: authentication, authorization, rate limiting, load balancing, failover, and observability. That's what an API gateway does. So we built one for MCP. The Problem Here's a typical MCP setup: LLM Client → MCP Server A (database queries) LLM Client → MCP Server B (email sending) LLM Client → MCP Server C (code execution) This works fine on a developer's laptop. In production, you need answers to: Who is allowed