Bifrost
by maximhq
Bifrost is a high-throughput AI gateway fronting 20+ LLM providers behind one OpenAI-compatible API, with a built-in MCP gateway that adds tool-calling to any connected model.
About
Bifrost is built as an AI gateway first: a single OpenAI-compatible endpoint in front of 20+ providers (OpenAI, Anthropic, Bedrock, Vertex, Azure, Groq, Mistral, Ollama, and more), with automatic failover, load balancing, semantic caching, budget controls, and Prometheus/tracing built in, claiming sub-100µs overhead at 5k RPS.
On top of that routing layer, Bifrost runs as an MCP gateway: it registers external MCP servers — filesystem, web search, databases — and turns a plain chat model with no native tool-calling of its own into one that can call those tools through Bifrost, with a visual web UI for wiring up and monitoring which MCP servers are attached to which model traffic.
Key features
- Single OpenAI-compatible API in front of 20+ LLM providers
- Automatic failover and load balancing across providers with no configuration downtime
- Semantic response caching to cut redundant calls and latency
- MCP gateway that adds tool-calling to models that don't natively support it
- Hierarchical budget management, usage tracking, and rate limiting
- Native Prometheus metrics and distributed tracing for observability
Use cases
- Routing traffic across multiple LLM providers with automatic failover instead of hardcoding one vendor
- Giving a tool-calling-incapable model access to filesystem, search, or database MCP tools via the gateway
- Cutting redundant LLM spend with semantic caching on repeated or similar prompts
- Enforcing per-team or per-key budget limits across all providers from one control plane
Available tools
MCP gateway
Registers external MCP servers and exposes their tools to any model routed through Bifrost, even ones without native tool-calling.
Adaptive load balancer
Distributes requests across configured provider keys/instances and fails over automatically.
Semantic cache
Serves cached responses for semantically similar requests to cut cost and latency.
Governance/budget controls
Enforces hierarchical spending and rate limits per team, key, or model.
Frequently asked questions
Does Bifrost let a model without tool-calling use MCP tools?
Yes — its MCP gateway registers external MCP servers and exposes their tools through Bifrost's own interface, so even a plain chat-completion model gains tool-calling behavior as far as the caller is concerned.
Is it fully open source?
The core gateway — routing, failover, caching, the OpenAI-compatible API — is open source and self-hostable via npx or Docker; clustering and some guardrails are positioned as enterprise-tier additions.