Now routing across every major model provider

One endpoint for AI requests.
Clearer control.

Send OpenAI-compatible chat completions and embeddings through one API. MLRouter picks the model, absorbs provider failures, enforces the limits you set, and shows you exactly what happened on every request.

View docs
✓ Drop-in OpenAI SDK replacement✓ Streaming with Server-Sent Events✓ Automatic fallback on provider failure✓ Per-key spend caps and guardrails
models in the catalog
270models in the catalog
model providers
46model providers
with tool calling
192with tool calling
largest context window
2Mlargest context window
/v1MLRouterGATEWAYOpenAIAnthropicGoogleMistralHuggingFace

Why teams switch

Stop betting your product on one model

Every provider you integrate directly is a contract, an outage, and a migration waiting to happen. MLRouter turns that into a setting.

1

One integration, every model

Add 46 providers' worth of models to your product once. Switching between them later is a setting, not a sprint.

0

Lines of code to migrate

MLRouter speaks the API your team already uses. Point it at a new base URL and existing applications keep working unchanged.

Auto

Failover nobody gets paged for

When a provider degrades, traffic moves to a healthy one mid-flight. Your users see a response, not an incident.

Per-key

Spend you can actually govern

Give every team, product, and environment its own budget, limits, and permitted models — and see the cost of each request.

Capabilities

The parts of MLRouter

Every surface lives in the same workspace: request routing, model analysis, key management, session services, policy enforcement, and operational visibility.

Smart routing

Select models by cost, latency, weighted pools, prefix, or automatic routing. Configure fallback behavior when a primary route cannot serve a request.

Cost · Latency · Weighted · Prefix · Auto

TokenDrift

Compare token counts and estimated costs across models, check tokenizer drift, and assess migration safety using representative prompts before you switch.

Compare · Drift checks · Migration safety

Context memory

Conversations keep their context across turns, interrupted responses pick up where they stopped, and sensitive requests can skip storage entirely.

Sessions · Work memory · Stream resumption

MCP gateway & builder

Build and publish MCP servers, then expose MLRouter's own tools over MCP with per-category scopes: chat, memory, billing, observability, team, and webhooks.

Build · Publish · Scoped tool access

Observability

Inspect request logs, token usage, cache metrics, provider health, and circuit-breaker state from one workspace — every request carries a trace ID.

Request logs · Analytics · Provider health

Governance

Manage API keys, roles, routing strategies, spend caps, rate limits, guardrails, and provider data policies without handing provider credentials to callers.

Keys · Roles · Limits · Policies

Guardrails & classifiers

Enforce allowed and blocked models, per-request token ceilings, and spend limits. Classifiers inspect prompts and responses for PII, injection attempts, and unsafe content.

Allow/block · Token ceilings · Injection checks

Presets

Reusable configuration templates that pin a system prompt, request defaults, and an allowed model subset, so every team starts from a known configuration.

System prompts · Defaults · Allowed models

Self-hosted models

Register an OpenAI-compatible endpoint your organization operates. Its model IDs join the catalog and route ahead of a hosted model sharing the same ID.

Bring your own endpoint · Catalog priority

Routing

Five strategies, one decision

A key's strategy decides which model serves a request — but only after the candidate set has been narrowed to models that are actually allowed to see it.

cost

Cheapest model that can still serve the request

Scores candidates on input and output price against your estimated token count, so the choice reflects the actual shape of the request rather than a headline rate.

latency

Fastest healthy provider

Uses observed p50 and p95 latency per provider, continuously updated from real traffic, and skips providers whose circuit breaker is open.

weighted

Split traffic across a pool

Distributes requests over an admin-defined pool by weight — useful for canarying a new model or splitting load across two providers.

prefix

Send exactly what was asked for

Passes the requested model through untouched. This is the default for a key with no strategy configured.

auto

Balance quality against cost

A quality/cost dial picks from the models a key is allowed to use. Requesting "auto" as the model works under every strategy.

Applied before any strategy runs

These filters shape the candidate set. If they eliminate every model, the request fails with a reason header rather than quietly routing somewhere you didn't authorize.

Capability filter

Requests needing tool calling, vision, or long context only ever see models that declare them. If nothing qualifies, the request fails loudly instead of silently downgrading.

Model tier exclusions

Exclude free or preview tiers from automatic selection. These constrain what the router may pick on your behalf — naming a model explicitly still works.

Data policy filter

Per-provider policies such as no-training and zero retention. Providers that don't satisfy the required policies are removed from the candidate set before selection.

Compliance certifications

Route only through providers carrying the certifications you require — SOC 2, HIPAA, ISO 27001, or GDPR — with escalations reported in the response headers.

Context-window fallback

When a prompt outgrows the selected model's context window, routing escalates to a model that can hold it rather than returning a truncation error.

Circuit breakers

Providers that start erroring are tripped out of rotation and probed for recovery, so a degraded provider stops absorbing your traffic.

Governance

Every request follows your rules

Nine checks sit between your application and the provider. Any of them can stop a request before it costs you money, and all of them are on the record afterwards.

  1. 01

    Identify

    The key identifies the team, product, or environment behind the request, and the rules that apply to it.

  2. 02

    Throttle

    Traffic limits protect you from a runaway job burning a month of budget in an afternoon.

  3. 03

    Budget

    The cost is checked against the workspace balance before a provider is ever contacted.

  4. 04

    Screen

    Your policies run against the prompt: permitted models, size limits, and checks for sensitive or unsafe content.

  5. 05

    Shortlist

    Only models that meet the request's needs and your compliance requirements stay in the running.

  6. 06

    Choose

    Your routing rule picks from that shortlist, skipping any provider currently having a bad day.

  7. 07

    Deliver

    The request is translated to the chosen provider's format and streamed back to your application.

  8. 08

    Recover

    If that provider fails mid-request, the next one in line takes over without your users noticing.

  9. 09

    Account

    Actual usage settles against your balance, and the whole decision is recorded for you to review.

Request controls

Make routing choices explicit

MLRouter keeps application requests and workspace controls separate: your client sends a familiar API request, while your team defines the rules around it.

Read the API reference

Routing strategy

Choose cost, latency, weighted, prefix, or automatic routing. Request-scoped overrides are also supported.

Fallback behavior

Configure fallback chains, or supply a request-specific model list when a primary route cannot complete a request.

Key-level limits

Set a routing strategy, spend cap, rate limit, and guardrail policy for an API key without exposing provider credentials to callers.

Request explanation

Review the resolved model, routing strategy, estimated cost, and trace identifier in response headers and request logs.

Self-hosted models

Register an organization-operated OpenAI-compatible endpoint. Its active model ID appears in the catalog and is routed ahead of a hosted model with the same ID.

Getting started

Follow a clear request path

01

Use your existing client

base_url: "https://mlrouter.com/v1"
api_key: "mlr-your-api-key"

Set the MLRouter base URL and API key, then send OpenAI-shaped chat completion requests from your application.

02

Set workspace controls

prod-key-alphaactive
StrategyLatency
Spend capPer-key limit
sk-mlr-••••••••••••••••••

Create API keys and configure routing strategy, spend caps, rate limits, guardrails, and fallback behavior.

03

Inspect each request

Resolved modelSelected
Routing strategyLatency
Request recordAvailable

Use response headers and request logs to review the resolved model, routing strategy, estimated cost, provider, and latency.

OpenAI-compatible

Change the base URL, keep the code

from openai import OpenAI
 
client = OpenAI(
base_url="https://mlrouter.com/v1",
api_key="mlr-your-api-key-here",
)
 
response = client.chat.completions.create(
model="auto", # let the router choose
messages=[{"role": "user", "content": "Hello!"}],
)
 
print(response.choices[0].message.content)

Nothing to rewrite

Your existing code, SDKs, and prompts keep working. Two lines of configuration is the whole migration.

Every answer is accounted for

Each response reports which model served it, why it was chosen, and what it cost — no extra instrumentation.

Change models without shipping

Move a workload to a cheaper or faster model from the dashboard. Production picks it up on the next request.

API surface

Endpoints your application already recognizes

Inference speaks OpenAI. Everything your team configures around it — keys, policies, presets, logs — is a first-class REST endpoint too, so the whole control plane is automatable.

Inference

  • POST/v1/chat/completions

    OpenAI-shaped chat requests, including streaming responses.

  • POST/v1/embeddings

    Vector embeddings through an OpenAI-compatible endpoint.

  • GET/v1/models

    Available models with catalog details and provider health.

Control

  • GET/v1/api-keys

    Create, scope, rotate, and revoke keys without touching provider credentials.

  • GET/v1/guardrails

    Policies for allowed models, token ceilings, and spend limits.

  • GET/v1/presets

    Reusable prompt and parameter templates.

  • GET/v1/classifiers

    Evaluators for PII, intent, and unsafe content.

Insight

  • GET/v1/requests

    Request log with routing decisions, usage, and cost per call.

  • GET/v1/providers/health

    Live provider scores, error rates, and latency percentiles.

  • GET/public/status

    Public provider status — no authentication required.

Data & compliance

Decide where prompts are allowed to go

Compliance requirements are a routing input, not a policy document. A request that cannot be served within your constraints is refused rather than downgraded.

Certification routing

Require SOC 2, HIPAA, ISO 27001, or GDPR coverage and route only through providers that carry it.

Provider data policies

Attach no-training and zero-retention requirements to provider keys, applied per organization.

No-store requests

Mark sensitive workloads so their content is processed and then discarded, never written to storage.

Regional preference

Express a region for providers that serve more than one, using measured regional latency baselines.

SSO enforcement

Require single sign-on for workspace access, with role-based permissions on every control endpoint.

Self-hosted deployment

Run the gateway against your own OpenAI-compatible endpoints when traffic must stay inside your network.

Pricing

Pay for what you use

Model usage is billed from workspace credits on every plan, including Free. A subscription only changes how often you can use the MLRouter-specific features listed below.

1 credit = $0.0001

$1 buys 10,000 credits.

Reserved, then settled

Estimated cost is held up front and reconciled against real token usage.

5% purchase fee

Applied when you buy credits. Token costs themselves are passed through.

Free

$0

Credits-based pay-as-you-go. Every routing and observability feature included.

5 uses / month of each gated feature

  • Unlimited routing, fallback, and observability
  • Full model catalog and API key management
  • Pay only for the tokens you actually use
Get started
Most popular

Pro

$20/month

For small teams and growing projects.

50 uses / month of each gated feature

  • Everything in Free
  • 10× the monthly allowance on gated features
  • All models included
Choose Pro

Unlimited

$50/month

For production workloads at scale.

No cap on gated features

  • Everything in Pro
  • Unlimited feature usage
  • Priority routing
Choose Unlimited

What the monthly allowance covers

Each of these has its own counter, reset at the start of every calendar month. Routing, fallback, observability, key management, and the model catalog are never capped.

  • TokenDrift comparisons
  • Guardrail policies
  • Presets
  • Prompt-injection checks
  • MCP session persistence
  • MCP Builder

Questions

Before you integrate

No. MLRouter speaks the OpenAI chat-completions and embeddings API, so pointing an existing OpenAI client at https://mlrouter.com/v1 with an MLRouter key is usually the whole change. Streaming, tool calls, and multi-modal inputs keep the same request shape.

Ship on every model, commit to none

Create a workspace, generate a key, and route your first request in minutes. No card required to start.

Compare plans