Local provider keys
Keep upstream provider credentials in gateway config while clients use gateway-facing tokens.
Adoption fit
Start with the operational controls your team actually needs.
AI Model Gateway is useful when the gateway is part of your infrastructure boundary. It keeps provider keys, routing policy, request telemetry, audit records, diagnostics, and rollback workflows inside your environment instead of outsourcing the control plane to a hosted broker.
Routing policy
Centralize model routing, fallback preferences, health checks, and cooldown behavior in one local runtime.
Operational telemetry
Review traffic, latency, request logs, provider health, diagnostics, and cost signals from the Admin UI.
Change safety
Preview, diff, validate, config publish, audit, and rollback gateway changes before they affect traffic.
Shortest proof
Prove fallback behavior before routing real traffic.
The executable provider fallback demo starts fake OpenAI-compatible upstreams. The primary returns 429, the gateway serves the fallback provider, rewrites the forwarded model, and records the fallback route mode.
go test ./examples/provider-fallback -run TestProviderFallbackDemo -v
Checklist
Walk through the adoption decision in one pass.
- Fit. Confirm local control over provider keys, routing policy, telemetry, audit records, and admin access is a requirement.
- Run. Start from the release archive, Docker Compose path, or source build, then open the Admin UI and health endpoints.
- Connect. Point Codex CLI, Claude Code, OpenClaw, OpenAI SDK clients, or curl smoke tests at the gateway endpoint.
- Operate. Test provider fallback, request logs, provider probes, config publish, rollback, diagnostics, and update recovery.
Scope
Use it for local operations control, not as a hosted model marketplace.
The gateway supports OpenAI-compatible, Anthropic-compatible, and Responses-style entry points, but each client and upstream should be smoke-tested before production use. It does not claim to replace every hosted platform feature, account workflow, or provider-specific capability.
Adopt when
- Provider keys, routing policy, telemetry, logs, and audit records need to stay local.
- Multiple clients should use one internal gateway URL and gateway client keys.
- Operators need provider fallback, health probes, diagnostics, and replay evidence.
- Config changes need preview, diff, publish history, rollback, and reviewable operations traces.
Check alternatives when
- You want a hosted model marketplace to manage provider access, billing, and routing.
- You only need a small SDK wrapper inside one application.
- You require exact provider-specific platform behavior without compatibility testing.
- You do not want to operate a gateway runtime or Admin UI in your environment.
Review evidence
Check installability, quality evidence, and security before adopting it.
Quality evidence
Review CI gates, local reproduction commands, runtime smoke checks, feature proof points, and current capability boundaries.
Open quality evidenceSecurity and trust model
Inspect admin auth, same-origin browser writes, provider-key handling, SSRF defenses, telemetry sensitivity, local files, and update trust.
Open security modelComparison guide
Compare the project against broader gateway, observability, and hosted routing options before committing operational ownership.
Open gateway comparisonDecision
Run the checklist, then decide whether it earns a star.
Start with one non-production request, the fallback demo, and the review evidence. If AI Model Gateway fits your self-hosted LLM gateway workflow, Star on GitHub so similar operators can discover it.