Routing
Open Inference → Routes. Model routes define how a client model (and optional route group) maps to model + vendor + upstream model name, and in which priority order to try them. Multiple routes in the same group enable automatic failover.
The list supports rich filters; the current filter set is reflected in the browser address bar so you can share a link to the same view.
What to do in Admin
Section titled “What to do in Admin”- Open Routes.
- Create a route: pick model, vendor, upstream model name, protocol (must match the base URL you configured for that protocol on the provider), route group (often
default), priority (lower tries first), status (only active routes serve production traffic). - Pricing: override defaults with the tier editor when needed (follow the UI).
- After saving, use Playground for a single-route upstream check; use Simulator to exercise the full Proxy path with a real
sk-key.
Client impact
Section titled “Client impact”Clients only send model and protocol-specific payloads — no hard-coded vendor details. Prefer changing routes when switching upstreams.
Playground probes one route upstream and does not mirror full multi-route failover inside the gateway; validate failover with real Proxy calls or Simulator.