Routing

Open Inference → Routes. Model routes define how a client model (and optional route group) maps to model + vendor + upstream model name, and in which priority order to try them. Multiple routes in the same group enable automatic failover.

The list supports rich filters; the current filter set is reflected in the browser address bar so you can share a link to the same view.

OctaFuse Admin Routes page grouped by provider with route cards for each model. — The Routes page groups route cards by provider and model. One model can have active and fallback routes for priority and failover.

What to do in Admin

Open Routes.
Create a route: pick model, vendor, upstream model name, protocol (must match the base URL you configured for that protocol on the provider), route group (often default), priority (lower tries first), status (only active routes serve production traffic).
Billing: set base multipliers for charged to user and supplier cost relative to the catalog list price. For vendor peak / off-peak rates, add a Daily schedule (time-of-day multipliers; timezone comes from System config business timezone).
After saving, use Playground for a single-route upstream check; use Simulator to exercise the full Proxy path with a real sk- key.

Client impact

Clients only send model and protocol-specific payloads — no hard-coded vendor details. Prefer changing routes when switching upstreams.

Routing boundary: priority, weight, and sticky

Layer	Controls
Route `priority`	Which upstream route to try first in a `route_group` (lower first). Routes have no weight.
Provider key `weight`	After a route is chosen, weighted random among that Provider’s keys when headroom is nearly tied. See Providers.
Sticky	Opt-in on Models per protocol × route group. Keeps the same user on the same upstream key when possible to improve prompt cache hits; short-wait on soft rate limits; still fail over on hard upstream errors.

Note

Playground probes one route upstream and does not mirror full multi-route failover or sticky behavior; validate with real Proxy calls or Simulator. Image routes must use the OpenAI protocol — see Image generation / edit.