OctaFuse 2.0: make complex routing visible
Published
AI gateway routing used to be a collection of Provider keys, priorities, weights, and policies spread across forms. Operators configured the fields and then reconstructed the real request path in their heads.
OctaFuse Gateway 2.0 made that path explicit:
Client request → Request Surface → Route Pool → Upstream Target → Provider
Surface describes the client protocol and operation. Pool groups upstreams that share failover behavior. Target represents the actual provider model to try. Configuration now sits on the stage where it takes effect.
Priority, strategy, and weight
Section titled “Priority, strategy, and weight”Priority chooses the tier, strategy orders a tier, and weight expresses preference inside it.
All Targets in the current tier are exhausted before the next tier is considered. Disabled Providers and Targets never enter the attempt plan. This makes patterns such as load sharing among primary accounts, cheaper fallbacks, and an expensive last-resort provider visible on one topology.
Admin also previews the result of a strategy without sending a real request and explains whether the effective policy comes from the Pool, model, global setting, or system default.
A Provider represents one upstream account
Section titled “A Provider represents one upstream account”Version 2.0 also moved to one API key per Provider. Multiple accounts at the same vendor become multiple Providers, matching the upstream entities shown in routing. Credentials, status, routing, and troubleshooting now refer to the same real account.
This release was a coordinated change to storage, runtime scheduling, and Admin UX. Its Surface → Pool → Target foundation enabled later Gemini routing consolidation and per-priority-tier policies.
Learn more about OctaFuse or explore the source:
If OctaFuse is useful to your project, consider giving the repository a Star. Your interest and feedback help us keep improving routing and the self-hosted experience.