AI Gateway & Control PlaneOctaFuse

AI capabilities are expanding beyond text models into image generation, Agent Tools, private models, and specialized services. Providers, compatible endpoints, coding plans, self-hosted services, and internal accounts each bring their own Base URLs, keys, quotas, pricing, and logs. Applications need something simpler: one stable entrypoint, controlled routing, clear budgets, and traceable request history.

OctaFuse is a self-hostable AI capability gateway and operations control plane. Unify official providers, third-party compatible endpoints, image generation and editing, speech transcription, Agent Tools, and private model services. Request Surface → Route Pool → Upstream Target topology keeps routing, failover, Provider circuits, budgets, billing, audit, and Admin APIs under your control. Clients still use one Gateway URL and one user key.

Application Scenarios

Individual developers

Consolidate model subscriptions, coding plans, image generation, Agent Tools, and backup upstreams behind one Gateway. Cursor, CLIs, scripts, and personal tools only need one Gateway URL and one user key.

AI apps and agents

Build against stable model IDs and common protocols. Move model swaps, canary routes, backup providers, degradation, and default parameters into the gateway layer.

Platform and IT teams

Create separate users and API keys for departments, projects, members, or customers, with budget periods, status, and metadata outside application services.

API proxy businesses

Connect multiple upstream Providers; represent multiple accounts at the same vendor as separate Providers. Issue downstream customer keys and manage each customer through request logs, cost lenses, and audit records.

Private and hybrid models

Put self-hosted compatible services, internal models, Ollama / vLLM paths, and public model providers behind one routing system with a consistent external surface.

Cost and reliability ops

Observe usage and failures by model, Provider, user, and reliability view, then tune Route Pool strategy, priority / weight, budgets, and time-of-day pricing.

Product Capabilities

Multi-protocol, multimodal entrypoint

Supports OpenAI Chat, Images, Audio Transcriptions, and Models; Anthropic Messages; and Gemini generateContent / streamGenerateContent. Portals can discover capabilities through public GET /catalog/models.

Image generation / edit

OpenAI-compatible POST /v1/images/generations and edits, with token-metered and per-image (per_image) catalog pricing. Playground / Simulator cover image smoke and billing logs.

Agent Tools

Extensible product APIs for agents (/v1/tools/*). Web Search supports Bocha, Tavily, CleverSee, and Tencent Cloud WSA; Web Fetch supports Firecrawl, Tavily Extract, and Jina Reader; Deep Search supports Firecrawl Search and Jina Search.

Surface → Pool → Target routing

Client requests resolve a protocol and operation Request Surface, then enter a Route Pool. Pools group Upstream Targets by priority and order each tier with strategy plus weight before automatic failover.

Provider and model presets

One-click Admin import for a large preset catalog: official model vendors plus aggregation platforms and Coding / Token Plans, with Base URLs and catalog pricing prefilled so you are not hunting docs to maintain endpoints by hand.

Single-key Providers and presets

Each Provider owns one upstream API key and status; represent multiple vendor accounts as separate Providers. Import Provider / Model presets in Admin, add the real key, then attach routes.

Four strategies, Provider Sticky, and circuits

Choose hash_affinity, weighted_random, weight_priority, or weighted_round_robin, with optional per-priority-tier overrides. Provider Sticky can remember a successful Target across requests; 429, 401 / 403, and consecutive 5xx responses trigger Provider-level cooldowns.

User keys, budgets, and pricing

Govern access through External system → User → API Key. Budgets and reset periods belong to Users; API Keys authenticate and attribute usage. Model requests record supplier cost, catalog list price, and charged to user, with business-timezone daily schedule multipliers.

Logs, audit, analytics

Request logs capture protocol / operation, model, route group, Surface, Pool, Target, Provider, tokens, status, and three cost views. Audit logs track budget charges, resets, and user / key lifecycle events.

Playground and Simulator

Playground tests a single Admin route. Simulator uses a real user key from the browser to verify auth, routing, billing, and logs.

Deployment and integration

Default Cloudflare Workers + D1 (individuals and light traffic can usually stay in the free tier); or Node / Docker with PostgreSQL or MySQL 8. External portals can provision users, keys, and budgets through /api/admin/*.

Next Steps

Default path is Cloudflare: try Wrangler + local D1, then one CLI onto your account; configure Provider, model, and route, create a user key, and point clients at the Gateway Base URL.

• Quick start — local D1 → Cloudflare deploy
• OctaFuse updates — releases, product progress, and field notes
• Providers · Models · Routing
• Provider Catalog · Model Catalog — browse the static presets bundled for direct Gateway import
• Images · Agent Tools · Analytics
• Request logs · Audit logs · System config
• Cloudflare deployment · Docker deployment
• GitHub technical reference — APIs, architecture, migrations, and operations index
• Source repository · Contributing