AI Gateway & Control PlaneOctaFuse

AI capabilities are expanding beyond text models into image generation, Agent Tools, private models, and specialized services. Providers, compatible endpoints, coding plans, self-hosted services, and internal accounts each bring their own Base URLs, keys, quotas, pricing, and logs. Applications need something simpler: one stable entrypoint, controlled routing, clear budgets, and traceable request history.

OctaFuse is a self-hostable AI capability gateway and operations control plane. Unify Chat, Responses, image generation and editing, ASR, TTS, realtime speech, Agent Tools, and private model services. Request Surface → Route Pool → Upstream Target topology keeps routing, failover, Provider circuits, budgets, billing, audit, and Admin APIs under your control. Clients still use one Gateway URL and one user key.

Application Scenarios

Individual developers

Consolidate model subscriptions, coding plans, image generation, Agent Tools, and backup upstreams behind one Gateway. Cursor, CLIs, scripts, and personal tools only need one Gateway URL and one user key.

AI apps and agents

Build against stable model IDs and common protocols. Move model swaps, canary routes, backup providers, degradation, and default parameters into the gateway layer.

Platform and IT teams

Create separate users and API keys for departments, projects, members, or customers, with budget periods, status, and metadata outside application services.

API proxy businesses

Connect multiple upstream Providers; represent multiple accounts at the same vendor as separate Providers. Issue downstream customer keys and manage each customer through request logs, cost lenses, and audit records.

Private and hybrid models

Put self-hosted compatible services, internal models, Ollama / vLLM paths, and public model providers behind one routing system with a consistent external surface.

Cost and reliability ops

Observe usage and failures by model, Provider, user, and reliability view, then tune Route Pool strategy, priority / weight, budgets, and time-of-day pricing.

Product Capabilities

Multi-protocol, multimodal entrypoint

Supports OpenAI Chat, Responses, Images, Audio, and Models; Anthropic Messages; Gemini generateContent / streamGenerateContent; and native DashScope audio. Portals can discover capabilities through public GET /catalog/models.

Image generation / edit

OpenAI-compatible POST /v1/images/generations and edits, with explicit Token or per-image (per_image) pricing. Client cancellation, Gateway timeout, and explicit upstream failures do not charge.

Agent Tools

Extensible product APIs for agents (/v1/tools/*). Web Search supports Bocha, Tavily, CleverSee, and Tencent Cloud WSA; Web Fetch supports Firecrawl, Tavily Extract, and Jina Reader; Deep Search supports Firecrawl Search and Jina Search.

Surface → Pool → Target routing

Client requests resolve a protocol and operation Request Surface, then enter a Route Pool. Pools group Upstream Targets by priority and order each tier with strategy plus weight before automatic failover.

Provider and model presets

One-click Admin import for project-scoped Vertex AI, BytePlus, Model Studio International, Meta Model API, Cerebras, SambaNova, Hugging Face, Vercel, and many model, aggregation, and Coding / Token Plan presets.

Single-key Providers and presets

Each Provider owns one upstream API key and status; represent multiple vendor accounts as separate Providers. Import Provider / Model presets in Admin, add the real key, then attach routes.

Four strategies, Provider Sticky, and circuits

Choose hash_affinity, weighted_random, weight_priority, or weighted_round_robin, with optional per-priority-tier overrides. Provider Sticky can remember a successful Target across requests; 429, 401 / 403, and consecutive 5xx responses trigger Provider-level cooldowns.

User keys, budgets, and pricing

Govern access through External system → User → API Key. Model requests record supplier cost, catalog list price, and charged to user. Matching Daily schedule windows override default factors while legacy configuration keeps multiplication semantics.

Logs, audit, analytics

Request logs capture protocol / operation, model, route group, Surface, Pool, Target, Provider, tokens, status, and three cost views. Audit logs track budget charges, resets, and user / key lifecycle events.

Playground and Simulator

Playground uses built-in samples to test one route and observe streams and Gemini tool calls. Simulator derives the protocol from the client request surface and uses a real user key to verify auth, routing, billing, and logs.

Deployment and integration

Default Cloudflare Workers + D1 (individuals and light traffic can usually stay in the free tier); or Node / Docker with PostgreSQL or MySQL 8. External portals can provision users, keys, and budgets through /api/admin/*.

Next Steps

Default path is Cloudflare: try Wrangler + local D1, then one CLI onto your account; configure Provider, model, and route, create a user key, and point clients at the Gateway Base URL.

• Quick start — local D1 → Cloudflare deploy
• OctaFuse updates — releases, product progress, and field notes
• Providers · Models · Routing
• Provider Catalog · Model Catalog — browse the static presets bundled for direct Gateway import
• Images · Agent Tools · Analytics
• Request logs · Audit logs · System config
• Cloudflare deployment · Docker deployment
• GitHub technical reference — APIs, architecture, migrations, and operations index
• Source repository · Contributing