Inference API · By invitation

Open models, one endpoint.

tokenflow serves open-weight models from a pool of GPUs we run or rent. Point your OpenAI or Anthropic client at it and keep your code.

Request access See the quickstart

api.tokenflow.dev

# Same request your OpenAI client sends
curl https://api.tokenflow.dev/v1/chat/completions \
  -H "Authorization: Bearer $TOKENFLOW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.8-27b",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Speaks

OpenAI Chat Completions /v1/chat/completions
OpenAI Responses /v1/responses
Anthropic Messages /v1/messages

Features

Built for long agent sessions

Agents resend the same growing conversation on every turn. tokenflow is designed around that.

Cache-aware routing

Each conversation goes back to the GPU that already holds its prompt prefix, so each turn only prefills what is new.

Failover within a model

If a node fails before your response starts, the request is retried on another node serving the same model, without your client doing anything. That node may run a different quantization.

Native protocols

OpenAI and Anthropic requests reach the engine in their own format. Tool calls, streaming events and usage come back as your SDK expects.

Metering you can check

Every key’s usage is split into cached input, uncached input and output tokens, with monthly quotas per organization.

Images in context

Up to 16 inline PNG, JPEG, WebP or GIF images per request, including screenshots that browser agents carry in their history.

Stops when you do

Close the connection and generation stops on the GPU. You pay for the tokens that were sent, not the ones that would have been.

How it works

One endpoint in front of a pool of GPUs

The gateway runs at the edge. The GPUs run wherever capacity is: our own hardware or rented servers.

Your request arrives at the gateway with your key and is checked against your quota.
It is routed to the node that served the earlier turns of the same conversation, if that node is healthy.
Tokens stream back through the gateway, and the usage is recorded against your key.

Quickstart

Change the base URL. That’s it.

Use the SDK you already have. Once you’re invited, your API keys are in the console.

OpenAI SDK · Python

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokenflow.dev/v1",
    api_key=os.environ["TOKENFLOW_API_KEY"],
)

reply = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[{"role": "user", "content": "Hello"}],
)

Anthropic SDK · Python

import os
from anthropic import Anthropic

client = Anthropic(
    base_url="https://api.tokenflow.dev",
    api_key=os.environ["TOKENFLOW_API_KEY"],
)

reply = client.messages.create(
    model="qwen3.8-27b",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello"}],
)

Available today: qwen3.8-27b and gemma4-12b, a smaller and faster model, each also as -direct (qwen3.8-27b-direct, gemma4-12b-direct), the same model with an adapter that declines fewer requests. GET /v1/models lists what your key can use.

Coding agents

Runs in the agent you already use

Claude Code and Codex work on tokenflow with a few environment variables or one profile file. No proxy, no plugin.

Claude Code

Anthropic Messages API, every model tier mapped to qwen3.8-27b.

Set up Claude Code →

Codex

OpenAI Responses API, in its own --profile tokenflow.

Set up Codex →

By invitation, for now

tokenflow is open to existing customers while we grow capacity. Tell us what you’re building and how much you expect to run.

Request access