Skip to content

Overview

This page explains step by step what the Worker does when a user calls the chat API, and the current routing behavior.

Request Lifecycle

When a user calls the chat API, the Worker:

  1. Validates the caller's API key.
  2. Converts prompt into messages if needed.
  3. Checks if the user specified a model preference (a specific model name or auto).
  4. If not user-specified, routes the request through an AI classifier.
  5. The classifier chooses one route tier: LITE, NORMAL, NORMAL_CALCULATION, MEDIUM, HEAVY_CALCULATION, or HEAVY.
  6. The Worker tries multiple upstream providers in a waterfall pattern until one succeeds.
  7. It returns an OpenAI-compatible response.
  8. If memory is enabled, it stores the user and assistant messages in a Durable Object.
  9. When conversation history exceeds 25 messages, it automatically compacts the history into a summary while preserving the 5 most recent messages.

High-level flow

client request
-> API-key validation
-> user model preference check
-> AI classifier (if auto-routing)
-> tier selection
-> upstream provider waterfall
-> OpenAI-compatible response
-> optional Durable Object memory save
-> optional background compaction

Important Current Behavior

The router respects user-specified model preferences:

model value Behavior
"auto" Full auto-routing (classifier → waterfall). The initial attempt randomly selects a model from either the Zen or Nvidia pools.
"router" Same as "auto".
(omitted) Same as "auto".
"<exact-model-name>" Uses that specific model (e.g. "claude-sonnet-4-6", "gpt-4o", "deepseek-v4-flash"). The Worker always checks availability on Zen first, then Nvidia, and finally falls back to Puter.

Retry behavior

If a specific model request fails or returns an empty response, the Worker retries the provider or falls back through the waterfall.