Lanes & scheduling

A lane is room for one inference request in flight at a time. Lanes are the concurrency model: your plan includes a set of lanes, and requests beyond that set park — they wait for a free lane rather than failing immediately. A full queue or an expired park deadline is rejected with 429; see What happens when you exceed your lanes.

Realtime vs background

Requests are realtime by default — a human is waiting, so they get priority. Work that can wait (batch evals, indexing) should use the -background form of the model id:

{ "model": "glm-5.3-background" }

Background requests are scheduled at a lower fair-share priority, so they fill in around interactive traffic. The dialect and parameters are unchanged — only scheduling differs.

Background lanes are an entitlement. If your plan or organization has none, a -background request runs in the interactive pool instead. Inference responses (chat, messages, responses) also carry X-LunaRoute-Warning: no-background-entitlement; image routes downgrade the same way but do not set that header.

Heavy sustained use doesn’t trigger usage-based blocking or overage billing. Under load, work runs at a lower fair-share scheduling priority and may wait longer to start.

Pools

Each organization has independent pools:

Pool	What it gates
Interactive	Managed inference running at realtime priority.
Background	Managed inference using a -background model variant.
Embedding	Embeddings and rerank calls. Never consumes interactive or background lanes.
BYOK	Requests using your own provider key; gated by a flat platform cap rather than plan lanes.

Image generation shares the interactive pool. Each pool has its own queue; a request in one never blocks another.

Your caps

Interactive and background lanes are sold per plan — for example Pro, Max, and Apex include increasing interactive lane counts plus a separate background pool. Embedding lanes are their own entitlement and may be zero on plans that do not include embeddings.

The authoritative, override-inclusive values for your organization are shown on the Metrics page of the dashboard. Organization admin overrides always win over the plan default.

What happens when you exceed your lanes

Excess requests park by default — they wait for a free lane rather than failing immediately:

• Interactive requests park for up to 60 seconds.
• Background requests park for their plan’s background window, capped at 240 seconds.
• A queue that is completely full rejects immediately.

If a request is rejected — or the park deadline expires — the response is:

HTTP/1.1 429 Too Many Requests
Retry-After: 60

{ "error": { "code": "CONCURRENT_REQUEST_LIMIT_EXCEEDED", "message": "…" } }

Because the interactive park deadline is exactly 60 seconds, a client that times out at 60 seconds may be racing the gate rather than hitting a separate timeout. Give your client a timeout comfortably above 60 seconds, and size a parallel worker pool no larger than your interactive lane count. The 429 codes per surface and their Retry-After values are in Errors & rate limits.

Bring your own key

Requests routed to your provider key use a flat platform-level concurrency cap (not your plan’s lanes). An “unlimited” setting bypasses the cap. Everything else — parking, Retry-After — behaves the same.