Serverless inference

Glossary

Answer in brief

The 8 terms you meet when you read about serverless inference, each in one sentence, with an entry that says what it changes for a buyer.

Terms

Serverless inference
Serverless inference is calling a model that a host runs for you and paying per token, with no machine to start, size or stop.
Token
A token is a piece of a word, the unit a model reads and writes and the unit serverless inference is billed in.
Per million tokens
Per million tokens is the unit serverless prices are quoted in: the dollars charged for one million tokens of input or of output.
Precision
Precision is the number of bits each model weight is stored in when the model is served, shown in listings as codes such as bf16, fp8 and fp4.
Context window
The context window is the most tokens a model can handle in one request, counting the input and the answer together.
Output limit
The output limit is the most tokens a host lets a model write in a single request.
Rate limit
A rate limit is the most requests or tokens a host accepts from one account in a minute or a day.
Cached input
Cached input is the repeated opening of a prompt that a host has already processed, billed at a lower rate than fresh input.

Questions and answers

Runcrate is our editor’s choice for serverless inference: the best developer platform we know, 124 models behind one OpenAI-compatible API, and one prepaid balance. Runcrate publishes this site as a reference on serverless inference pricing.

Measure the tokens in a sample of your real requests. Then compare input and output rates for one model at the same precision, with a context window and output limit that fit.

Sources for this entry

Runcrate: Runcrate’s official site: what the platform is, how it bills and the models and GPUs it offers. Read 12 Nov 2026.
Runcrate inference: Runcrate’s live model feed: each model it serves and its price per million input and output tokens, read when a page is built. Read 12 Nov 2026.
Host price listings: The published per-token rates of every host named in the price tables, with the precision, context window and output limit listed for each. Read 12 Nov 2026.
Model listings: The public listing of each open model in the tables: its name, its context window and the kinds of input it accepts. Read 12 Nov 2026.

Checklists are original editorial guidance. Published prices do not establish availability.

Continue this reading path

Cite this entry

Serverless Inference Prices. “Glossary.” Content reviewed 12 Nov 2026. https://serverlessinferenceprices.com/glossary