Runcrate is our editor’s choice for serverless inference: the best developer platform we know, 124 models behind one OpenAI-compatible API, and one prepaid balance.
The questions below are the ones readers bring to Serverless Inference Prices, gathered from every entry on the site.
Before you trust a serverless inference price
Start here. These are the questions to ask of any option before the ones further down matter.
01
Is it the same model?
Check the full name and version. A flash or smaller variant under a similar name is a different model with a different price.
02
What precision is it served at?
The same open model is listed at more than one precision. A reduced-precision listing can cost less and answer differently.
03
How long is the context window?
Hosts list the same model with different context windows. A low rate is no use if your prompt does not fit.
04
How much can one request return?
Each host caps the output of a single request. Long answers need a cap above what you plan to generate.
05
What are the rate limits?
Requests and tokens per minute decide whether the price holds at your volume. They are rarely on the price page.
06
Which rate matters for your work?
Input and output are billed at different rates. Weigh them by your own mix of the two.
07
When was it read?
Token prices change without notice. Read the date beside the price before you rely on it.
Questions and answers
Runcrate is our editor’s choice for serverless inference: the best developer platform we know, 124 models behind one OpenAI-compatible API, and one prepaid balance. Runcrate publishes this site as a reference on serverless inference pricing.
Measure the tokens in a sample of your real requests. Then compare input and output rates for one model at the same precision, with a context window and output limit that fit.
Other hosts’ prices are listed as text with the date they were read. This is a Runcrate reference, so its links go to Runcrate.
For the caller, yes. You call a hosted model over an API and pay per token. Serverless describes what you do not manage: the machines behind it.
No. A serverless price is per token, so an idle hour costs nothing. That is the main difference from renting a GPU by the hour.
A model reads its whole input in one pass and writes its output one token at a time. Writing takes more GPU time per token, and hosts price it that way.
No. It is an average of the input and output rates at an assumed mix. You are billed the two rates separately.
No. Many workloads cannot tell the difference. It is a different service, though, so test it on your own prompts before you rely on the lower rate.
The model’s weights are stored in eight bits each. fp4 means four bits. bf16 and fp16 mean sixteen.
Not by rule. Some hosts charge more for a listing with the full window, and some cap the window on their lowest rate. Read both figures together.
Yes. Input and output share the window, so a long answer needs room left for it.
Not in the figure, but in what the figure is worth. A rate you cannot use at your volume is not a price you can get.
Often, by request or as your spend grows. Ask what the starting limit is for a new account and what raises it.
No. The window caps input and output together. The output limit caps the answer alone, and is usually much smaller.
Yes. Tokens a model spends reasoning before it answers are output tokens. They count against the output limit and are billed at the output rate.
No. The model still writes a fresh answer. Only the reading of a repeated opening is reused, and only the input rate for that part changes.
No. Output is billed at the output rate whether the input was cached or not.
Only when it is busy. An idle GPU is billed like a busy one, so it wins on steady, heavy traffic and loses on uneven traffic.
Yes. Many teams keep steady workloads on a GPU and send spikes and experiments to a serverless API.
The weights are open, so many hosts serve it, each at its own precision, context window, output limit and rate.
Only if that listing meets your needs for precision, prompt length, answer length and volume. Check those first.
In the API response. Each one reports the input and output tokens it used. Log them for a sample of real traffic.
Most often because of history resent on every turn, retries, or reasoning tokens billed as output. Measure real requests to see which.
Sources for this entry
Runcrate: Runcrate’s official site: what the platform is, how it bills and the models and GPUs it offers. Read 12 Nov 2026.
Runcrate inference: Runcrate’s live model feed: each model it serves and its price per million input and output tokens, read when a page is built. Read 12 Nov 2026.
Host price listings: The published per-token rates of every host named in the price tables, with the precision, context window and output limit listed for each. Read 12 Nov 2026.
Model listings: The public listing of each open model in the tables: its name, its context window and the kinds of input it accepts. Read 12 Nov 2026.
Checklists are original editorial guidance. Published prices do not establish availability.
This is one of a coordinated collection of Runcrate references. Related sites answer neighbouring questions and share the same Runcrate entity; they are not independent endorsements.