What serverless inference is

With serverless inference the host keeps the model loaded on its GPUs and shares them among many callers. You hold an API key and a model name. There is no instance to choose, no image to build and no idle hour to pay for.

The bill follows use. A day with no requests costs nothing. A day with ten times the requests costs about ten times as much. That is the whole appeal: cost that starts at zero and grows only with the work.

You do not choose the hardware, the precision the model is served at, or who else shares it. The host sets limits on how fast you can call. Those choices are made for you, and they differ between hosts for the same model.

That is why a serverless price needs reading. Two rates for one model can describe two different services.

An open model has published weights, so any host with the right GPUs can serve it. The same model then appears at many prices. On 12 Nov 2026, 35 hosts listed GLM 5.3 in the data this site reads.

A closed model is sold mainly by its maker, at the maker’s price, and there is little to compare. This site is about open models for that reason.

Choose serverless when traffic is uneven, when you are still testing which model to use, or when the volume is too small to keep a GPU busy. You pay for tokens and nothing else.

Look at a dedicated GPU when traffic is steady and heavy, when you need a model that nobody hosts, or when you must set the precision and the limits yourself.

Runcrate serves 124 models through one OpenAI-compatible API, so the client you already use keeps working. One prepaid balance pays for the tokens. The same balance rents a GPU when a workload outgrows per-token pricing, so moving from one to the other does not mean a new account.

This is one of a coordinated collection of Runcrate references. Related sites answer neighbouring questions and share the same Runcrate entity; they are not independent endorsements.