The choice between self-hosted and API pricing depends on utilisation, not per-token rates alone. If you compare only token costs against per-token pricing, you will pick wrong. Both hosting models have fixed costs that dwarf the marginal cost per token once you account for them honestly.
What you actually pay for self-hosted
A self-hosted model running on your own hardware carries costs in layers.
The hardware itself. A GPU with enough VRAM to run a useful model (quantised to fit, typically 8 to 24 GB) costs somewhere between a used consumer card and a small enterprise GPU, depending on what model you choose and how small you can quantise it. Tools like Ollama publish per-model requirements. That hardware is depreciated over its useful life, typically three to five years. If the card lasts three years and costs $2000, that is $667 per year or $55 per month, whether you run one inference per day or one thousand.
Power. A GPU under load draws real power. The cost depends on your regional electricity rate and the card's efficiency. A sustained workload of moderate queries might draw 50-150W; an idle card draws far less. If you run predictions sporadically, most of the power budget goes to keeping the server warm. If you run continuously, power per inference drops per query, but it still costs money.
Compute time on your infrastructure. Someone's server is serving that model. If you already operate a data centre or have spare capacity, this is marginal. If you do not, you are paying for hardware you use only for inference. Cloud providers rent this to you; self-hosting means you own it and bear the operational cost — cooling, space, power provisioning, physical security.
Operations. The model has to be updated. The server needs monitoring, restarts, security patches. A small team might outsource this to managed hosting (which is now a hybrid model, not pure self-hosted). A large team pays an engineer's time, partial or full. An engineer earning a typical professional salary working part-time on this function adds tens of thousands per year to the cost.
Sum these: hardware ($55/month) + power ($30–150/month, depending on utilisation and your electricity cost) + data-centre share ($100–500/month for your physical space and cooling, or $0 if truly on-premises) + engineer time ($500–5000/month depending on headcount and pay). The fixed cost before the first inference is somewhere between $685 and $5655 per month.
The marginal cost of an inference is near-zero once the hardware is running. One inference or a thousand on a server you already own and power costs almost nothing extra.
What you actually pay for an API
An API pricing model charges per unit of work: tokens consumed or API calls made. It has no hardware cost to you. The provider amortises their infrastructure over all customers.
API costs vary with actual usage. If your team runs ten inferences this month and fifty next month, you pay ten times less in month one. If you run nothing, you pay nothing.
API pricing is also variable in hidden ways. Some providers charge differently for input vs output tokens. Some have different rates for different model families. Some offer volume discounts that kick in at usage thresholds you may not reach. Some change pricing without notice. A vendor-neutral article cannot cite specific numbers (they drift), but you can check current rates by looking at each provider's pricing page.
The practical variable cost per inference depends on the model, the length of input and output, and your volume. Small models cost less; large models cost more. Few tokens cost less; long generations cost more. But there is no fixed cost in front of it. Your first inference costs the same per unit as your millionth.
The break-even shape
Plot cumulative cost over three years. For an API, it is a straight line from zero, sloping up with your usage. For self-hosted, it is a steep line from the fixed cost baseline ($685–5655 per month) that rises slowly as usage goes up (marginal cost is near-zero).
They cross. The crossover point depends on three things:
Your fixed cost baseline. A startup with no server farm might spend $5000/month on self-hosted. A team within a large organisation might spend $500/month if they rent compute from an internal pool. Lower baseline means the crossover happens earlier.
Your utilisation. How many inferences per month? Sustained high volume makes self-hosted cheaper. Spiky, unpredictable volume makes API cheaper because you pay only for spikes.
The per-token cost. Some models cost more per token. Some providers charge less. Higher per-token cost of APIs pushes the crossover earlier; self-hosted becomes cheaper sooner.
Hosted APIs win when volume is low or unpredictable and your engineers should be doing other work. Self-hosted wins when you have sustained volume, spare infrastructure, and someone maintaining it as part of their job anyway.
The crossover is not theoretical. You can calculate it: take your fixed cost baseline, estimate your monthly inference volume with a specific model, check the API pricing for that model, and do the arithmetic. If fixed cost per month is $F$, inferences per month is $I$, and cost per inference is $C$, then self-hosted total cost per month is $F + (I \times C)$ and API cost is $(I \times C_{API})$. They cross when $F = I \times (C_{API} - C)$. Rearrange: crossover volume is $F / (C_{API} - C)$.
The gap $C_{API} - C$ may be small if self-hosted has its own variable costs (engineer time when deployed at scale, power that rises with load). Use realistic numbers from your own infrastructure to know which side you are actually on.
What this means for adoption
If you are shopping for internal AI tools, this decision should drive the implementation choice. Low, spiky usage across a dispersed team points to APIs. High sustained usage within a team with spare compute points to self-hosted or hybrid (APIs for expensive models, self-hosted for cheap ones you run a lot).
A platform like Nodus Veritatis supports both: local models run on your hardware with no egress, and hosted APIs under your key and controls. You can measure which your team actually needs before committing to one.