Experiment 2 · November 2026

Count your tasks before you buy a GPU.

Should a team buy a graphics card for its AI models, or pay a provider per token? I loaded my own card until users would start to wait, scored every answer, and priced the same work at a German cloud. The answer is a number of tasks a day.

  • 2 open models
  • 4 tasks
  • 84 runs
  • 131,939 answers scored against a key
  • 1 RTX 5090
  • Run on 13 November 2026

The short version

Below a number of tasks a day, the API is cheaper. Above it, owning is.

Count your tasks per working day, by type, before you buy. A machine that waits is the expensive one.

The results on one page

Four kinds of task, three numbers each.

How many the card finishes in an hour while every user still gets a fast answer, how many of those answers are correct, and the number per working day from which owning is cheaper than the API.

TaskPer hourCorrectAPI per 1,000Owning pays from, a dayWith staffCard does in 8 h
Agent question4,34099%€2.352,8507,50034,700
Long solution1,13097%€1.345,35014,1009,050
Short answer7,60098%€0.3122,40059,10060,800
Sorting a message72,40077%€0.2132,40085,400578,900

Qwen3.8-27B on one RTX 5090, against the list price of the same model at a German cloud [1]. “Per hour” and “correct” are measured at the highest load at which users still get a fast answer; “with staff” counts 5 staff hours a month. The fastest task pays off last: what decides is the price the API charges for one task.

Finding 1 · The break-even

Owning pays from about 2,850 agent questions a working day.

An API bills every question. A machine of your own costs the same every day, whether it works or waits. So the two lines cross at a number of tasks a day, and below that number the API is cheaper.

For an agent question, one that looks things up with tools, the API charges about a quarter of a cent (€0.24 per 100). My test machine costs €6.60 per working day for the hardware and its power. The lines cross at 2,850 questions a day.

Then count the people. My machine needs between 1 and 5 hours of care a month. At €45 an hour, 5 hours add €10.80 a working day, more than the hardware itself, and the crossing moves to 7,500.

Count your tasks per working day before you compare prices per token.

The lines cross at 2,850 agent questions a day, 7,500 with staff.

Cost per working day (€), agent questions on Qwen3.8-27B: German cloud at list price against one owned machine

€0€5€10€15€20€2502,5005,0007,50010,000Agent questions per working dayAPIOwn machineOwn machine + 5 staff hours a monthThe lines cross at 2,846 agent questions per working day2,850The lines cross at 7,503 agent questions per working day7,500

Source: own measurement, Dr. Kalun Ho (2026); STACKIT list prices, 13 November 2026 [1]

Finding 2 · By task

Agents reach the line first. A team’s chat never does.

The break-even depends on what the API charges for one task. An agent question is expensive at the API, because the agent sends its whole context again at every step: 89 % of its price is input. So owning pays from 2,850 a day.

A short answer costs the API about an eighth of that. Owning needs 22,400 short answers a day to pay, and 32,400 sorted messages. Long worked solutions sit in between at 5,350.

As an illustration: a team of 50 people asking 20 short questions each sends 1,000 a day, a twentieth of what owning needs. 250 people with 20 agent questions each send 5,000: above the line with hardware only, below it once staff is counted.

Compare each kind of task with its own break-even. One number for “AI usage” hides the answer.

Owning needs 2,850 agent questions a day, or 22,400 short answers.

Tasks per working day from which the owned machine is cheaper than the API (hardware and power only; with 5 staff hours a month beside it)

Agent question: owning pays from 2,846 a working day with hardware only, 7,503 with 5 staff hours a monthAgent question€2.35 per 1,000 at the API2,850 · with staff 7,500Long solution: owning pays from 5,343 a working day with hardware only, 14,083 with 5 staff hours a monthLong solution€1.34 per 1,000 at the API5,350 · with staff 14,100Short answer: owning pays from 22,409 a working day with hardware only, 59,068 with 5 staff hours a monthShort answer€0.31 per 1,000 at the API22,400 · with staff 59,100Sorting a message: owning pays from 32,387 a working day with hardware only, 85,370 with 5 staff hours a monthSorting a message€0.21 per 1,000 at the API32,400 · with staff 85,400

Source: own measurement, Dr. Kalun Ho (2026); STACKIT list prices, 13 November 2026 [1]

Finding 3 · Capacity

One card does up to 11 times the work of one request at a time.

A graphics card can work on many requests in the same pass. I raised the load until users would start to wait: the first word within 2 seconds, then at least 10 words a second, for 99 % of calls.

Inside that limit the card finished 4,340 agent questions an hour with 16 agents at once, against 540 with one at a time. In an 8-hour day that is about 34,700 questions, 12 times the break-even.

For long solutions the margin is thin: the card finishes about 9,050 in 8 hours, against a break-even of 5,350. That only pays if the machine also works at night.

Capacity is rarely the problem. Check it for long answers, and plan work for the quiet hours.

At the waiting limit the card answers 4,340 agent questions an hour.

Tasks finished per hour on one card, Qwen3.8-27B. Bars are scaled within each task

  • One request at a time
  • As many at once as the waiting limit allows
Agent question×8 at 16 at onceAgent question, one request at a time: 540 per hour540 per hourAgent question, 16 requests at once: 4,342 per hour4,340 per hourLong solution×9.4 at 12 at onceLong solution, one request at a time: 120 per hour120 per hourLong solution, 12 requests at once: 1,132 per hour1,130 per hourShort answer×11 at 16 at onceShort answer, one request at a time: 690 per hour690 per hourShort answer, 16 requests at once: 7,596 per hour7,600 per hourSorting a message×2.7 at 8 at onceSorting a message, one request at a time: 26,688 per hour26,700 per hourSorting a message, 8 requests at once: 72,363 per hour72,400 per hour

Source: own measurement, Dr. Kalun Ho (2026)

Finding 4 · Quality

A busy card answers as well as an idle one.

The worry is obvious: if the card shares its work across many requests, answers could get worse. Every answer was scored against a fixed key, 131,939 in total, with no AI judging another AI.

Under load the card answered 99 % of the agent questions correctly, against 99 % with one at a time. On all four tasks the difference stayed within 3.3 points, and the two small drops come from tiny samples at one request at a time (18 long solutions).

One limit: my card runs a compressed, 4-bit copy of the model. I did not compare it with the full-precision version a cloud may serve.

Load does not cost quality. Test the compressed model on your own tasks before you rely on it.

Under load the answers are as good as with one request at a time.

Share answered correctly against a fixed answer key, Qwen3.8-27B (4-bit)

  • One request at a time
  • At the waiting limit
Agent question673 answers under loadAgent question, one request at a time: 98.8% correct of 8499%Agent question, at the limit: 99.4% correct of 67399%Long solution181 answers under loadLong solution, one request at a time: 100.0% correct of 18100%Long solution, at the limit: 96.7% correct of 18197%Short answer1,196 answers under loadShort answer, one request at a time: 100.0% correct of 102100%Short answer, at the limit: 97.7% correct of 1,19698%Sorting a message11,449 answers under loadSorting a message, one request at a time: 75.8% correct of 4,22876%Sorting a message, at the limit: 76.8% correct of 11,44977%

Source: own measurement, Dr. Kalun Ho (2026)

Finding 5 · Memory

Memory, not speed, set the limit. My hypothesis did not hold.

On your own card no token is billed, so an agent’s re-sent context is free. I expected more: that the card keeps earlier context in a cache and barely has to work on it again. My hypothesis before the runs: “at least 80 % from cache”.

I measured 29 % with one agent and 0.6 % with 16. The model’s weights take most of the card’s 32 GB. What is left holds about 95,000 tokens of context, it was full from 16 agents on, and the server never ran more than 19 requests at once.

A smaller model shows the trade. gpt-oss-20b left five times the memory, served 64 agents at once without reaching a limit, and took 84 % of its input from cache. It answered 86 % of the agent questions correctly, against 99 %.

If you own, check the memory left after the model before you look at speed.

The larger model took 0.6% of an agent’s input from cache, the smaller one 84%.

Share of the agent’s input tokens served from the card’s cache, at low load and at each model’s limit

Qwen3.8-27Broom for 95,000 tokens of contextQwen3.8-27B, 1 agents at once: 29.4% of the input came from cache29% with 1 agent at onceQwen3.8-27B, 16 agents at once: 0.6% of the input came from cache0.6% with 16 agents at oncegpt-oss-20broom for 500,000 tokens of contextgpt-oss-20b, 1 agents at once: 82.7% of the input came from cache83% with 1 agent at oncegpt-oss-20b, 64 agents at once: 83.5% of the input came from cache84% with 64 agents at onceHypothesis before the runs: at least 80%

Source: own measurement, Dr. Kalun Ho (2026)

Finding 6 · What changes the answer

The answer depends on people, not on the electricity price.

The 2,850 counts only the machine and its electricity. Each bar shows that number with one thing changed. A longer bar means you need more work before owning pays off.

People matter most: 5 hours of care a month cost €10.80 a day, more than the machine, and raise the number to 7,500. A spare machine raises it to 5,700. A cloud that bills repeated text at a quarter raises it to 8,900, because an agent sends the same text again at every step.

Electricity barely matters: between €0.15 and €0.35 per kWh, the number stays between 2,700 and 3,000.

Before you buy, count the people and the spare machine. The electricity price can wait.

Care and a spare move the line. Electricity barely does.

Agent questions a working day you need before owning pays off, with one thing changed at a time

My machine: hardware and electricity only: owning pays from 2,846 agent questions a working dayMy machine: hardware and electricity only2,850You need more questions before owning paysvs. startCloud bills repeated text at a quarter: owning pays from 8,916 agent questions a working day, ×3.1 the starting point of 2,846Cloud bills repeated text at a quarter8,900×3.1Plus 5 hours of care a month: owning pays from 7,503 agent questions a working day, ×2.6 the starting point of 2,846Plus 5 hours of care a month7,500×2.6Cloud price halved: owning pays from 5,778 agent questions a working day, ×2.0 the starting point of 2,846Cloud price halved5,800×2.0Plus a spare machine: owning pays from 5,693 agent questions a working day, ×2.0 the starting point of 2,846Plus a spare machine5,700×2.0Electricity at €0.35 instead of €0.24 per kWh: owning pays from 3,013 agent questions a working day, ×1.06 the starting point of 2,846Electricity at €0.35 instead of €0.24 per kWh3,000×1.06You need fewerMachine kept 5 years instead of 3: owning pays from 1,829 agent questions a working day, ×0.6 the starting point of 2,846Machine kept 5 years instead of 31,850×0.6Work on every day, not only working days: owning pays from 1,950 agent questions a working day, ×0.7 the starting point of 2,846Work on every day, not only working days1,950×0.7

Source: own measurement, Dr. Kalun Ho (2026); STACKIT list prices, 13 November 2026 [1]; Eurostat [2]; Destatis [3]; DeepInfra [5]

Your own numbers

How to count for your team.

The formula has one line. Everything in it is a number you can look up.

Break-even tasks per working day = cost of owning per working day ÷ (API price per task − electricity per task)

InputTest machineWhere to find yours
Price of the machine€4,423 without VAT, written off over 3 years: €1,474 a yearthe quote for the machine, your write-off period
Power when idle85 W at the wall, at €0.2366 per kWh: €176 a year [2]a plug with a meter, your electricity contract
People0, 1 or 5 hours a month at €45 an hour [3]who installs, updates and watches it, and for how long
Working days250 a yearthe days on which the tasks arrive
API price€0.45 per million tokens in, €0.65 out, for the same model in a German cloud [1]your provider’s price list
Tokens per taskagent question: 4,678 in, 382 outyour API bill or your logs, by kind of task
Example (illustrative)Tasks a dayBreak-even, hardware onlyBreak-even, with staffCheaper
50 people, 20 short questions each1,00022,40059,100API, by far
50 people, 20 agent questions each1,0002,8507,500API
250 people, 20 agent questions each5,0002,8507,500Own machine, if nobody has to look after it
1,000 people, 20 agent questions each20,0002,8507,500Own machine
  • Below the break-even: use the API. It costs nothing when nobody asks.
  • Above it: own, and fill the quiet hours with batch work, such as long documents at night.
  • Watch your agents. They multiply tokens per task and an API bills every one, so they are what moves a team towards the line.
  • If your data may not leave the building, owning is the only option, and the question becomes whether you can keep the card busy. The cloud in this comparison is already German.

How it was measured

One card, loaded step by step, every answer scored.

A program plays the users. It keeps a fixed number of requests in flight, from 1 to 64, for 4 minutes per run, and every answer is checked against a key.

TaskDataItems scoredStands forCounts as correct
Sorting a messageBanking77 test set (PolyAI, CC BY 4.0), 3,080 customer messages, 77 intents104,178routing a support ticketexact intent, of 77
Short answerGSM8K test set (OpenAI, MIT), 1,319 school maths problems6,847a multi-step business calculationfinal number matches
Agent questionSynthetic invoice ledgers: 500 invoices, 12 suppliers, 4 quarters; tools to list suppliers, fetch invoices and calculate; a new ledger every 20 questions19,883an analyst question answered by looking things upmatches the key computed from the ledger
Long solutionMATH-500 (HuggingFaceH4, MIT), level 5 with integer answers, 74 problems1,031a long worked answerinteger matches
ModelMaker, releaseBuild on the cardTasksRoom for context
Qwen3.8-27BAlibaba, August 2026NVFP4, 4-bit, thinking offall four95,000 tokens
gpt-oss-20bOpenAI, August 2025MXFP4, effort lowagent questions500,000 tokens
  • Hardware. One NVIDIA RTX 5090 with 32 GB in my own PC, bought privately. No sponsor, no employer hardware, no employer data. This is not a recommendation to buy this card: the method works for any card.
  • Load. 1, 4, 16, 32 and 64 requests at once, then two levels in between where the limit lies, two runs per level, 45 seconds warm-up and 240 seconds measured.
  • The waiting limit. A call counts as good when its first word arrives within 2 seconds and the answer then flows at 10 words a second or more [4]. A load level counts only if 99 % of its calls are good.
  • Quality. Every answer is compared with a fixed key. No AI judges another AI. An answer cut off at the length limit counts as wrong.
  • Prices. The API side is the public price list of STACKIT, a German cloud, for the same open models, without VAT [1]. I did not run anything against that API. Electricity is the German business price [2], labour the German average cost per hour worked [3].
  • Power. Read at the wall by a smart plug, for the whole machine: 85 W when idle with the model loaded, 500 to 740 W under load.

Hypotheses

Five hypotheses, set before the runs. Two did not hold.

I fixed the method and the hypotheses before the card ran, so that I could not choose the story afterwards. Both hypotheses that did not hold overrated the owned card.

Hypothesis, set before the runsMeasuredResult
1.Under load, one card does at least 10 times the work of one request at a time.11 times for short answers, 8 times for agent questions.Partly confirmed
2.At least 80 % of an agent’s input comes from the card’s cache, and that doubles its throughput.0.6 % at the limit, and the cache made no measurable difference.Not confirmed
3.Answers under load are within 5 points of answers at one request at a time.Within 3.3 points on all four tasks.Confirmed
4.Owning pays from 1,500 to 4,000 agent questions a day, and from more than 10,000 short answers.2,846 agent questions and 22,409 short answers.Confirmed
5.One card can do at least 50 times the break-even for agent questions.37 times over 24 hours, 12 times in an 8-hour day.Not confirmed

Full results

Every task, every load level.

Exact figures. The same numbers are in the repository as CSV, one row per run, per answer and per call.

Cost and break-even: every task, both models
Model, taskTokens inTokens outAPI price per taskInput share of the priceFixed cost per working dayElectricity per taskBreak-even, hardware only1 staff hour a month5 staff hours a monthCapacity in 8 hoursCapacity in 24 hours
Qwen3.8-27B, sorting a message4515€0.0002198%€6.60€0.00000232,38742,98385,370578,9041,736,712
Qwen3.8-27B, short answer105403€0.0003115%€6.60€0.00001522,40929,74159,06860,766182,299
Qwen3.8-27B, agent question4,678382€0.0023589%€6.60€0.0000342,8463,7787,50334,733104,198
Qwen3.8-27B, long solution1441,957€0.001345%€6.60€0.0001025,3437,09114,0839,05827,173
gpt-oss-20b, agent question5,126319€0.0008591%€6.64€0.0000047,86010,41720,645318,403955,208

Break-even = fixed cost per working day ÷ (API price per task − electricity per task). API prices: Qwen3.8-27B €0.45 in and €0.65 out per million tokens, gpt-oss-20b €0.15 and €0.25 [1]. Capacity assumes the card runs at its limit for the whole period.

What moves the agent break-even: 17 changes
Change, one at a timeBreak-even, agent questions per working dayAgainst the base case
Base case: hardware and idle power only2,846×1.00
Staff, 1 hour a month3,778×1.33
Staff, 5 hours a month7,503×2.64
Staff, 10 hours a month12,160×4.27
Write-off over 2 years instead of 34,118×1.45
Write-off over 5 years instead of 31,829×0.64
Resale at 30% of the price after 3 years2,084×0.73
Cost of capital, 5% a year on the purchase price3,228×1.13
A second machine for redundancy5,693×2.00
Machine price incl. 19% VAT (a private buyer)3,330×1.17
Electricity at 0.15 EUR per kWh2,721×0.96
Electricity at 0.35 EUR per kWh3,013×1.06
Tasks arrive on 365 days a year, not 2501,950×0.68
API price halved5,778×2.03
API price doubled1,413×0.50
Cached input billed at a quarter (some providers)8,916×3.13
Token count of yt-experiment-001 (5,613 per question)2,567×0.90
All of: 5 staff hours, second machine, 2-year write-off12,892×4.53
Quality under load, with ranges
TaskOne request at a timeAnswersAt the limitAnswersDifference in points, with its 95 % range
Agent question98.8%8499.4%673+0.6 (-0.8 to +5.9)
Long solution100.0%1896.7%181-3.3 (-7.0 to +14.4)
Short answer100.0%10297.7%1,196-2.3 (-3.4 to +1.4)
Sorting a message75.8%4,22876.8%11,449+1.0 (-0.5 to +2.5)

The range is a 95 % interval for the difference of two independent shares. Every range includes zero.

Load levels: agent question, Qwen3.8-27B
At onceTasks per hourCalls inside the waiting limitFirst word, slowest 1%Time per word, slowest 1%CorrectInput from cacheContext memory in useRunning at once, peakPower at the wallUsable
1540100.0%0.33 s15 ms98.8%29.4%8%1522 Wyes
41,815100.0%0.46 s22 ms98.9%26.6%27%4556 Wyes
164,342100.0%0.90 s40 ms99.4%0.6%100%16712 Wyes
214,34992.5%2.42 s42 ms98.8%0.0%100%19717 Wno
274,3343.5%4.21 s43 ms99.5%0.0%100%19691 Wno
324,3490.0%5.33 s43 ms99.4%0.0%100%19706 Wno
644,4770.0%14.87 s44 ms98.6%0.0%100%19718 Wno
Load levels: long solution, Qwen3.8-27B
At onceTasks per hourCalls inside the waiting limitFirst word, slowest 1%Time per word, slowest 1%CorrectInput from cacheContext memory in useRunning at once, peakPower at the wallUsable
1120100.0%0.09 s15 ms100.0%0.0%9%1528 Wyes
4337100.0%0.13 s17 ms94.5%0.0%32%4514 Wyes
8945100.0%0.18 s17 ms96.5%0.0%54%8558 Wyes
121,132100.0%0.29 s18 ms96.7%0.0%82%12571 Wyes
161,28272.5%9.96 s23 ms97.0%0.0%100%16584 Wno
321,3424.5%70.61 s26 ms98.2%0.0%100%19585 Wno
641,2825.3%146.39 s28 ms97.6%0.0%100%19588 Wno
Load levels: short answer, Qwen3.8-27B
At onceTasks per hourCalls inside the waiting limitFirst word, slowest 1%Time per word, slowest 1%CorrectInput from cacheContext memory in useRunning at once, peakPower at the wallUsable
1690100.0%0.09 s15 ms100.0%0.0%5%1517 Wyes
41,980100.0%0.13 s17 ms96.2%0.0%23%4504 Wyes
167,596100.0%0.16 s19 ms97.7%0.0%87%16566 Wyes
218,24891.1%2.95 s20 ms97.3%0.0%100%19577 Wno
278,2783.4%6.20 s20 ms97.8%0.0%100%19586 Wno
328,4060.0%8.74 s20 ms97.4%0.0%100%19583 Wno
648,1740.0%26.68 s20 ms97.7%0.0%100%19580 Wno
Load levels: sorting a message, Qwen3.8-27B
At onceTasks per hourCalls inside the waiting limitFirst word, slowest 1%Time per word, slowest 1%CorrectInput from cacheContext memory in useRunning at once, peakPower at the wallUsable
126,688100.0%0.09 s16 ms75.8%0.0%5%1600 Wyes
456,782100.0%0.20 s51 ms76.7%0.0%21%4719 Wyes
872,36399.9%0.32 s84 ms76.8%0.0%42%8739 Wyes
1278,63894.6%0.42 s115 ms76.7%0.0%60%12742 Wno
1681,54154.5%0.53 s146 ms76.5%0.0%75%16742 Wno
3283,6223.1%1.09 s169 ms76.5%0.0%99%19738 Wno
6483,6380.0%2.46 s169 ms76.6%0.0%91%19738 Wno
Load levels: agent question, gpt-oss-20b
At onceTasks per hourCalls inside the waiting limitFirst word, slowest 1%Time per word, slowest 1%CorrectInput from cacheContext memory in useRunning at once, peakPower at the wallUsable
12,850100.0%0.09 s4 ms85.0%82.7%0%1586 Wyes
46,689100.0%0.13 s6 ms88.8%83.7%1%4594 Wyes
1617,502100.0%0.19 s10 ms86.2%83.7%3%16680 Wyes
3227,670100.0%0.23 s13 ms87.2%83.7%7%32724 Wyes
6439,800100.0%0.32 s18 ms85.9%83.5%12%64727 Wyes

Every level stayed inside the waiting limit, so the true limit of this model is above 64 agents at once. The two models ran on different versions of the serving software.

The cache switched off
Task, at oncePer hour with the cachePer hour with the cache defeatedWith ÷ without
Agent question, 1540600×0.90
Agent question, 164,3424,491×0.97
Sorting a message, 126,68826,590×1.00
Sorting a message, 872,36370,337×1.03
Sorting a message, 1681,54178,439×1.04

Every request got a unique first line, so nothing could be reused. Throughput barely changed: on this setup the cache did not help.

Limits

A measurement of this setup, not of GPUs in general.

  • One machine, one night. One consumer card, two runs per load level. Another card, driver or software version will give other capacities.
  • The limit of 19 requests is specific. It comes from how this version of the serving software divides the card’s memory for this model. Another version may fit more.
  • A compressed model. The 27B model ran as a 4-bit copy and was not compared with full precision.
  • List prices, no API run. The API side is a price list applied to the tokens my card counted. Its speed and its answers were not measured.
  • Even load. The test keeps a constant number of requests in flight. Real traffic comes in bursts, which lowers what a card can take.
  • Benchmarks stand in for work. The tasks have answer keys, real business tasks rarely do. The agent task is synthetic.
  • A simple cost model. No cost of capital, no resale value, no spare machine, no room or cooling in the base case. The table under “Full results” shows what each would change.

Sources

Where the other numbers come from.

  1. STACKIT (Schwarz Digits) (2026). STACKIT AI Model Serving, price table; price list v1.0.45. stackit.com/en/products/data-ai/stackit-ai-model-serving. Accessed 13 November 2026.
  2. Eurostat (2026). Electricity prices for non-household consumers, bi-annual data (nrg_pc_205), Germany, 20–499 MWh, first half of 2026. ec.europa.eu/eurostat/databrowser/view/nrg_pc_205/default/table. Accessed 11 November 2026.
  3. Statistisches Bundesamt (Destatis) (2026). Eine Arbeitsstunde kostete im Jahr 2025 durchschnittlich 45,00 Euro. Press release no. 148. destatis.de/DE/Presse/Pressemitteilungen/2026/04/PD26_148_624.html. Accessed 14 November 2026.
  4. MLCommons (2026). MLPerf Inference Rules: the waiting limits for conversational use. github.com/mlcommons/inference_policies/blob/master/inference_rules.adoc. Accessed 13 November 2026.
  5. DeepInfra (2026). Pricing: cached input at a quarter of the input price. deepinfra.com/pricing. Accessed 13 November 2026.
  6. Kalun Ho (2026). Cheaper Tokens, Costlier Tasks: Measuring Tokens per Task. Experiment 1. kalunho.com/experiments/tokens-per-task. Accessed 13 November 2026.
  7. Kwon, W. et al. (2023). Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM). SOSP 2023. arxiv.org/abs/2309.06180. Accessed 11 November 2026.
  8. Patil, C. (2026). Beyond Per-Token Pricing: A Concurrency-Aware Methodology for LLM Infrastructure Cost Estimation. arxiv.org/abs/2606.11690. Accessed 13 November 2026.
  9. Pan, G. et al. (2025). A Cost-Benefit Analysis of On-Premise Large Language Model Deployment. arxiv.org/abs/2509.18101. Accessed 13 November 2026.
  10. Microsoft Azure (2024). Azure LLM inference trace 2024. github.com/Azure/AzurePublicDataset/blob/master/AzureLLMInferenceDataset2024.md. Accessed 13 November 2026.
  11. Qwen Team, Alibaba (2026). Qwen3.8-27B model card. huggingface.co/Qwen/Qwen3.8-27B. Accessed 13 November 2026.
  12. OpenAI (2025). gpt-oss-20b model card. huggingface.co/openai/gpt-oss-20b. Accessed 13 November 2026.
  13. NVIDIA (2025). License for Customer Use of GeForce Software. nvidia.com/en-us/drivers/geforce-license. Accessed 13 November 2026.

Data and code

Check it, rerun it, cite it.

The test code, the answer keys and the raw results, one row per run, per answer and per call, are on GitHub, together with a technical report that adds the full method, confidence intervals, the hypotheses as written before the runs and every deviation from the plan. One command prices the same measurements with your machine and your provider. Questions and corrections are welcome there as issues.

How to cite

Kalun Ho (2026). Count Your Tasks First: The Break-Even Between an Owned GPU and a Per-Token API, per Task Type, Under a Latency Constraint. Technical report. https://github.com/kalunho17/yt-experiment-002

Licences: code MIT; results, figures, report and the invoice ledgers CC BY 4.0. The benchmarks keep their own licences: Banking77 (PolyAI, CC BY 4.0), GSM8K (OpenAI, MIT), MATH-500 (HuggingFaceH4, MIT).

The video explains what it means for your budget.

Subscribe on YouTube to see it. The handout has every chart and source.