Tell us your machine and how many people will use it — we'll tell you which open-source models you can run, how fast they'll feel, what to buy, what it costs to keep running, and how to set it all up. No jargon required.
Three simple doors into the same engine. Every answer updates live as you move the sliders.
family, team or company members who will chat with your AI
Speed and fit are estimates from published hardware specs — real numbers vary with software versions. Tune the details in Advanced settings.
Capable — everyday assistant quality
CapEx is what you pay once; OpEx is what you pay every month. Home mode keeps it minimal; enterprise mode adds the team and upkeep costs that on-prem deployments really carry.
Costing: Example — Llama 3.1 8B on a used RTX 3090 (24 GB) (pick something in the calculator above to replace this example)
Pick your level of ambition. Level 1 genuinely takes about fifteen minutes.
The easiest path. Ollama is a free app that downloads and runs models for you — no configuration files, no code.
ollama run llama3.1:8bOLLAMA_HOST=0.0.0.0 ollama serveTwo or more graphics cards can pool their memory to hold a model neither could hold alone. The software slices the model across cards — you don't manage that yourself.
vllm serve meta-llama/Llama-3.3-70B-Instruct --tensor-parallel-size 2 --quantization awqWhen one box isn't enough, you run several and need a coordinator. That's where Kubernetes comes in (explained below) — for nano scale, use its lightweight cousin k3s.
curl -sfL https://get.k3s.io | sh -Honest answer: probably not until you have two+ machines AND uptime requirements. A single strong box with vLLM serves 50+ light users. Kubernetes earns its complexity when downtime costs money or you juggle several models across several machines.
VRAM, quantization, Kubernetes — the nine ideas behind this whole page, one card each.
The "brain" you download. Bigger = smarter but hungrier.
An open-source model is a single big file of numbers ("parameters"). An 8B model has 8 billion of them. More parameters generally means better answers — and more memory needed. The open-source community (Meta, Mistral, Qwen, DeepSeek…) releases these free for anyone to run.
Your graphics card's own memory — the #1 thing that decides what you can run.
The whole model must sit inside the GPU's memory (VRAM) to run fast. A card "with 24 GB" can hold roughly a 30B-parameter model at 4-bit compression. Regular RAM can substitute, but it is ~10× slower — that's why VRAM is the first number this calculator asks about.
Smart compression: the same brain in half the memory, barely dumber.
Models are trained with very precise numbers (16-bit). Quantization rounds them to 4–8 bits, shrinking memory 2–4× with only a small quality loss. Q4 (4-bit) is the community sweet spot — it's why a 70B model fits on two consumer GPUs at all.
AI writes in tokens (~¾ of a word). 12 tokens/sec feels like reading speed.
Every response streams out token by token. Below ~5 tok/s users get impatient; 10–15 tok/s matches reading speed; 30+ feels instant. Generation speed is mostly limited by how fast the GPU can re-read its memory — which is why memory bandwidth, not raw compute, rules this calculator.
The software that serves the model. Ollama = easy, vLLM = many users.
Ollama (built on llama.cpp) is a one-click app, perfect for one to a handful of users. vLLM is server software that interleaves many conversations at once ("continuous batching"), serving 10–100× more users from the same GPU — at the cost of a slightly more technical setup.
Two cards pool their memory; the software slices the model across them.
With "tensor parallelism" every layer of the model is split across your GPUs, so 2× 24 GB cards behave like one 48 GB card (with a small efficiency tax). You enable it with a single flag — the setup guide shows exactly how. Separate computers can NOT pool memory this way; that requires the cluster tools below.
An "operations manager" for a fleet of machines. Overkill for one box.
Kubernetes (K8s) is software that treats several computers as one pool: you declare "I want the AI service running with 2 copies", and it decides which machine runs what, restarts crashes, rolls out updates without downtime, and gives users one stable address. In a nanodatacenter its role begins when you have 2+ machines and uptime matters — and the lightweight k3s edition is the right size. One home PC running Ollama does not need Kubernetes, the same way one food truck does not need an airport control tower.
How much conversation the AI remembers — it costs memory per user.
The context window is the AI's short-term memory (2K tokens ≈ 3 pages, 128K ≈ a novel). While answering, the GPU keeps a "KV-cache" for every active conversation — so memory needs grow with both context length AND simultaneous users. That's why the calculator asks how many people use the system at once.
Feed it clean power, and remember: every watt becomes heat.
A big GPU draws 300–450 W under load; the power supply (PSU) should have ~1.7× headroom. A small UPS (battery) rides through flickers so the machine never hard-crashes. All consumed power turns into heat in the room — in hot climates the air conditioner then spends extra energy removing it (the "cooling factor" in Advanced settings).
Hyperscale planners calculate megawatts, hectares and water rights. At nano scale the same physics collapses into different, friendlier questions — here's the side-by-side.
The takeaway: hyperscale calculators optimize megawatts, land and water; a nanodatacenter calculator optimizes VRAM fit, felt speed per user, one power circuit, and a realistic monthly bill. Same physics, very different knobs.
Everything above is an estimate built on the assumptions below. If you know your real electricity tariff, cooling situation or workload shape, set it here and every number on the page updates.
your actual tariff beats our preset
1.1 temperate room · 1.4 hot climate with AC
Hot climates (Gulf summers) make the air-conditioning work harder for every watt the GPUs draw — that's the cooling factor. Cheap electricity (Saudi Arabia ≈ $0.05/kWh) can make self-hosting dramatically cheaper to run than in Germany (≈ $0.40/kWh).
of total users, typing at the same moment
~12 tok/s matches human reading speed