A self hosted LLM sounds simple in theory: download the model, run it, done. In practice, the infrastructure underneath that decision determines whether it actually works as a production system or collapses the moment real usage arrives. The gap between “I ran a model once” and “this reliably serves my team or my customers” is almost entirely a question of what it runs on.
This guide explains why companies choose to self-host a large language model at all, what that choice actually demands from infrastructure, and why the answer is rarely a laptop, and just as rarely a cloud API.
📖 The broader case for dedicated AI infrastructure
Read How Dedicated Servers Support AI and Machine Learning Workloads for the foundational case before this guide’s specific focus on self-hosted LLMs.
Why Companies Self-Host an LLM At All
According to Kong’s 2025 enterprise research, surveying 550 IT leaders and engineers, 72 percent of enterprises expect to increase LLM spending this year, and 44 percent identify governance and security as their single greatest barrier to adoption. Every prompt sent to a third-party API leaves the company’s network, a dealbreaker for healthcare, finance, and any business handling regulated or sensitive data.
Self-hosting answers that directly: the model runs on infrastructure the company controls, prompts never leave the network, and there is no third-party retention policy to trust. Cost predictability is the second driver. API pricing scales per token, which compounds unpredictably at real usage volumes; infrastructure you control has a fixed, known cost regardless of how heavily it gets used.
What Self-Hosting Actually Demands From Infrastructure
This is where most first attempts fail. A VRAM pool large enough to hold the model’s full parameter set is non-negotiable, and it scales fast.
VRAM requirements scale with model size, roughly
| Model size | VRAM, FP16 | VRAM, INT4 quantised |
|---|---|---|
| 7B parameters | ~14 GB | ~4 GB |
| 13B parameters | ~26 GB | ~7 GB |
| 70B parameters | ~140 GB | ~35 GB |
| Real usage | Add 10-20% for KV cache and activations | |
Table showing VRAM requirements scaling with model size, from roughly 4GB for a quantised 7B parameter model to 140GB for a 70B parameter model at full precision, with a note to add 10-20% for real-world overhead.
Beyond VRAM, the model has to be available continuously, not just running when someone happens to be testing it. A production LLM serving a team or a customer-facing feature needs the same uptime discipline as any other production service, which a consumer GPU in someone’s home office was never built to provide.
📖 What full control over that infrastructure actually means
Read What Is Root Access? Why Full Control Actually Matters, since self-hosting an LLM stack means configuring every layer yourself.
Why a Home Setup or a Basic Cloud Instance Falls Short
A gaming PC under a desk is not a production environment. There is no redundancy, no guaranteed uptime, no data centre network, and a single power outage takes the whole system down. It is a genuinely useful way to experiment, and a genuinely poor way to serve a business-critical feature.
A basic cloud VM sits in an uncomfortable middle ground: virtualised GPU access often comes with the same noisy neighbour risk and variable performance covered elsewhere on this blog, while still charging cloud-style variable pricing that erodes the cost-control benefit that motivated self-hosting in the first place.
📖 The measured cost of that virtualisation overhead
Read Physical Server vs Virtual Server: What’s the Real Difference? for the actual overhead figures behind that claim.
📖 Why fixed, predictable infrastructure beats variable billing for steady workloads
Read Cloud Hosting vs Colocation: Where Dedicated Servers Fit, the same cost logic that applies directly to a continuously-running LLM.
A dedicated server removes both problems at once: the full GPU is yours, with no virtualisation layer and no other tenant competing for the same VRAM, running in an actual data centre with the uptime and network infrastructure a production system needs.
Three ways to run it, compared
| Home PC | Basic Cloud VM | Dedicated Server | |
|---|---|---|---|
| Uptime | None guaranteed | Provider SLA | Provider SLA |
| GPU access | Consumer card | Often shared/virtualised | Exclusive |
| Cost model | One-time, no redundancy | Variable, hourly | Fixed monthly |
| Production-ready | No | Partially | Yes |
Table comparing home PC, basic cloud VM, and dedicated server across uptime guarantees, GPU access exclusivity, cost model, and production readiness, with dedicated server the only option meeting all four criteria.
Infrastructure built to actually run this
Swify’s dedicated servers give your self-hosted LLM exclusive GPU access, full root access, and a European data centre with GDPR data residency by default. GPU configurations available on request.
→ Talk to Us About a ConfigurationFrequently Asked Questions
What is a self-hosted LLM?
A self-hosted LLM is a large language model running on infrastructure you control, rather than through a third-party API like OpenAI or Anthropic. The model weights live on your own hardware, and prompts never leave your network, giving full control over data privacy and eliminating per-token API costs.
Read How Dedicated Servers Support AI and Machine Learning Workloads for the broader infrastructure case.
Can I self-host an LLM on a regular VPS?
Technically for very small models, but it is rarely the right choice for production use. Most VPS plans lack dedicated GPU access entirely, and where GPU access exists, it is typically shared or virtualised, introducing the same resource contention risk that affects any virtualised infrastructure under load.
Read Physical Server vs Virtual Server: What’s the Real Difference? for the measured performance gap.
Why do companies choose self-hosting over just using the OpenAI or Anthropic API?
Primarily data privacy and cost predictability. According to Kong’s 2025 enterprise research, 44 percent of organisations cite governance and security as their top barrier to LLM adoption, since every API request sends data to third-party infrastructure. Self-hosting also converts unpredictable per-token costs into a fixed infrastructure expense.
How much VRAM do I need to self-host an LLM?
It depends entirely on model size. A 7B parameter model needs roughly 14GB at full precision, or around 4GB quantised. A 70B parameter model needs roughly 140GB at full precision, or around 35GB quantised, plus 10-20% overhead for real-world usage. The model must fit entirely in VRAM to run at all.
Read Setting Up a Dedicated Server for Machine Learning for the complete setup sequence.
Does self-hosting an LLM help with GDPR compliance?
Yes, directly. Self-hosting on a server within the EEA keeps all prompt data and model activity within EU jurisdiction, avoiding the GDPR data transfer questions that arise when prompts are sent to a third-party API’s infrastructure, which may be located outside the EEA entirely.
Read Dedicated Servers and GDPR: What You Need to Know for the full picture.
Is self-hosting an LLM cheaper than using an API long-term?
Not automatically, and not immediately, but often yes at sustained volume. API pricing scales per token and compounds as usage grows, while dedicated infrastructure carries a fixed monthly cost regardless of how heavily it is used. The crossover point depends on usage volume, but consistent, high-volume usage generally favours owned infrastructure.
Read Cloud Hosting vs Colocation: Where Dedicated Servers Fit for the same cost logic applied more broadly.