Private LLM hosting on dedicated GPU servers
Run your own large language models on a dedicated NVIDIA GPU server instead of paying per token to a third-party API. With LLM hosting at IPHOST, the model, the prompts and the answers stay on hardware that only you use, in our ISO/IEC 27001 certified data center in Chișinău. You get full root access, a fixed monthly price and no limits on requests or tokens.
Which GPU for which model
The amount of GPU memory (VRAM) decides which models you can run. As a rule of thumb, a model needs about 2 GB of VRAM per billion parameters in FP16, about 1 GB in 8-bit and about 0.6 GB in 4-bit, plus room for the context.
| Model size | Example models | Recommended plan |
|---|---|---|
| 7–8B in FP16 | Llama 3.1 8B, Mistral 7B, Qwen 2.5 7B | NVIDIA L4 24 GB |
| 13–32B in 4-bit | Qwen 2.5 32B, DeepSeek-R1 Distill 32B, Gemma 2 27B | NVIDIA L4 24 GB or A40 48 GB |
| 13–20B in FP16 | Phi-4 14B, Qwen 2.5 14B | NVIDIA A40 48 GB |
| 70B in 4-bit | Llama 3.3 70B, Qwen 2.5 72B | NVIDIA A40 48 GB |
| 70B in 8-bit, fine-tuning | Llama 3.3 70B, LoRA on 13–34B models | 2x NVIDIA A100 40 GB |
See the details for each card: NVIDIA L4 dedicated server, NVIDIA A40 dedicated server and 2x NVIDIA A100 dedicated server.
Ollama, vLLM and an OpenAI-compatible API
Every GPU server comes with Linux, the NVIDIA driver and CUDA installed, so you can start a model in minutes:
- Ollama – the simplest way to download and run open models with one command, with a local REST API.
- vLLM – a high-throughput inference server for production, with batching, tensor parallelism across GPUs and an OpenAI-compatible API.
- llama.cpp – efficient inference for GGUF quantized models.
- Open WebUI – a ChatGPT-style interface for your team on top of Ollama or vLLM.
Because Ollama and vLLM expose an OpenAI-compatible endpoint, most applications built for the OpenAI API can switch to your own server by changing the base URL.
Self-hosted LLM or a paid API?
A paid API is convenient for occasional use, but the bill grows with every token. When your application sends requests all day, a dedicated GPU server usually costs less: the price stays the same whether you process a thousand or a million requests. Self-hosting also lets you choose and fine-tune open models, keep full control over updates and keep sensitive data out of third-party services.
What teams use it for
- Internal assistants and chatbots trained on company documents (RAG).
- Customer support automation and email classification.
- Code assistants for development teams.
- Document summarization, translation and data extraction.
- AI features inside SaaS products without per-token costs.
How to order
Choose a GPU plan below and the billing period: 3 months, or 12 months at up to 14% off. GPU servers are delivered on pre-order within 14 days; if we miss the date, you get a full refund. Compare all cards on the GPU server hosting page.
