You want an assistant that can summarize your notes, help with scripts and coordinate home automations. There are three ways to run the “brain”, and the right one depends on how private, cheap and capable it needs to be.
The three options
| Local (your PC or server) | VPS you rent | Cloud API (OpenAI, Anthropic, Google, xAI…) | |
|---|---|---|---|
| Where your prompts go | Stay on your hardware | A provider’s server you control | The AI provider (check their data policy) |
| Model quality | Open-weight models (Llama, Qwen, Mistral, Gemma…). Good, but generally behind the top cloud models | Same as local, limited by the VPS hardware | Strongest models |
| Upfront cost | A GPU, if you want speed | None | None |
| Running cost | Electricity | Monthly VPS fee (GPU VPSs are expensive) | Pay per use |
| Setup effort | Medium (Ollama makes it easy) | Medium to high (you secure a public server) | Low |
| Works offline | Yes | No | No |
Hardware rules of thumb (local)
- Memory is what matters most. A model must fit in GPU VRAM (or system RAM, if it runs on the CPU). Quantized models (“Q4”) need roughly half a gigabyte per billion parameters, plus overhead. A 7-8B model fits comfortably on an 8 GB GPU. Bigger models need more.
- CPU-only works, just slowly. That’s fine for overnight summaries, painful for chat. See our GPU vs CPU in Proxmox guide.
- Apple Silicon Macs share memory between the CPU and GPU, which makes them capable local hosts.
Privacy and security
- Local is the most private, but only if you don’t expose it to the internet. Ollama’s API has no authentication by default. Keep it on localhost or your LAN, or behind Tailscale. Our exposing Ollama safely post covers this.
- VPS: you’re now running a public server. Patch it, use SSH keys, put a firewall in front, and never leave the model API open.
- Cloud API: read the provider’s data-use and retention terms for API traffic. Don’t send secrets, and use a separate API key per project with a spending limit.
A sensible hybrid
Many home setups do this: a local model for private, routine work (summarizing your notes, logs and emails) and a cloud model for hard tasks where you only send non-sensitive context. A small router script decides which one to use.
Setup in brief (local)
- Install Ollama and pull a model:
ollama pull llama3.1:8b(or a current Qwen or Gemma model). - Add a UI: Open WebUI, on your LAN only.
- Connect automations (Home Assistant, scripts) through the local API.
- Track updates. Ollama and Open WebUI release often, and both are on SafeToUpdate.
Decision guide
- Privacy first, and you have a decent GPU or Mac: go local.
- No suitable hardware, want it always on, and comfortable hardening a server: VPS, ideally a CPU VPS with a small model, reached over Tailscale.
- Want the best answers with the least setup: cloud API, careful about what you send.
Model names and hardware needs change quickly. Check the model card for current memory requirements.