I built a one-click installer for local LLMs on Windows: it picks a model for your hardware and sets up Ollama

dev.to AI2026-11-03 15:270 views

Running an LLM locally is not hard, but the first time is a small quest: install Ollama, figure out which of a hundred models fits your GPU, get the tag right, wait for gigabytes to download, find a UI. I kept getting asked to set this up for friends, so I built Local AI: a free, open-source (MIT) Windows app that looks at your hardware, picks a model, installs Ollama and the model, and opens a chat.

What it does

One LocalAI.exe (about 42 MB), no Python or anything else to install first.
Detects the GPU, VRAM, RAM and free disk space, and recommends a model from a catalogue of 34 (Qwen3, Gemma 3/3n, Llama 3.1/3.2, Mistral, Phi-4, DeepSeek-R1, Qwen2.5-Coder and -VL, LLaVA, gpt-oss).
Downloads and installs Ollama without administrator rights, then pulls the model with progress.
Chat with streaming replies, a "think" mode, attachments (text files, and images for vision models), and an "about this model" panel.
Choose folders for Ollama and models, move models to another drive, update everything, and remove it all without a trace.
English and Russian UI, two themes.

Conversations never leave the PC: the chat talks to Ollama on 127.0.0.1. The network is only used for downloads and update checks.

How it is built

It is Python plus pywebview (WebView2). main.py starts a local HTTP server on 127.0.0.1 and opens the window; the UI is plain HTML/CSS/JS with no build step. API requests are protected by a per-session token header and a Host check, so a web page in your browser cannot call the local server.

Some parts were more fun than I expected:

Hardware detection without WMI. WMI is broken on my machine and probably on some of yours, so the GPU comes from nvidia-smi or the display-class registry key, and RAM from GlobalMemoryStatusEx via ctypes.
Model choice. Each catalogue entry has a size and a rough memory estimate (file size x 1.2 + 0.6 GB). If it fits in VRAM it is a GPU model, if it fits in 70% of RAM it runs mixed or on CPU, otherwise it is hidden.
Resumable download of the 1.6 GB Ollama installer with Range requests, plus a signature check before running it.
No stray console windows: ollama serve must be started with CREATE_NO_WINDOW and without DETACHED_PROCESS, otherwise every child process opens its own black console.

Limits and download

Windows 10/11 only. Without a GPU it runs on the CPU, slower and with a smaller model. The app is not code-signed, so SmartScreen may ask you to confirm ("More info" → "Run anyway"), and some antivirus tools dislike PyInstaller executables; the source is open if you prefer to build it yourself.