✨Ollama
Ollama with a GUI: discover, download and chat locally.
✨llama.cpp
The engine almost everything else runs on. Plain C++, no dependencies.
✨vLLM
The serving engine production runs on. PagedAttention and continuous batching, on your own GPU.
✨MLX
Apple's array framework. On an M-series Mac it is the fastest way to run a model locally.
✨llamafile
One file that is both the model and the runtime. Download, chmod +x, run. No install at all.
✨KoboldCpp - Run GGUF AI models locally with a KoboldAI UI. One File. Zero Install.