Ollama

The default choice for local LLM inference — and for good reason. 176K GitHub stars, 9M+ users, MIT license, one-command setup. Runs on any hardware from Raspberry Pi 5 to dual H100s, with a model catalog spanning 4,500+ open-weight options. The OpenAI-compatible API with streaming, tool calling, structured outputs, and embeddings means no vendor lock-in — swap from Ollama to OpenAI (or vice versa) by changing one URL. Ollama Cloud (Pro $20/mo, Max $100/mo) extends the same surface to managed inference. The recent $65M raise confirms sustained investment. For any developer who wants to run AI locally — whether for privacy, cost control, or offline use — Ollama is where you start.

Each inference request is a stateless REST API call — no carried AI context between requests