Models & inference
Ollama
An open-source tool and API for running open models on your own machine or on Ollama's cloud.
What Ollama does
Ollama runs open-weight language models. Install it on macOS, Linux or Windows (or use the official Docker image), pull a model from the Ollama library, and chat with it from the command line or call it over a local HTTP API on port 11434. The project is MIT-licensed on GitHub, and running models on your own hardware has no usage cap.
The same tool can also send requests to Ollama's cloud. Cloud models run on Ollama's servers, so they need no download or local GPU; in the CLI you add a :cloud suffix to the model name, and from code you call the API at ollama.com with an API key. Ollama's API has OpenAI- and Anthropic-compatible endpoints, plus official Python and JavaScript libraries.
The docs cover hooking Ollama into other tools. Listed integrations include coding agents such as Claude Code, OpenCode, Codex CLI and Cline, desktop apps such as Claude Desktop and ChatGPT Desktop, and editors such as VS Code, JetBrains, Xcode and Zed. Cloud usage is paid with credits priced per million tokens, with cheaper off-peak rates for some models.
Ollama suits developers who want to run open models locally for privacy or offline work, and those who want one API for both local and hosted open models. How fast a local model runs depends on your hardware, and Ollama's download page warns that large models are slow without a strong GPU.
- Run an open model locally for chat or coding without sending prompts off the machine.
- Point a coding agent such as Claude Code or Codex CLI at a local or cloud open model.
- Call local or hosted open models from an app using Ollama's API or its OpenAI- and Anthropic-compatible endpoints.
- Use larger open models on Ollama's cloud from a computer without a strong GPU.
Who should use it
- Developers who want to run open models on their own hardware.
- Teams that want the same API for local and hosted open models.
Who should look elsewhere
- People who need closed frontier models, since Ollama focuses on open models.
- Users on low-powered machines who expect large models to run quickly locally.
Strengths
- MIT-licensed, with unlimited use of models on your own hardware.
- Same CLI and API for local and cloud models.
- OpenAI- and Anthropic-compatible endpoints plus Python and JavaScript libraries.
- Documented setups for many coding agents, desktop apps and editors.
Limitations
- Local speed depends on your hardware, and large models are slow without a strong GPU.
- The Free plan's cloud access covers a smaller set of starter models until credits are bought.
- Included monthly cloud credits do not roll over.
- Some listed plan features, such as fast mode and shared team projects, are marked coming soon.
Privacy notes
Ollama's privacy policy says it does not collect, store or access prompts and responses processed locally. Cloud prompts and responses are processed transiently, not stored beyond the request, and not used to train AI models.
Sources
- GitHub - ollama/ollama: Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. · GitHubRepositoryPrimarygithub.com
- OllamaOfficial sitePrimaryollama.com
- Pricing · OllamaPricing pagePrimaryollama.com
- Ollama documentation - OllamaOfficial docsPrimarydocs.ollama.com
- ollama.com/downloadWeb
- docs.ollama.com/cloudOfficial docsPrimary
- docs.ollama.com/integrationsOfficial docsPrimary
- docs.ollama.com/llms.txtOfficial docsPrimary
- ollama.com/privacyWeb
Ollama alternatives
LM Studio
Models & inference
Desktop apps for running open LLMs locally, with a local API server and Bionic, an agent for documents, code and automation.
FreemiumFree tierAPIVerifiedReplicate
Models & inference
A cloud API for running, fine-tuning and deploying open-source and proprietary AI models, billed by usage.
PaidAPIVerifiedLloyal
Models & inference
A TypeScript framework for AI apps that run an open-weight model inside the app, offline and without an API key.
Open sourceFree tierVerifiedClef
Models & inference
Cloudflare's open-weight decision models that return typed answers with probabilities, hosted on Workers AI.
Open sourceFree tierAPIVerifiedej
Models & inference
An 11 MB open-weight model from HITL Systems that answers routing, yes/no and priority questions about text on CPU.
Open sourceVerifiedPocketWebTools for Mac
Models & inference
Mac app that runs open AI models up to 35B parameters offline for chat, transcription, OCR, upscaling and video clipping.
PaidVerified
Related open-source projects
VoiceStudio is a desktop app that runs speech AI on your own computer: it clones and designs voices, generates speech from text, dubs videos, transcribes audio and has a dictation widget. The app is built on Electron with a Python backend. Its default speech engine is k2-fsa's OmniVoice, and the feature catalog lists more than a dozen alternatives, among them CosyVoice 3, KittenTTS, MLX-Audio and IndexTTS 2.5. Transcription defaults to WhisperX, with Faster-Whisper, Parakeet, Moonshine and others available. Models are downloaded when a workspace first needs them. The project says its tools cover 646 languages, a figure that also appears in the transcription tool of its MCP docs. Beyond the desktop workspaces, VoiceStudio exposes a local API and an MCP server on port 3900, so coding agents can generate speech, clone a voice or transcribe a file. The README states that local workflows stay on your hardware, remote workers are optional and usage analytics only run with consent. Version 0.5.3 was the last release built on Tauri; current releases are Electron only, and Tauri users have to install the new app separately.
- Stars
- 61K
- Language
- Python
- License
- AGPL-3.0
- Momentum
- +16.0% 7dTrending
oMLX is a local LLM inference server for Macs with Apple Silicon, built on MLX and managed from a menu bar app. It exposes OpenAI-compatible and Anthropic-compatible endpoints, so existing clients can connect to http://localhost:8000/v1 without code changes. Its main idea is a two-tier KV cache. Frequently used cache blocks stay in RAM, and when that hot tier fills up, blocks are offloaded to the SSD in safetensors format so they can be reused instead of recomputed. The author pitches this at long coding sessions with tools such as Claude Code. Concurrent requests go through continuous batching using mlx-lm's BatchGenerator. The server loads text LLMs, vision-language models, OCR models, embeddings and rerankers. Several models can stay loaded at once, with least-recently-used eviction, pinning and a per-model idle timeout. A web admin dashboard at /admin covers monitoring, model management, chat and benchmarks. Source builds also include an experimental mode that splits one model across several Macs. You can install it as a .dmg from GitHub Releases, through Homebrew, or from source, and it is licensed under Apache-2.0.
- Stars
- 23K
- Language
- Python
- License
- Apache-2.0
- Momentum
- Established
llmfit is a terminal tool, written in Rust, that inspects your hardware and tells you which open-source large language models it can run well. It detects CPU cores, system RAM, GPUs, VRAM and unified memory (NVIDIA CUDA, Apple Silicon, AMD ROCm and Intel OneAPI), then estimates memory use and tokens per second for each model across quantizations. Each model gets scores for quality, speed, fit and context, plus a fit verdict that runs from Perfect to Too Tight. The default interface is an interactive TUI. A classic CLI mode, a web dashboard and a REST API (/api/v1/system and /api/v1/models) are also available. llmfit works with Ollama, llama.cpp, MLX, Docker Model Runner and LM Studio, and can download a chosen model through them. Speed figures come from a memory-bandwidth formula unless a measurement exists. A newer benchmark feature measures real tok/s on your machine and lets you submit the result to the project as a pull request from the TUI, so later releases carry measured numbers for matching hardware. The tool runs on macOS, Linux and Windows and is MIT licensed. The latest release is v1.1.17.
- Stars
- 38K
- Language
- Rust
- License
- MIT
- Momentum
- Established
AI Engineering from Scratch is a free, MIT-licensed course on GitHub that teaches how AI systems work by having you write each piece yourself, starting with linear algebra and ending with multi-agent systems and production infrastructure. The material is split into 20 phases and 523 lessons, with code in Python, TypeScript, Rust and Julia. Each lesson folder holds a written explanation, runnable code and an outputs directory. The pattern is to build an algorithm such as backpropagation, a tokenizer, attention or an agent loop from plain math first, and only then run the same idea through a library like PyTorch or scikit-learn. Lessons end with something you keep: a prompt, a SKILL.md file, an agent or an MCP server. The repository holds 396 skills and 99 prompts produced this way, plus a script that copies them into an agent's skills folder. There are three ways to follow it. You can read lessons on the website with no setup, clone the repository and run the code locally, or install the course's tutor skills into a coding agent such as Claude Code or Codex. Those skills run a placement quiz, write a study plan to LEARNING.md and teach one lesson per session. Separate tracks prepare for the four Claude certification exams and the MCP Associate exam; the README says these are independent and not affiliated with the exam bodies.
- Stars
- 68K
- Language
- Python
- License
- MIT
- Momentum
- +8.2% 7dEstablished
Related guides
NewsAssistants & chatCharacter.ai enters Chapter 11 and puts the company up for a court-supervised sale
Character.ai began a Chapter 11 sale process on October 12, 2026. What stays the same for users for now, and how to export chats before the sale ends.
3 min read
- NewsModels & inference
NVIDIA releases Nemotron 3.5 Super VL and a 24GB Bolt model for local agents
NVIDIA released Nemotron 3.5 Super VL, a 120B open vision model, and Bolt 74B-A9B for 24GB local use. What developers can run now and what is missing.
4 min read
- TutorialModels & inference
Fine tune LLM LoRA adapters: data, training and a first test on one GPU
Fine tune LLM LoRA adapters on one GPU: check that fine-tuning fits, prepare data, train with PEFT and TRL or Unsloth, and compare against the base model.
6 min read
- TutorialEvaluation & observability
How to evaluate a RAG pipeline with Ragas: metrics, a test set and a first run
Set up RAG evaluation with Ragas: build a small test set, match each metric to retrieval or generation, run a first score and compare two versions.
7 min read
- News
Aleph Alpha releases Kolibri, an open-weight German and English MoE model
Aleph Alpha released Kolibri, a 78B MoE model with 3.46B active parameters and Apache 2.0 weights. What changed, what it needs to run, and what is unknown.
4 min read
- News
NVIDIA adds a 64GB DGX Spark for local AI, with two-unit clustering
NVIDIA's DGX Spark gets a 64GB configuration at $4,999 from six partners, plus two-unit clustering. What changes for local AI developers.
4 min read