TutorialModels & inference
Serve an open model with vLLM behind an OpenAI-compatible API
Install vLLM, start a vLLM OpenAI compatible server with vllm serve, tune GPU memory and context length, and point the OpenAI SDK at your own GPU.
6 min read
Blog, page 4
Posts 38 to 46 of 46, newest first. Every post lists the sources it was built from.
- RoundupAudio & speech
Open source text to speech you can run yourself: Piper, Kokoro and more
Compare open source text to speech engines Piper, Kokoro, Coqui TTS and Chatterbox by licence, languages and hardware, then pick one you can ship.
7 min read
- GuideAudio & speech
Transcribe audio locally with Whisper: openai-whisper, whisper.cpp or faster-whisper
Transcribe audio locally with Whisper using openai-whisper, whisper.cpp or faster-whisper, export SRT or VTT subtitles, and match the tool to your hardware.
6 min read
- ComparisonRAG, search & data
pgvector, Qdrant, Chroma or Milvus: how to choose a vector database
Choose a vector database for RAG or search by comparing pgvector, Qdrant, Chroma and Milvus on deployment, filtering, index types and licence.
7 min read
- TutorialRAG, search & data
Build a RAG pipeline with Postgres and pgvector, step by step
Set up RAG with pgvector in your existing Postgres: a chunks table, an HNSW or IVFFlat index, the right distance operator and a filtered retrieval query.
6 min read
- GuideCoding & dev tools
How to connect Claude Code MCP servers and keep them safe
Add Claude Code MCP servers with the claude mcp command, pick local, project or user scope, share them in .mcp.json and vet each server first.
7 min read
- TutorialAgents & automation
How to build an MCP server in Python with the official SDK
Build an MCP server in Python with the official mcp SDK: add a tool and a resource, run it over stdio or HTTP, and test it in the MCP Inspector.
7 min read
- ComparisonModels & inference
llama.cpp vs Ollama vs LM Studio: which local model runner fits your setup
Compare llama.cpp vs Ollama vs LM Studio on APIs, GPU and OS support, model formats and licences, then pick a local model runner with a decision table.
7 min read
- TutorialModels & inference
How to run a local LLM with Ollama and call it from your code
Run a local LLM with Ollama on macOS, Windows or Linux, call it over its REST and OpenAI-compatible APIs, size models to your RAM and manage disk space.
7 min read