Model releases, model serving, local AI, inference engines and fine-tuning: what new models can do and how to run or adapt them.
Fine tune LLM LoRA adapters on one GPU: check that fine-tuning fits, prepare data, train with PEFT and TRL or Unsloth, and compare against the base model.
6 min read
Compare prompt caching API rules for OpenAI, Claude and Gemini, order prompts so the cache hits, avoid silent misses and measure savings from usage fields.
8 min read
Intel detailed the GPU D270 inference card: 480 GB LPDDR5X on partner cards, 160 GB on its own, 350 W PCIe. No price or ship date yet for buyers.
4 min read
Gemini 4 Argon is now on the paid Gemini API at $2 input and $10 output per 1M tokens until November 30, with a 1M-token window and long decode.
4 min read
NVIDIA released Nemotron 3.5 Super VL, a 120B open vision model, and Bolt 74B-A9B for 24GB local use. What developers can run now and what is missing.
4 min read
Get schema-valid JSON from OpenAI and Claude with LLM structured outputs, JSON schema and strict tool use, and know what each API guarantees.
6 min read
Install vLLM, start a vLLM OpenAI compatible server with vllm serve, tune GPU memory and context length, and point the OpenAI SDK at your own GPU.
6 min read
Compare llama.cpp vs Ollama vs LM Studio on APIs, GPU and OS support, model formats and licences, then pick a local model runner with a decision table.
7 min read
Run a local LLM with Ollama on macOS, Windows or Linux, call it over its REST and OpenAI-compatible APIs, size models to your RAM and manage disk space.
7 min read