Microsoft rebuilds Agent Lightning to train agents in their real harnesses

Microsoft Research Asia published a blog post on October 7, 2026 introducing Agent Lightning v1.0, an open-source reinforcement learning (RL) framework of about 3,500 lines of code. The post describes a training approach in which the harnesses agents use in deployment take part in RL training directly. The code is on GitHub under the MIT License, where the v1.0.0 release is dated 17 Aug and the latest tag is v1.0.2.

What Agent Lightning v1.0 changes

Most agent RL systems ask developers to rebuild their agent inside the training framework. The Microsoft Research announcement (opens in a new tab) names verl, AReaL and slime as earlier systems built on that assumption, where the trainer owns the loop between model and environment. The cost, according to the post, is that the trained agent differs from the agent that ships.

Agent Lightning v1.0 reverses the arrangement. An OpenAI-compatible LLM proxy sits between the agent and the model. Existing harnesses point their model endpoint at the proxy, and the framework records prompts, responses and log probabilities while the agent runs.

The post gives mini-SWE-agent, OpenHands, OpenCode, Claude Code, Codex and OpenClaw as examples of real harnesses that manage their own context, tools and execution. Those are the kind of systems the proxy approach is meant to leave untouched.

The framework has three components:

• API Gateway: stores rollouts, models and events, and records the model traffic.
• Rollout Controller: starts and manages agent runs as local processes or Kubernetes jobs.
• Customized Trainer: is built on verl and turns rollouts into training samples through a sample adapter.

The Agent Lightning README (opens in a new tab) states that the project was completely refactored in v1.0. Pre-1.0 code lives on a separate v0.x branch.

Kubernetes jobs and GPU scheduling in Agent Lightning

Agent Lightning v1.0 runs agents as standard Kubernetes jobs on self-managed clusters, cloud Kubernetes or local infrastructure. The post contrasts this with paid sandbox services and names Modal Sandbox and E2B. For a team that already operates Kubernetes, rollouts stay on hardware it controls.

The post also describes Collocated Async RL, a scheduling method where rollouts and model updates share the same GPUs. Microsoft reports about a 2x end-to-end speedup over synchronous RL, using fewer GPUs than conventional asynchronous RL.

Four training problems get specific attention in the write-up: retokenization and sample merging, advantage calculation, loss normalization, and fitting variable agent workloads onto fixed training resources.

The coding agent result

The main experiment combines SWE-smith data, mini-SWE-agent and Qwen3.5-9B. With about 6,000 training samples, Pass@1 on SWE-bench Verified rose from 41.8% to 56.4%, a gain of 14.6 percentage points.

A design detail matters for anyone writing their own recipe. Rollout-level advantage combined with rollout-level normalization reached a higher validation reward (about 0.37) and steadier policy entropy than sample-level handling.

The v1.0.2 release notes (opens in a new tab) add MoE coding-agent training with Qwen3.5-35B-A3B and multimodal training, where image inputs reach VERL training batches. Those notes also report a pure-RL experiment in which SWE-bench Verified rose from 47.8% to 61.6%.

Who Agent Lightning v1.0 is for

The design suits teams that already run agent harnesses and want to improve the model behind them with RL. Because those harnesses only change their model endpoint, the agent that gets trained is the same code that runs in production.

Researchers who want to read and modify an RL control plane are the second audience. A codebase of about 3,500 lines is small enough to study end to end.

This is GPU work. The Calc-X example in the README needs one GPU, and the setup script installs a verl stack for CUDA.

How to try Agent Lightning v1.0

The README gives this setup sequence for a CUDA 13.0 machine:

cd <this-repo>
uv sync
bash scripts/setup_verl.sh 0.8.0 cu130

The README does not list a pip package. Its examples are Calc-X, GSM8K, ScienceWorld, Search-R1, LLM-in-Sandbox, Coding Agent and Coding Agent: MoE.

The v1.0.1 release added the Agent Lightning Skill, which guides coding agents in optimizing other agents. It installs with gh skill install for Claude Code, Codex or GitHub Copilot.

Caveats and what is not known yet

Several practical questions are open after reading the post, the README and the release notes.

• Migration: the v1.0.x release notes list no breaking changes or migration steps, even though v1.0 is a full refactor. Teams on v0.x code have no published upgrade path.
• Python version support is not stated in the README.
• The post describes v1.0 as rebuilt with more emphasis on being lightweight, on real harnesses and on reproducibility, but gives no measured comparison with the original Agent Lightning.
• Its coding result uses mini-SWE-agent. The post does not report a training result for any other agent harness it names.
• The v1.0.2 notes say the local runner fails fast on native Windows.

Microsoft also published a technical report on the design, linked from the post, with more detail than the blog.