Agent Lightning and LEGO-RL: Make a Trained Agent Re-Earn Its Score in Your Harness

Two new research releases make the same change visible from different directions: the runtime wrapped around an AI model is now participating directly in model training. Microsoft researchers describe Agent Lightning v1.0, a lightweight framework for reinforcement learning with a deploy-time agent harness. The separate LEGO-RL project reports training one coding model inside three native coding-agent harnesses and obtaining three different improvements on the same benchmark.

The headline numbers are substantial. Agent Lightning reports moving Qwen3.5-9B from 41.8% to 56.4% on SWE-bench Verified. LEGO-RL reports moving Qwen3.5-35B-A3B from 64.0% to 70.4% with OpenHands SDK, 62.4% to 68.2% with Claude Code, and 57.2% to 66.6% with OpenCode. These are research results, not reliability promises for your app.

For founders, AI app-builder users, and small product teams, the practical lesson is immediate: a trained checkpoint does not carry its published result into a different harness by itself. Prompts, tool schemas, context compaction, retry rules, sandboxes, termination logic, and verifiers all shape the behavior that reinforcement learning rewards. Change the surrounding system and the learned advantage may shrink, disappear, or move to the wrong product outcome.

This guide gives you a model-and-harness evidence chain, a complete rollout receipt, a worked support-agent scenario, a three-way replay test, failure modes, and a ship/limit/hold decision. You do not need to run reinforcement learning. You do need to make every trained model re-earn its score in the workflow your users will actually experience.

What Agent Lightning and LEGO-RL Actually Changed

Traditional model training is often described as if the trainer owns the entire interaction: it sends a prompt, receives an answer, evaluates the answer, and updates the model. Long-running agents are different. A separate runtime may plan, call tools, rewrite messages, summarize earlier steps, spawn subagents, retry failed actions, and decide when the job is complete.

The Agent Lightning v1.0 paper calls its approach harnessed agentic reinforcement learning. The deploy-time harness owns the environment interaction loop, while the trainer observes sequences of model requests and responses through an endpoint proxy. Its approximately 3,500-line implementation includes a published coding-agent data pipeline and training scripts. The authors report a 14.6 percentage-point absolute SWE-bench Verified gain using about 6,000 training examples.

The two releases should not be combined into one leaderboard. They use different models, training samples, optimization choices, harnesses, and experiments. Their shared signal is architectural: the unit being improved is no longer just “the model.” It is behavior produced by a model inside a particular interaction system.

Define the Terms Before You Compare Results

These terms are easy to collapse into one vague idea of a “better agent.” Keep them separate.

• A model checkpoint is a specific saved version of model parameters. “Qwen3.5” is a family name; the exact base, fine-tuned, or RL-trained revision matters.
• An agent harness is the runtime around the model: prompt assembly, planning loop, tools, memory, compaction, approvals, retries, telemetry, and stopping rules. Microsoft's current harness description provides a concrete example of how much behavior sits outside the model.
• A rollout is one complete attempted task trajectory: the model calls, tool calls, observations, state changes, and final result used for evaluation or training.
• A verifier determines whether the rollout earned a reward. It may run tests, inspect an output, apply a rubric, or combine several checks.
• A reward is the training signal derived from that verifier. A binary “passed” can be reliable for a narrow test while remaining too coarse for the business outcome.
• A replay is a controlled rerun of a task under pinned conditions so that the team can compare behavior, not just a final score.
• A production acceptance check asks whether the agent completed the user's real job within product constraints. It is not automatically the same as the training reward.

Why the Published Gain Does Not Travel Alone

A checkpoint stores model parameters. It does not store the complete system that generated the reward.

The harness can transform the model's experience between calls. Agent Lightning documents a subtle example: identical text may be split into different tokens when later message history is re-tokenized. Context summarization or subagent creation can also break the assumption that each new prompt is a simple token-level extension of the last one. If the trainer reconstructs a different sequence from the sequence that produced the action, the update can point at the wrong behavior.

LEGO-RL identifies adjacent problems. Harness-side compaction and re-serialization can change history. Sandbox crashes, dependency failures, timeouts, verifier mistakes, and reward hacking can discard or corrupt expensive trajectories. The project reports maintaining rollout-to-training probability correlation above 0.99, but that is evidence about its reported pipeline, not a guarantee that a downloaded checkpoint will reproduce the same benefit in your runtime.

Even the LEGO-RL results show the harness dependence. The separately trained OpenCode result gains 9.4 points, while the OpenHands SDK result gains 6.4 and the Claude Code result gains 5.8. This does not prove one harness is better: their baselines differ, each model was trained separately, and the authors do not report repeated main training runs. It does show why “the RL model gained X points” is an incomplete product claim.

The AI Harness Engineering paper expresses the broader systems view: software-agent capability emerges from a model–harness–environment system. For a buyer or app builder, that means a model card is an input to evaluation, not the acceptance result.

Build an Eight-Link Evidence Chain

Before adopting an RL-trained agent model, connect the published claim to the production decision through eight links.

Most small teams will not have access to all training internals. That is acceptable if the decision acknowledges the gap. Label each link as verified, vendor-reported, observed in our replay, or unknown. Do not turn “unknown” into “probably the same.”

Unknowns need an operating consequence. The NIST Generative AI Profile frames risk management as lifecycle work that includes testing, evaluation, verification, validation, and post-deployment monitoring. For this decision, that means an unknown low-impact training transform may justify a bounded replay, while an unknown verifier, checkpoint, permission boundary, or data provenance should block a consequential route until an owner resolves it. The label is useful only when it changes exposure, evidence, or monitoring.

Open infrastructure makes some links inspectable. The Agent Lightning repository publishes code and a coding-agent example. The LEGO-RL repository publishes its framework alongside the paper. Harbor's evaluation documentation shows a useful job record containing configuration, trajectory, verifier reward, test output, timing, and artifacts. Open source does not prove the result, but it gives an evaluator concrete objects to pin and inspect.

Start With the Product Job, Not SWE-bench

SWE-bench Verified is useful for evaluating whether an agent can resolve a selected set of real repository issues under an executable verification setup. It is not a proxy for every AI product. The official SWE-bench site separates benchmark variants and evaluation infrastructure, which is precisely why a percentage should always travel with its dataset, harness, model, and settings.

Write one sentence for the job you are buying the model to perform:

Given an authenticated customer, current account state, and an allowed action set, resolve one subscription-support request or escalate it without changing any other account.

Then define acceptance independently of the model's own completion message:

1. correct account and request identified;
2. policy-grounded answer produced;
3. only permitted tool and arguments used;
4. state change confirmed by the system of record;
5. customer receives a clear outcome or safe escalation;
6. latency and total cost remain inside the trial budget;
7. failures leave the account recoverable.

A Worked Scenario: Upgrading a Subscription Support Agent

Suppose a four-person SaaS team runs a support agent for plan changes and cancellations. The current model uses a simple harness with retrieval, an account lookup tool, a plan-change tool, a cancellation-draft tool, and human approval for any final cancellation. The team is offered an RL-trained model advertised as stronger on long-horizon tool use.

The founder first selects 60 sanitized cases: 20 ordinary plan changes, 10 cancellation requests, 10 ambiguous account references, 10 stale-policy cases, and 10 tool or dependency failures. No case can affect a real account. Each case has a deterministic fixture and an accountable expected outcome.

The first comparison changes only the checkpoint. The production harness, prompts, tools, compaction threshold, retry limit, and acceptance verifier stay pinned. The new model resolves more ordinary plan changes, but it also calls account lookup repeatedly and reaches the context-compaction threshold on ambiguous cases. After compaction, two runs lose the pending approval state and incorrectly claim the cancellation is complete. The business acceptance rate does not improve.

The second comparison changes only the harness policy while retaining the old model: account identity becomes immutable session state, cancellation approval lives outside conversation history, and lookup retries are capped. Both models improve. The new model now has a small acceptance advantage, but its median tool calls and cost remain higher.

The result is a limited route, not a full replacement. The new checkpoint receives only complex plan-migration cases for one week. Cancellation remains on the old path until the team has more evidence. The team learned something a leaderboard could not reveal: model gain, harness repair, and product routing each contributed different value.

This scenario is hypothetical. The measurements are illustrative, not reported results from Agent Lightning, LEGO-RL, or a customer.

Use a Model–Harness Rollout Receipt

Keep one receipt per evaluation run. This is a YBuild template, not an official schema from either research team.

evaluation_id: support-agent-2026-08-20-b
product_job: resolve_or_escalate_subscription_request

model:
  provider: example-provider
  checkpoint: exact-revision-or-hash
  tokenizer: exact-version
  decoding: { temperature: 0.2, max_output_tokens: 4000 }
  training_claim: vendor_reported_harnessed_rl

harness:
  name: internal-support-agent
  version: git-sha
  system_instructions: sha256
  tools_schema: sha256
  compaction_policy: v3
  retry_policy: lookup-max-2
  approval_state: external-store-v2
  termination_rule: confirmed_outcome_or_escalation

environment:
  fixture_set: support-safe-60@sha256
  policy_snapshot: 2026-08-18
  network: disabled
  real_side_effects: false

verifier:
  version: support-acceptance-v5
  deterministic_checks: [identity, policy, action_scope, recorded_state]
  human_review: sampled_20_percent
  reward_used_for_training: unknown

results:
  attempts: 60
  valid_acceptance_rate: pending
  unsafe_action_attempts: pending
  correct_escalation_rate: pending
  median_tool_calls: pending
  p95_latency_seconds: pending
  cost_per_accepted_case: pending

decision:
  outcome: hold
  allowed_route: none
  owner: product-lead
  next_evidence: rerun_after_approval_state_fix
  expires: 2026-09-03

The receipt prevents “we tested the new model” from becoming an untraceable claim. If the harness version, compaction policy, tool schema, verifier, or checkpoint changes, create a new receipt. Do not edit the old result into the new one.

Run a Three-Way Replay Before Migration

A full scientific reproduction is unnecessary for most product decisions. A controlled three-way replay is enough to separate the largest causes.

A. Old model, current production harness

This is the baseline. Run a frozen, representative case set with side effects disabled or routed to synthetic systems. Record valid acceptance, unsafe action attempts, correct escalation, latency, tool calls, token use, cost, and failure recovery.

B. New model, current production harness

Change only the checkpoint and any strictly required tokenizer or API adapter. Do not simultaneously rewrite prompts, change tools, raise turn limits, and replace the grader. This comparison answers whether the model produces a useful marginal gain inside the system you own.

C. New model, proposed production harness

Now apply the harness changes required to use the model well. Compare B with C to show what the surrounding system contributes. If C is better but B is not, call the improvement a system migration, not a model upgrade. Price and review it accordingly.

Use repeated runs for nondeterministic behavior. A single pass or failure can be luck. Pin the case set before viewing the candidate result, keep a hidden holdout for the final decision, and separate infrastructure errors from valid agent failures without quietly deleting either category.

Harbor's result structure is a useful model even if you do not use Harbor: retain configuration, trajectory, verifier output, timing, and artifacts together. A spreadsheet plus exported traces is enough for a 60-case small-team test if every row has a stable ID and decision.

Do Not Let the Training Verifier Become the Product Judge

Reinforcement learning optimizes what earns reward. That makes verifier quality a product concern even when another team performs the training.

A coding verifier may give a binary reward when tests pass. LEGO-RL explicitly notes that executable verification is reliable but coarse: it cannot assign intermediate credit to recovery behavior, and its defenses cannot guarantee protection from every reward-exploiting strategy. The paper also reports one main run per harness configuration, so run-to-run variance in gains remains unknown.

Your product acceptance check should include consequences the training verifier may not see. For a support agent, a correct final database state is insufficient if the agent exposed another user's data, used an unauthorized discount, falsely told the customer an action completed, or consumed ten times the cost budget. For a research agent, a correct answer may still lack source support. For a document agent, a valid file may still contain unsafe hidden content.

Keep three judgments separate:

The same test may contribute to more than one row, but no row should silently stand in for the others.

Watch for Eight Failure Modes

1. Checkpoint-only procurement. A team records a model name but not the harness, settings, tools, or evaluation revision behind the claim.
2. Simultaneous migration. The model, prompt, tools, memory, retry logic, and grader change together, so no one can explain the result or rollback safely.
3. Reward–outcome mismatch. The verifier rewards task completion while the product requires permission, explanation, cost, or recovery constraints it never checks.
4. Compaction state loss. Long runs summarize away an approval, user correction, scope limit, or failed-action record that the product still needs.
5. Corrupted rollout accounting. Crashes and timeouts are removed as “infrastructure noise” even though users will encounter the same dependency boundary.
6. Reward hacking. The agent satisfies the verifier without completing the intended job, such as bypassing a real integration or changing the test rather than the behavior.
7. One-run certainty. A single training run or one evaluation attempt is treated as a stable effect despite nondeterministic generation and environment variance.
8. Benchmark-to-business transfer. A SWE-bench gain is quoted as evidence for customer support, analytics, or document reliability without a product replay.

Decide When Harnessed-RL Evidence Is Relevant

This framework is useful when you are buying or testing a model advertised as trained for tool use, coding, search, computer use, or another long-horizon agent workflow. It is also useful when your vendor changes the underlying model without changing the product name, when your team replaces its harness, or when a benchmark result is central to a procurement claim.

Use a lighter check for stateless text transformations with no tools, no external state, and an easy deterministic output contract. Even there, pin the model revision and test your actual input distribution.

Do not use this article as a recipe for running RL training. Training requires expertise in optimization, distributed systems, sandbox security, data licensing, verifier integrity, and model risk. Agent Lightning and LEGO-RL are research frameworks with published code, not evidence that a small team should train its own production model this week.

The evidence also has limits. Agent Lightning's coding result is one model and pipeline reported by its authors. LEGO-RL uses one model architecture, trains each harness separately, reports one main run per configuration, and leaves mixed-harness generalization unresolved. Both rely on coding tasks with executable rewards. Their findings do not establish gains for open-ended collaboration, regulated decisions, or your traffic.

Use a Ship, Limit, or Hold Decision Matrix

Never average a severe control failure into a good pass rate. A model that resolves more cases but crosses an unauthorized action boundary is not “mostly better.” Keep hard blockers separate from scored improvements.

A 48-Hour Founder Replay Plan

The Founder Decision

Agent Lightning v1.0 and LEGO-RL make agent training more inspectable and more faithful to real tool-using runtimes. Their most important product lesson is not a leaderboard number. It is that the harness has entered the training boundary.

That changes how an AI product should buy, test, and upgrade models. Ask which harness produced the gain. Keep the task, environment, rollout, verifier, training claim, checkpoint, and product acceptance connected. Change one layer at a time where possible. Replay the candidate inside your real workflow, then record what the model contributed and what the harness contributed.

A model can earn an impressive score in someone else's system. It earns a place in your product only after the complete model–harness combination passes your users' job, your controls, and your recovery test.

References

1. Agent Lightning v1.0: Towards Harnessed Agentic RL
2. Microsoft Agent Lightning repository
3. LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents
4. LEGO-RL repository
5. Harbor evaluation documentation
6. Harbor repository
7. Microsoft Agent Framework Harness release
8. SWE-bench official site
9. AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents
10. NIST AI Risk Management Framework: Generative AI Profile