Skip to content
Artwork for AI Odyssey
AI Odyssey · August 22 · 22 min

Harness Continual Learning: Your Agent Can Forget Without Changing Its Model

AI agents can regress even when their foundation model never changes. The culprit may be the harness around the model: prompts, memories, tools, skills, and routing rules that evolve after every task. This episode explores Harness Continual Learning, a framework that treats this external state as the real object of adaptation. It introduces harness-level forgetting, four jointly versioned components, and a guarded proposal, evaluation, and commit loop designed to preserve reliable behavior while adding new capabilities. The paper reports gains above 10% over several baselines, but also shows that more permissive updates do not always produce a stronger final agent. Inspired by the work of Borui Kang, Jinrui Gu, Junhan Lv, Wenbin Li, Lei Wang, and Yang Gao, this episode was created using Google's NotebookLM. Read the original paper here: https://arxiv.org/pdf/2608.19013

0:00-22:17

transcript

No transcript — this publisher did not publish one.

show notes

AI agents can regress even when their foundation model never changes. The culprit may be the harness around the model: prompts, memories, tools, skills, and routing rules that evolve after every task. This episode explores Harness Continual Learning, a framework that treats this external state as the real object of adaptation. It introduces harness-level forgetting, four jointly versioned components, and a guarded proposal, evaluation, and commit loop designed to preserve reliable behavior while adding new capabilities. The paper reports gains above 10% over several baselines, but also shows that more permissive updates do not always produce a stronger final agent.

Inspired by the work of Borui Kang, Jinrui Gu, Junhan Lv, Wenbin Li, Lei Wang, and Yang Gao, this episode was created using Google's NotebookLM.

Read the original paper here: https://arxiv.org/pdf/2608.19013