開新對話又要重講一次？Google 讓模型睡一覺記住，成績單卻要連條件一起看

Google Research 部落格介紹的睡眠研究，讓模型暫停處理新輸入，把短期記憶搬進新擴充的長期參數。類別增量學習的提升裡，睡眠只佔兩到四成，10M tokens 接近滿分的成績則來自微調過的小模型。 ⭐ 文章深度讀：Google 讓大型語言模型睡覺！拆開來其實是一場打烊關帳 → https://heymaibao.com/google-language-models-sleep-closing-books-5f04f3/ ⚡ 章節重點 模型記不住的老毛病 00:00 打烊關帳式的睡眠 01:33 模型自己出題的夢 03:55 睡過之後的成績單 05:16 權重裡的新頁與難題 09:03 帶得走的讀法與節奏 10:15 📝 懶人包 ∙ 論文把災難性遺忘重新定義成容量不足的問題，解法是逐步長參數。 ∙ 拿掉作夢後，知識吸收單一段落成績掉到 35.7。 ∙ BABILong 長上下文比較裡，所有小模型都依官方流程微調過。 ∙ 我的觀點：模型的睡眠拆開來是打烊關帳，比作夢更接近它實際做的事。 📚 參考資料 The sleep framework: A new way for LLMs to retain knowledge in continual learning → https://research.google/blog/the-sleep-framework-a-new-way-for-llms-to-retain-knowledge-in-continual-learning/ Catastrophic interference → https://en.wikipedia.org/wiki/Catastrophic_interference Language Models Need Sleep: → https://arxiv.org/pdf/2606.03979 Q&A: UW researchers answer common questions about language models like ChatGPT → https://www.washington.edu/news/2024/01/09/qa-uw-researchers-answer-common-questions-about-language-models-like-chatgpt/ What Are LLM Parameters? → https://www.ibm.com/think/topics/llm-parameters chrF: character n-gram F-score for automatic MT evaluation → https://aclanthology.org/W15-3049.pdf BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack → https://proceedings.neurips.cc/paperfiles/paper/2024/file/c0d62e70dbc659cc9bd44cbcf1cb652f-Paper-DatasetsandBenchmarksTrack.pdf What is Fine-Tuning? → https://hai.stanford.edu/ai-definitions/what-is-fine-tuning