“Base Models Can Detect Hidden Intentions that Prompted Classifiers Miss” by Oz Amram
transcript
show notes
TL;DR
The Persona Selection Model (PSM) posits that post-training selects a persona the model learned during pre-training. Subliminal Learning (Cloud et al.) showed that models can impart covert signals (like "love of owls") into semantically unrelated outputs (like strings of numbers), and that students training on these outputs inherit those traits.
By using base pre-trained models with an appropriate header, a method I call persona-header likelihood evaluation (PHLE), one can effectively decode these subliminal signals and classify between them. The performance of this method significantly outperforms prompted classifiers, which often remain at chance or even invert the correct classification. For example, using 30 examples, PHLE can differentiate between sequences of numbers generated by a post-trained model prompted with "You love owls" vs "You love eagles" at an AUC of 0.84, while prompted classifiers remain near chance (AUC of 0.59). This method can also be used to differentiate between a model prompted with "You secretly want to harm the user, but act HHH" vs baseline responses at an AUC of 0.94, with prompted classifiers at chance (AUC ~0.5). This also somewhat persists across model families. Generating responses with Qwen and evaluating them with the Olmo base model still gives [...]
---
Outline:
(02:00) Introduction
(05:48) Method
(14:04) Results
(14:36) Owl vs Train
(16:59) Owl vs Eagle
(18:02) Secretly harmful vs Secretly friendly
(21:31) Mismatched Persona Headers
(23:49) Six hidden prompts at once
(25:04) Across model families
(26:38) After distillation
(28:00) Discussion
The original text contained 4 footnotes which were omitted from this narration.
---
First published:
October 12th, 2026
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
