Skip to content
Artwork for Analyzing Trends
Analyzing Trends · October 29 · 8 min

Foresight Hallucinations

This fall, two randomized trials of AI tutors pointed in opposite directions. Maryland undergraduates who were offered a tutor in their course finished with lower grades, while secondary students in England who revised with a tutor scored higher. Both results are real, and both come with details a headline drops: the Maryland tutor gave direct explanations by default, and only about one in seven students offered it ever used it. This episode follows a finding like that as it loses its conditions on the way into a scenario, and gives the result a name: a foresight hallucination. Joseph Voros's futures cone and Nele Fischer and Sascha Dannenberg's work on plausibility show where a team's assumptions about the present go unrecorded. The 2024 Wharton and Harvard tutoring studies show how two well-built experiments became opposite headlines. Fabricated citations get caught when someone looks them up, but a foresight hallucination cites real studies, so the only defense is keeping each finding attached to the conditions that produced it.

0:00-8:20

transcript

No transcript — this publisher did not publish one.

show notes

This fall, two randomized trials of AI tutors pointed in opposite directions. Maryland undergraduates who were offered a tutor in their course finished with lower grades, while secondary students in England who revised with a tutor scored higher. Both results are real, and both come with details a headline drops: the Maryland tutor gave direct explanations by default, and only about one in seven students offered it ever used it. This episode follows a finding like that as it loses its conditions on the way into a scenario, and gives the result a name: a foresight hallucination. Joseph Voros's futures cone and Nele Fischer and Sascha Dannenberg's work on plausibility show where a team's assumptions about the present go unrecorded. The 2024 Wharton and Harvard tutoring studies show how two well-built experiments became opposite headlines. Fabricated citations get caught when someone looks them up, but a foresight hallucination cites real studies, so the only defense is keeping each finding attached to the conditions that produced it.