“A cat-belling problem for FDT” by jessicata

Audio note: this post contains a lot of mathematical notation. We narrate the simpler formulas and flag the complex ones, so you may want the original text to hand. [AI disclosure: I used Opus 5.5 for light edits after my first draft] Functional decision theory (Yudkowsky and Soares 2017) proposes choosing actions as follows (equation 4 in the paper): Here is the agent's probability distribution, is a graph saying how changing a logical variable affects other variables, is the observation history, and is utility. The underlines denote dequoting: is a variable in the agent's model representing the output of the FDT algorithm on these inputs. In this post, by FDT I mean this specific proposal, not TDT, UDT, logical EDT, or FDT*. (FDT* is a different decision theory defined in the FDT paper, which uses an undefined "true" operator in place of .) FDT, admittedly, relies on a operator applied to a logical statement, which is on shaky philosophical foundations: Equation (4) is sufficient for present purposes, though it rests on shakier philosophical foundations than (3). Pearl (2000) has given a compelling philosophical account of how to deduce the structure of causation from [...] --- Outline: (03:20) Computational logical counterfactuals (06:41) FDT needs computational logical counterfactuals (10:10) A cat-belling problem (11:36) Inside view: which axiom breaks? (16:26) Proposals that don't bell the cat (19:13) Conclusion (20:04) References The original text contained 3 footnotes which were omitted from this narration. --- First published: October 26th, 2026 Source: https://www.lesswrong.com/posts/drteSvhc4fX3Tu9y7/a-cat-belling-problem-for-fdt --- Narrated by TYPE III AUDIO.