“My Grantmaking Strategy for Surviving Superintelligence” by A_donor

AI is humanity's first through fifth largest problem, but one stands head and shoulders above the rest. Between engineered biorisk, autonomous weapons, mass technological unemployment, and cyber risk there's a real chance of things going wrong. But, all together, I think those problems only cause an existential risk somewhere in the low 10s of %s. Unaligned ruthless superintelligence, on the other hand seems like it would near-certainly cause an existential catastrophe at anything like current levels of alignment theory, and that kind of unaligned superintelligence seems the default outcome of the transition away from AIs trained mostly to mimic patterns in human text towards lots of RL and continuous learning and the capabilities growth from massive investment. As a result, my grantmaking strategy is focused narrowly on interventions which seem like they might help delay or avert unaligned superintelligence, especially those that increase the odds of aligned superintelligence coming first. Classes of project I'm interested in Technical AI safety work that is sufficiently ambitious that it might apply even to strongly superintelligent systems. Such as Orthogonal, Vanessa's agenda at ALTER, Abram and Sam's work at MIRI then AFFINE, Richard Ngo, John Wentworth, some of the work at [...] --- Outline: (01:08) Classes of project I'm interested in (02:37) Classes of grantee I'm excited by (03:20) Things I am mostly not excited by (04:21) Classes of thing I don't consider particularly important (04:39) Context & me as a grantmaker The original text contained 16 footnotes which were omitted from this narration. --- First published: August 28th, 2026 Source: https://www.lesswrong.com/posts/whToGm8WFRqpHiFCB/my-grantmaking-strategy-for-surviving-superintelligence-1 --- Narrated by TYPE III AUDIO.