Sign inOpen the app
The Deeper Thinking Podcast

The Permission Machine - The Deeper Thinking Podcast

July 25 · 28 min · Episode 325 · 19.7 MB
0:00-28:31

Streams straight from the publisher. podnod never proxies or re-hosts episode audio.

Open in podnod

The Permission Machine: Artificial Intelligence and the Disappearance of Responsibility

The Deeper Thinking Podcast is digitally narrated.

For those drawn to the politics of automation, the disappearance of responsibility, and the fragile conditions of human judgment.

The fantasy is not intelligence without limits. It is power without encounter.

#ArtificialIntelligence #Automation #AlgorithmicGovernance #CoryDoctorow #JamesCScott #HannahArendt #InstitutionalPower

Key Ideas

  • Artificial intelligence can conceal human choices behind technical outputs
  • Automation often relocates labour, uncertainty, and responsibility rather than removing them
  • A human presence inside a system does not necessarily create human control
  • Institutions can become cognitively rich while growing morally and perceptually impoverished
  • Contestability must exist before automated systems become indispensable

Thinkers and Concepts

What happens when an institution no longer has to admit that a person made the decision? A loan is declined. A welfare payment is suspended. A worker is ranked as unproductive. A patient is classified as high risk. Human beings designed the categories, selected the evidence, established the threshold, and determined the consequences. Yet when the decision reaches the person who must live with it, those choices have disappeared. The system has spoken.

This episode examines artificial intelligence not simply as a technical capability, but as an institutional arrangement. We explore the point at which a tool that extends human agency becomes a system that places human beings inside the remaining gaps of an automated process.

Through Frederick Winslow Taylor’s transfer of knowledge from workers to management, James C. Scott’s account of administrative legibility, and Hannah Arendt’s understanding of judgment, the episode traces a longer history of institutions trying to govern complicated lives from a distance.

The machine does not need to be conscious to dominate an encounter. It only needs the organisation to treat disagreement with the system as less credible than the system itself.

We examine automation debt, the loss of people, memory, skill, and redundancy before a system has demonstrated that it can carry what has been transferred to it. Immediate savings remain visible. The deeper costs emerge later, during exceptions, crises, and change. An organisation can adopt a powerful tool while quietly spending its own memory.

The deeper struggle is not over whether AI is useful. Its usefulness is real. The struggle concerns the arrangement into which that usefulness is placed. Will AI enlarge human judgment, or make judgment ceremonial? Will it reduce repetitive labour, or convert every saved minute into a higher quota? Will it help institutions explain themselves, or allow them to avoid the cost of being answerable?

Extractable Insights

  • The system’s apparent judgment contains human choices that have been made difficult to locate.
  • Automation can move uncertainty downward while allowing the institution to appear more certain.
  • The machine seems autonomous because its dependencies have been hidden inside other people’s lives.
  • The person closest to a system’s failure is often recoded as the failure.
  • Removing discretion does not remove values. It relocates them.
  • A fluent explanation is not the same as a traceable reason.
  • When institutions remove experienced people, they lose ways of knowing.

Reflections

An organisation can become cognitively rich and morally stupid. It can know more about a population while becoming less capable of seeing a person.

Other reflections surfaced along the way:

  • Friction is not always waste. Sometimes friction is information arriving from reality.
  • A system that always answers may prevent anyone from noticing that the question was malformed.
  • A culture cannot be preserved by generating culture-shaped objects.
  • An institution cannot preserve judgment by generating judgment-shaped sentences.
  • The most dangerous system may be the one whose ordinary decisions become too smooth to question.

Why Listen

  • Understand how AI can make institutional power less visible
  • Recognise the hidden labour and uncertainty produced by automation
  • Explore why human oversight often becomes symbolic rather than meaningful
  • See how automation debt weakens organisational memory and resilience
  • Consider the conditions required for AI to remain answerable to the world it affects

Listen On

Support This Work

If this episode stayed with you, you can support it here: Buy Me a Coffee, drop me and email and gain access to our new discord channel. 

Further Reading

  • Doctorow, Cory. The Internet Con: How to Seize the Means of Computation. Verso, 2023.
  • Scott, James C. Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed. Yale University Press, 1998.
  • Taylor, Frederick Winslow. The Principles of Scientific Management. Harper & Brothers, 1911.
  • Arendt, Hannah. Lectures on Kant’s Political Philosophy. University of Chicago Press, 1982.
  • Royal Commission into the Robodebt Scheme. Report of the Royal Commission into the Robodebt Scheme. Commonwealth of Australia, 2023.
  • Wallis, Nick. The Great Post Office Scandal. Bath Publishing, 2021.

A system capable of producing an answer is not necessarily capable of knowing when the answer has damaged the world. For that, it still needs someone who can answer back.