Skip to content
Sign in
Artwork for In Production: The CIO Perspective
TechnologyBusiness

In Production: The CIO Perspective

Graham O'Sullivan

The CIO Perspective on enterprise technology, data and AI transformation. I'm Graham O'Sullivan, a CIO with over 25 years of regulated financial services technology experience and currently serving as CIO with a global remit. Analysing the second order effects of AI, data and business transformation on people and operating models, as well as keeping ahead of the regulatory, cyber security and consumer technology landscape.

inproductionpod.substack.com
Play
Open in
  • 5 episodes
  • fortnightly
  • Avg 6 min
  • English

Episodes 5

NewestOldestLongest
All5Unplayed5In progress0Played0
  • October 15 · 4 min

    How we’re building a two tier AI society.

     What a crazy world we're living in. On the one hand, we're reading about frontier AI and needing to make ourselves safe from the advancing cyber threat posed by the latest AI models, and on the other side, customers in droves are signing up, sharing their credit card details with the latest AI agent chatbots. We are building a two-tier AI society. Choose your AI agent like you'd choose your bank. On Saturday, Elon Musk told X users they could photograph their credit card, drop it into a chat with Grok and let it find the best deal and buy it. Underneath this post, a community note pointed out that Musk's own company privacy policy tells its users not to put personal information into prompts. The product had a safer way to pay, but the owner told the public to skip it. This is where consumer AI is heading, and it's a two-tier system. On one side, Anthropic's Claude, Google's Gemini and OpenAI's ChatGPT detail privacy notices, access to drives and folders asked for and declared allow this one rather than always allow. On the other hand, the agentic consumer tools have arrived. Grok, now part of SpaceX, is one of them. The other, Meta's Muse, launched in the US on eighth of September, which asks for your email, your calendar, your cards and your bank accounts. It passed six point six million downloads and topped the US app charts within a fortnight. We never had a two-tier search engine market. Search was search. Some engines tracked you more aggressively than others, but none of them held your bank account ready to execute at will. Search engines were read-only. Agents have the power to write and hence do harm. There is no global regulation for any of this, no settled view of what agentic AI for the mass consumer should even look like. So each company is having a go at defining it through its own lens and its own values. Financial services has spent decades building controls for exactly this kind of thing: PCI DSS, strong customer authentication, the FCA's consumer duty in the UK, operational resilience. No regulated firm would get "photograph the card and send it in" past a risk committee. Consumer agents can now reach the same money with none of those obligations attached Which is where it gets uncomfortable for banks and frankly, every other financial services provider. Anne Boden, founder of Starling, made the case in the FT yesterday (at time of recording). She argues the next phase of banking will bypass apps entirely with a capable agent stepping into the gap between banks and their customers, and that the sector is looking the wrong way, treating AI mainly as a tool for cutting costs. Banks have spent years adding layer after layer of protection, insisting on a human presence so that every payment comes from the customer and not a fraudster or an automated attacker. What Bowden says customers need is an agent that knows their preferences, acts within the permissions they set, and completes the purchase. Get this wrong, and the relationship will follow the agent. Apollo's Torsten Slok has already warned that agents like Muse could sweep cash out of low-paying current accounts at scale. JP Morgan have recently announced banning such activity. Until regulation catches up, the banks will remain behind in the AI revolution. In the meantime, for you and me, the consumer, it's choose your agent at your own risk. I'm Graham O'Sullivan. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit inproductionpod.substack.com

  • September 22 · 6 min

    An AI Force is a good idea. The mandate is wrong.

    The US administration announced an AI Force on Saturday, with an AI czar to run it, modelled on the Space Force. The announcement was clear about the mandate: “We will not in any way hinder or stifle the Growth of this incredible Industry. Rather, we will cherish it, help it, and watch over it, as it grows!” That is a mandate to cheerlead, but I think it is the wrong one. I’m in favour of an AI Force, but I would point it in close to the opposite direction. Think about the analogy of the modern corporation. We decided a long time ago that a corporation could hold rights approaching those of a person, so that it can own property, sign contracts, sue and be sued, and outlive everyone who founded it. That was a useful invention but a potentially dangerous one, so we spent the following century building the guardrails around it. Company law, consumer protection, competition rules, disclosure regimes, personal liability for directors. Nobody seriously argues that any of it stifled the growth of the limited company. It is the reason the limited company is tolerable. We are now creating a second category of non-human actor with agency, and the proposal on the table is to cherish it. On development itself there is very little for government to fix. Private capital is funding this at a scale no state programme will improve on, and the frontier labs are not short of money, compute or people. What is absent is the other half of the arrangement, an assurance framework that keeps people safe whatever gets built. I’m a fan of Yudkowsky and Soares’ If Anyone Builds It, Everyone Dies, and I know that book is contested, but you don’t have to accept its conclusion to take the underlying risk seriously, and I would put it the way I’d put anything into a board risk paper. AGI is a real possibility, nobody can currently put a defensible bound on the probability that it goes badly, and the severity if it does is unrecoverable. Low confidence in likelihood, combined with catastrophic and irreversible impact, is exactly when you build the controls in advance, while the data is still poor. So what would I ask an AI Force to do? Not licensing for model training, which is close to unenforceable and mostly theatre. The work is deciding what responsible development actually obliges a developer to do, and then getting that into law in enough jurisdictions for it to bite. The Council of Europe Framework Convention on AI is a start and nowhere near sufficient. Then enforcement, including against the countries that decline to sign up. There will be no global court for this. People tend to reach for the International Criminal Court as the model, and it does not fit, because the ICC can only act on the territory or nationals of its member states, and only where a national system has failed to act first. The precedent that does work comes from financial crime. FATF has no treaty, no court and no enforcement power of its own. It writes standards, assesses countries against them, and publishes a grey list and a blacklist. Landing on either raises your cost of capital and complicates correspondent banking, and countries rewrite their laws to get off it. Attach that same mechanic to compute access, chip supply and model export and you have real leverage. The third area is response, and it is where I would put the money first. Assume a rogue AI agent is already out there. What is the national capability to find it and stop it? The standard answer is that this is only software and you can pull the plug. That comes from people who have never had to run an enterprise technology estate. The plug is not in one place. Models have already moved out of the data centre and into the operational technology behind power, water, transport and payments. I’ve spent the past few years writing operational resilience policy against DORA, and what becomes clear is how few organisations can enumerate their critical dependencies at all, let alone sever one deliberately and keep running. So what would stopping one actually involve? First you have to find it, and that is closer to tracing a piece of malware than to matching a fingerprint. You look at how the thing behaves, the patterns in what it produces and the mistakes it makes, and then at where it is running, because a model of any size has to run on somebody’s computers and that leaves accounts, traffic and a bill. Once it has been copied and retrained a few times those traces blur, so what you identify is a family rather than an individual. We have nothing like this at national scale today. Then you need an order of things to try, from the cheap to the drastic. Cut off its logins and its access to the systems it is using. Shut it down where it is running, which is the nearest thing to a real off switch, because nothing of this size runs without a great deal of computing power sitting in a small number of buildings, and those buildings have owners. Cut it off from the network. Cut off its money, using the same machinery we already run for sanctions. Turn off the power to the data centre. And run the physical infrastructure by hand. That last bit deserves a moment, because it is where the work is. We have spent thirty years engineering out that kind of capability in the name of efficiency. When Ukraine’s grid was attacked in December 2015, the attackers sabotaged the remote controls so that nothing could be switched back on from a screen. Power came back because people drove out to the substations and worked the switches by hand. The question worth asking is whether the infrastructure you rely on could still do that. At the far end sits the electromagnetic pulse, and it earns its place there because of what it costs. It targets nothing, and takes out the hospital, the water pumps and the phone network inside the same radius. Reaching for it means everything above it has already failed. The uncomfortable part is that a model which has already been copied and passed around sits outside all of this. You can take away its computing power, its logins, its money and its reach. You cannot take it back. Which is the argument for doing the first two properly, while the option is still there. None of this hinders anyone’s growth. It is what you build so that the growth is survivable, and it takes years, which is why the moment to start is while the whole idea still looks like an overreaction. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit inproductionpod.substack.com

  • September 4 · 5 min

    I only wanted to park the car

    Having been caught up in the recent data breach that impacted the customers of a number of UK airports, I got to thinking: if organisations keep losing our data, why should we trust them with so much of our personal details? This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit inproductionpod.substack.com

  • S1 · E2
    August 20 · 7 min

    The Great Technology Divide

    If we want to make enterprises more secure, we need to make staying safe online for consumers much simpler. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit inproductionpod.substack.com

  • S1 · E1
    August 8 · 7 min

    The Five Technology Shifts

    The first edition of Notes from a CIO, here in podcast form. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit inproductionpod.substack.com

Showing 1–5 of 5 episodes