Major US banks are scrambling to restrict third-party AI agents accessing customer accounts. According to Axios, some agents have appeared to take actions beyond users’ instructions, while banks struggle to distinguish authorised activity from potentially rogue behaviour.
AI agents are growing more capable. The risks include cyberattacks, mass manipulation, job disruption and, in the worst case, losing control of systems. This is no longer just a debate about tomorrow’s superintelligence. It is about what happens when technology can act faster than our ability to understand, monitor and control it.
Geoffrey Hinton warns that AI could escape human control. Yuval Noah Harari argues that AI agents are fundamentally different from tools because they can make decisions and act independently. In September, Anthropic researcher Jacob Coxon resigned, warning that racing towards self-improving AI risks humanity’s future.
Earlier, OpenAI’s Jan Leike criticised the company for prioritising “shiny products” over safety. Researchers from OpenAI and Google DeepMind also signed an open letter warning that financial incentives could undermine effective oversight.
As Ezra Klein explores in Why Are We Sprinting Off the AI Cliff?, the uncomfortable question is why we keep accelerating when control remains uncertain.
The incentives are clear: Geopolitical rivalry. Billions in investment. Commercial dominance. And the fear that slowing down means allowing competitors to pull ahead.
But this reminded me of Dan Ariely’s TED Talk on 'Are we in control of our decisions' from 2 decades ago quoting the 1995 Donald Redelmeier – Eldar Shafir study. Physicians considered a patient with a painful hip who was contemplating a hip replacement. When the patient is on his way to the surgery, the doctors realized they hadn't tried one drug, so the question was should they let him pursue with the surgery or have him come back and try out this one drug before he eventually proceeded with the surgery, and most doctors suggested he hold on to the surgery and try out this one drug first.
However in a different scenario, doctors were presented with a case where there were two drugs that could potentially help but hadn't been tried, and the question was would the doctors now choose to call him back and get the patient to try out either of these two drugs or let the patient proceed into surgery? And most doctors said, they would let the patient move into the surgery, without starting either medication.
When doctors faced two alternative drugs instead of one, decision complexity changed their recommendations. Rather than choose between the drugs, more opted for referral without trying either.
The study illustrated how added complexity can change decisions. Often in 'favor of the easier decision' instead of 'the right decision'.
Could something similar be happening with this AI race?
When the choices are complex, the stakes enormous and every competitor is accelerating, does continuing the race become the default — even when slowing down collectively might be safer?
Or is the problem our difficulty imagining distant consequences, while the rewards of acting now are immediate and tangible?
Perhaps this isn't just a technology problem, but a behavioural-science one.
I wonder if this AI race a giant behavioural-science problem? Are leaders defaulting to acceleration because slowing down is the harder choice? Or are distant, uncertain consequences simply too difficult to weigh against immediate rewards?