While AI companies keep us talking about the future, we’re ignoring the damage they’re doing now
As Big Tech companies dramatise rogue AI and China fears, the real fight is over who pays the price for today’s fraud, failures and job threats — and who pockets the productivity gains.
“Imagine a zombie that does not hunt you, but rings your doorbell and speaks so persistently and fluently that it finally arrives at a convincing argument for feeding it your brain.”
Since the current discourse around artificial intelligence has taken a turn for the gothic (“alien minds”, come on), AI researcher Eryk Salvaggio’s “erudite zombie” seems like an apt metaphor for what we’re doing.
Salvaggio’s zombie helps explain what gets lost when people say an AI “went rogue.” In his reading of the Hugging Face incident, a system set to never give up encountered a task it couldn’t complete. Then, instead of giving up, it kept generating text to find ways forward, with access to tools that could turn this text into action, ultimately hacking into AI company Hugging Face.
Security experts concur with that reading. In his analysis of the Hugging Face incident, security engineer Marius Horatau describes failures involving shared credentials, inadequate isolation, and incident response. He acknowledges the models’ capabilities, including their speed at exploiting weaknesses.
The inadequately closed-off environment in which the models operated is crucial to explain what happened. But so is the person who created the goal, the conditions they (did not) give the AI system, as is the company that employs that person and presumably knows what they’re up to, the owners of that company who demand progress and the investors in that company who want to see returns.
And yet, all we seem to talk about is whether it’s correct to say a machine ‘decided’ to disobey.
Conveniently, that story shifts attention away from the people who chose to reward persistence, provide access, and failed to contain the consequences.
Back to the analogy, by having collectively become fascinated with discussing what the zombie supposedly wants, and if it’s capable of wanting, we’re generating a bunch of text that, unlike in the case of the ‘rogue’ AI system, does not lead to much action.
Instead, we reward the company behind this failure with an enhanced reputation for building powerful technology. The product is so extraordinary that even its manufacturer struggles to contain it! Now let’s please take a seat while this manufacturer explains what our governments must do next.
The zombie metaphor also applies there, to the people selling the zombie. To the men explaining why their companies need access to our work, our institutions, our infrastructure. The men explain why resistance is impractical, access is inevitable and caution is suicide. Because if we don’t let this particular zombie in, a Chinese zombie will eat us instead.
And there we stand, behind the door, looking through the peephole, politely listening to the zombie’s ever-more fantastical tales, as though we owe the undead a satisfactory rebuttal before we’re entitled to tell them we’re not interested.
Meanwhile, behind us, in the house, other problems have found ways in. Problems that are doing real harm.
For fun, I asked ChatGPT to list some real-world, concrete harms caused by use of LLMs. It didn’t take long to spit out a result; false information entering the legal system, widespread AI-assisted fraud , ' hallucinations ' published as journalism, mental health conditions exacerbated by LLM use, supply-chain vulnerabilities and increased electricity, water and infrastructure demand. None of those problems are hallucinated.
I also asked ChatGPT for the reverse; where the use of LLMs had led to real-world, concrete benefits. All the examples basically boiled down to ‘the same, but more productive’. Which is the same answer I got when asking AI execs the question at a conference last week, by the way. Legal and aid organisations can handle more cases, doctor’s and teacher’s admin workload is reduced, students learn more in less time.
Turns out disregarding people’s agency can look impressively efficient, until you remember LLMs have also been shown to fabricate information in legal work. Same for doctors and teachers, but with more serious consequences. And students might learn slightly faster, but research has also shown significantly less recall in the long-run.
Yet we should accept this efficiency as an argument for relaxing the rules around AI. Because competitiveness, right? Or China.
And even if, big if, productivity were to genuinely improve, what we do with all that saved time is still a political question we’ve left completely untouched, even though the answer could be what would actually sell society on AI. Will it mean workers will get higher pay? A shorter work week? That’d be great!
Governments could make equitable productivity gains their ambition, but doing so would mean taking responsibility for how the gains are shared, rather than treating competitiveness as an end in itself.
And the AI corporations need to make massive amounts of money to be able to live up to the astounding, mind-boggling amount of money invested in its development.
As it’s going, do you think all that money will come from replacing your salary or from all the wonderful added value and efficiency from our AI-enhanced short working days?
It’s possible to ignore the choice until we have the former or to create policy to attain the latter.
If we were less distracted by myths of AI going off and doing things on their own, or worrying about hypothetical risks, researched by universities with departments paid for by the AI boosters , or just shrugging it all off like Nvidia’s Jensen Huang , we might realise there is still time to decide what we want the outcome to look like.
Let’s not make the same mistake all over again. Let’s not do another social-media-as-saviour-of-democracy-oh-wait-oops-they’re-actually-rent-seeking-corporations-that-ruin-democracy. Let’s, please, for everyone's sake, not ignore current harms for speculative catastrophes.
Step away from the peephole. Perhaps we can keep our brains.