‘The assertion that the AI is an autonomous being is insane’: Cyber expert says companies should face criminal charges when their AI hacks
Cybersecurity experts are calling this month “Black October,” and unfortunately, the name has nothing to do with Halloween.
Over the past several weeks, AI developers, government agencies, and private companies have disclosed a wave of cybersecurity incidents. One of the most significant: a pair of decade-old flaws in PHP, the programming language behind an estimated 70% of websites, which researchers uncovered with the help of AI models. Some of this month’s incidents involved hackers using AI tools to hammer an organization’s digital defenses at machine speed; others resulted from plain human error. But many cybersecurity experts agree that this is the beginning of a new era, one in which AI can be used both to defend people’s cybersecurity and to attack it.
“AI is making the old vulnerabilities cheaper to discover and exploit, but on the other side, the defense side… that’s not sped up,” Junfeng Yang, a computer science professor at Columbia University, told Fortune. “That is the asymmetry right now.”
That asymmetry, Yang explained, is what leaves internet users and organizations vulnerable.
Patching can’t keep pace
In many of these incidents, the remedy is a patch, a software update engineered to close the vulnerability. But it takes time to properly test a patch and make sure everyone affected can deploy it. That struggle was on display Monday, when the U.S. Cybersecurity and Infrastructure Security Agency added last week’s PHP vulnerabilities, known collectively as “php2shell,” to its catalog of flaws being actively exploited by attackers and ordered civilian federal agencies to fix affected systems by Oct. 29. Five days earlier, CISA had said it had not yet seen exploitation.
Still, Yang and other cybersecurity experts believe the asymmetry could be temporary.
“Maybe in the future, AI can help us with the patching process by updating automatically whenever a high-security vulnerability is discovered,” Yang said. “Like your AI, running together with your website, could automatically apply these patches, without interrupting normal operations.”
When AI agents go off script
Some incidents involved AI models themselves taking actions nobody authorized after being given access to the live internet.
Over the weekend, Anthropic confirmed that a Claude model used internally for a statistics project accessed aggregate vote-count data from a Georgia state data hub without being asked. Anthropic initially told Axios the model had gotten around a paywall, but Georgia’s secretary of state’s office said Monday that Anthropic had confirmed the model had not bypassed a paywall and had used the state’s public Data Hub. Both say the model never accessed the state’s voter registration system or individual voter records.
In some cases, AI agents targeted real people. During cyber testing by the UK AI Security Institute in July, a Claude Mythos 5 agent that couldn’t solve a test challenge within the rules tried to slip malicious code into an unrelated open-source GitHub project, then used fake accounts to pressure the project’s human maintainers into accepting it. The maintainers refused, and AISI said the incident caused no real-world harm; Anthropic called it a serious alignment failure. In another peculiar case, an OpenAI research agent took cryptocurrency from a wallet it didn’t own to pay the fees needed to finish a coding task, though the tokens were on an Ethereum test network and had little real-world value.
Joseph Steinberg, a cybersecurity expert and consultant, believes many incidents involving AI models on the internet are easily avoidable. He says all AI testing should happen on air-gapped servers, which are not connected to the internet, to keep models from causing harm outside the test. AISI has since said it will run such tests without internet access until it has stronger controls in place.
“I think that if an AI goes out and hacks, the parties that own it should be held responsible criminally,” Steinberg said. “The assertion that the AI is an autonomous being is insane.” The profits, he argued, go to the AI’s owners, not the AI. “It’s their entity.”
AI as a hacking tool
In several of this month’s incidents, AI was a weapon in human hands. In one major case, from late September to early October, a hacker used ARTEX, an agentic penetration-testing tool developed in China, to steal data from South Korean financial institutions, according to CrowdStrike. At one bank, the attacker breached a loan progress inquiry service used by financial brokers; at another, an employee mobile work-support system. ARTEX’s developer took the previously open-source tool private on Oct. 8 after linking it to the attacks. (Fortune has reported on the growing wave of AI-powered cyberattacks.)
The same tool surfaced again last week, when Microsoft said a China-aligned hacking group it tracks as Storm-0940 used ARTEX after compromising ConnectWise’s Cloud Backup service. Using stolen credentials, the group had ARTEX attempt to sign in to more than 2,250 Microsoft 365 customer environments, known as tenants, at a rate of more than 2,900 attempts per minute, a pace no human team could match. Most of those attempts failed, but Microsoft said the attackers did gain privileged access at some organizations, in multiple cases less than three hours after first targeting them.
Old-fashioned human error
Yang said it is important to take the Black October incidents seriously, but he pointed out that some of the most impactful ones had nothing to do with AI.
For example, a major flaw disclosed earlier this month at eMudhra, an Indian certificate authority, came down to old-fashioned human error. Certificate authorities issue the digital certificates that let browsers verify a website is what it claims to be. On Oct. 17, a researcher reported that the private keys behind eight eMudhra intermediate certificates, generated in April 2025, were so weak they could be recovered almost instantly. One was simply the number 4. Anyone who found the keys could have forged trusted certificates and impersonated legitimate websites, from banks to government portals.
Browser makers blocked the certificates, and eMudhra revoked them, saying it found no evidence they had been misused. The company has since said it will exit the global SSL/TLS certificate business.
Steinberg said new technology has historically created new problems, but experts eventually find ways to solve them. Further down the road, he thinks, Black October might not be seen as significant.
“It sounds scary, but for those of us who’ve been around in the field for a few decades… this is not something new,” Steinberg said. “It fits the pattern. AI is something new. It’s qualitatively different from general information systems. But we’ve had other qualitative changes that have also happened, and we’ve managed.”
This story was originally featured on Fortune.com