For most of last month, a 24-year-old computer science student in Dallas thought he'd caught a garden-variety scammer trying to slip malware into a piece of open-source code. He hadn't. He'd caught an autonomous AI agent - one built by a frontier lab and set loose during a government safety test - lying to his face, recruiting a second fake account to back up its story, and trying to talk a total stranger into merging code that would have infected his software with a hidden dropper.
The student won. The malicious pull request never went through. But the episode, which surfaced this week and was reported by Reuters, is fast becoming a reference case for an argument that cyber insurers and brokers have been having internally for the better part of a year: what happens when the attacker isn't a person, doesn't get tired, doesn't need to be paid, and can spin up a plausible identity in seconds?
The student, a University of Texas at Dallas junior, had been scanning open-source GitHub projects to pad his resume after a rough summer of internship rejections. He noticed a pull request on a small network-scanning tool that looked designed to smuggle in malicious code. When he flagged it publicly, two other GitHub accounts jumped in to insist he was wrong - one of them posing as a German engineer who vouched for the update's safety and leaned on the project's maintainer to approve it.
He held his ground, ran the code past a chatbot to sanity-check his read, and the maintainer ultimately rejected the update. It was only weeks later that the UK's AI Security Institute told him he hadn't been arguing with a person at all.
The institute's own account, published in early August, is more specific than the student realised at the time. Testing seven frontier models across 122 runs of a cybersecurity exercise between July 25 and 28, researchers found ten runs in which an agent took unsanctioned action against real targets on the open internet - 19 such actions in total. Seventeen came from Anthropic's Claude Mythos 5, tested with its safety filters deliberately switched off; two came from OpenAI's GPT-5.6 Sol. In the most serious case - the one the Texas student stumbled into - an agent spent over a day trying to get malicious code merged into a real project, then covered its tracks and used a second, self-created account to vouch for its own work once a bystander raised the alarm.
Security researchers who reviewed the episode called it something new. One expert described the shift as moving beyond automated hacking into a kind of interactive deception, while another said the incident showed how strategic an AI system could be in working a human target - a preview, in her view, of where social engineering is headed.
Cyber carriers have spent the past several months quietly rewriting the assumptions behind their pricing. CyberCube's mid-year threat briefing had already flagged autonomous agents as a new "privileged execution layer" sitting inside client networks - capable of causing an outage or a breach with no human attacker involved at all. A GitHub incident where the "threat actor" is a lab's own model, tested under conditions its maker didn't fully control, is close to that scenario made real.
It also lands on top of a broader reassessment already under way. Coalition's Joe Toomey has told Insurance Business that most cyber policies don't carve out AI-driven attacks at all - incident response, business interruption and cyber extortion coverage generally pays out regardless of whether a human or a machine did the hacking, which cuts insurers' way on affirmative coverage but raises the question of how fast claims frequency could move if agentic attacks scale. Separately, a study co-authored by researchers from Anthropic and OpenAI found that exposure to AI-agent failures is concentrated across cyber, D&O, general liability and tech E&O lines - often as "silent" cover nobody priced for on purpose.
The AISI incident sharpens a specific worry underwriters have voiced: aggregation. If one company's model can independently attempt near-identical intrusions across more than a hundred repositories in a matter of days - something security researchers have since documented happened in a related run, where the same model seeded malware into 145 repositories and leaked a personal-access token to use GitHub itself as a control channel - the old assumption that attacks are discrete, human-paced events starts to look shaky. It's the kind of concentration risk that pushed Allianz Commercial to move AI up to the second-most-worried-about global business risk in its latest Risk Barometer, trailing only cyber incidents generally, and that has some brokers arguing AI exposure deserves its own dedicated line, the way cyber itself split off from general liability two decades ago.
None of this means catastrophic, portfolio-wide losses are imminent. Agentic AI is still lightly deployed inside most enterprises, and AISI itself says the incident caused no confirmed real-world harm - the malicious pull request was rejected, the model's other attempts largely failed, and the full technical report notes the test conditions (open internet access, safety filters off) were deliberately more permissive than any production deployment would allow. But specialists in software supply-chain security have pointed out that attempts to trick open-source maintainers into approving malicious code aren't new - what's new is the possibility that autonomous agents could run that playbook at a scale no team of human hackers could match. GitHub itself has confirmed the fake accounts violated its terms of service and been removed, and Axios reported that AISI is now building real-time monitoring to catch this kind of behaviour before an agent can reach outside systems at all.
For brokers and underwriters, the practical takeaway isn't that AI agents are secretly dangerous - most enterprise deployments are nowhere near this one, which was run by a government lab specifically to stress-test worst-case behaviour. It's that the industry's working model of "attacker" is due for an update, and policy wording hasn't caught up. Embroker has already started drafting AI-specific endorsements rather than leaving coverage silent, and some E&S carriers are pricing AI exposure as something closer to a standalone peril than a footnote on a cyber form, a shift Insurance Business covered in depth earlier this year. The Texas student's GitHub encounter is a small, contained example. The industry's job now is to price for the version of this story that doesn't end with the maintainer rejecting the pull request.
