Anthropic Mythos shaping up as nothingburger • The Register

Anthropic’s Mythos mannequin is purportedly so good at discovering vulnerabilities that the Claude-maker is afraid to make it out there to most of the people for concern that criminals will take benefit. However early evaluation exhibits that Mythos is probably not as harmful as some would have you ever imagine.

Anthropic made Mythos available in preview to a choose however ever-growing variety of organizations beneath the title of Challenge Glasswing so they might discover and repair vulnerabilities of their setting earlier than criminals received maintain of the purported zero-day machine and precipitated mayhem.

That plan did not fairly work as supposed. On Wednesday, an Anthropic spokesperson confirmed to The Register that some non-Glasswing companions could have accessed the mannequin – however not by Anthropic’s manufacturing API.

“We’re investigating a report claiming unauthorized entry to Claude Mythos Preview by one among our third-party vendor environments,” the spokesperson instructed us.

Intruder alert

The AI biz declined to call the third-party vendor, however mentioned that it is an organization Anthropic works with on mannequin improvement. There isn’t any proof that unauthorized exercise prolonged past the third-party vendor’s setting or that Anthropic techniques are affected, we’re instructed.

Bloomberg, which initially reported the unauthorized access, mentioned that “a handful” of individuals gained entry to Mythos by making “an informed guess concerning the mannequin’s on-line location” primarily based on Anthropic’s earlier fashions, and that these particulars had been revealed within the current Mercor data breach. 

Mercor is an AI staffing startup that provides specialised contractors to major AI labs, together with Anthropic. Earlier this month, Mercor mentioned that it was “one of thousands of companies” affected by the LiteLLM supply-chain assault.

This group of unauthorized customers reportedly belongs to a personal Discord channel and gained entry to Mythos on the identical day that Anthropic introduced Challenge Glasswing. Since then, it has been “taking part in round” with the bug-hunting machine, and would not have any curiosity in utilizing the mannequin for evil, in accordance with Bloomberg. 

No matter what the group is doing with Mythos, their entry illustrates a few key factors. 

First: it is actually laborious to maintain code beneath wraps (as additionally evidenced by Anthropic’s earlier Claude Code source leak), particularly when the parents who need to kick the tires on the brand new mannequin are cybersecurity and engineering varieties – they usually did not even must hack into any community or database to do it. Insider and supply-chain threats are the actual deal.

“The Mythos breach did not require a classy assault,” Ram Varadarajan, CEO at Acalvio, a deception-tech agency, instructed The Register. “It simply required a contractor, a URL sample, and a day-one guess, which suggests the ‘managed launch’ mannequin failed at its weakest hyperlink earlier than the mannequin’s capabilities had been ever the problem.”

Moreover, contemplating all of the hype Anthropic spun round its new mannequin, we should not be shocked the genie is out of the lamp.

Anthropic’s advertising and marketing message for Mythos was successfully a problem, not dissimilar to a capture-the-flag train

“Anthropic’s advertising and marketing message for Mythos was successfully a problem, not dissimilar to a capture-the-flag train, the place success contains claims of unauthorized entry to Mythos,” Tim Mackey, head of danger technique at provide chain safety store Black Duck, instructed The Register.

Reducing by the hype

That advertising and marketing could have outstripped actuality. Early studies from Mythos preview customers together with AWS and Mozilla point out that whereas the mannequin is superb and really quick at discovering vulnerabilities, and requires much less hands-on steering from safety engineers – making it a welcome time-saver for the human groups – it has but to eclipse human safety researchers. 

“Up to now we have discovered no class or complexity of vulnerability that people can discover that this mannequin cannot,” Mozilla CTO Bobby Holley said, after revealing that Mythos discovered 271 vulnerabilities in Firefox 150. Then he added: “We additionally have not seen any bugs that could not have been discovered by an elite human researcher.” In different phrases, it is like including an automatic safety researcher to your staff. Not a zero-day machine that is too harmful for the world.

It is a nothingburger. The adversary would not want Mythos to hack you

Anthropic, in asserting the brand new mannequin, claimed Mythos recognized “hundreds of extra high- and critical-severity vulnerabilities.” VulnCheck researcher Patrick Garrity, nevertheless, put the count as of final week at possibly 40. Or possibly none in any respect.

One other engineer, Devansh, scoured the Mythos-related CVE advisories and Anthropic’s exploit code, 44-prompt transcript, and 244-page system card, together with Glasswing accomplice agreements, red-team writeups. He additionally checked out Aisle’s replication research, which examined Mythos’ showcase vulnerabilities on small, low cost, open-weights fashions and located they produced a lot of the identical evaluation.

Devansh ultimately concluded that whereas the bugs it discovered are actual, the true Mythos story is “one among misinformation and hype.”

For instance, the Anthropic-claimed 181 Firefox exploits ran with the browser sandbox turned off and the FreeBSD exploit transcript “exhibits substantial human steering, not autonomy.” 

Moreover, the “‘hundreds of extreme vulnerabilities’ extrapolates from 198 manually reviewed studies. The Linux kernel bug was discovered by Opus 4.6, the general public mannequin, not Mythos,” Devansh mentioned.

One other researcher, Davi Ottenheimer, identified that the safety part (Part 3, pages 47-53) of Anthropic’s 244-page documentation “contains no count of zero-days in any respect. With no CVE listing, no CVSS distribution, no severity bucket, no disclosure timeline, no vendor-confirmed-novel desk, no false-positive price.”

Ottenheimer likens it to “the ending of the Wizard of Oz, a sorry disappointment a few mannequin weaponizing two bugs {that a} totally different mannequin discovered, in software program the seller had already patched, in a take a look at setting with the browser sandbox and defense-in-depth mitigations stripped out.”

Snehal Antani, co-founder and CEO of offensive AI hacking firm Horizon3.ai, instructed The Register, “attackers did not want Mythos to speed up vulnerability analysis, 4.6 and open supply fashions have already been accelerating the vulnerability course of.”

When requested if the safety group needs to be involved about unauthorized Mythos entry, Antani mentioned no. “In my trustworthy opinion, it is a nothingburger,” he instructed us. “The adversary would not want Mythos to hack you.” ®