Frankenstein's AI

Image: Pixabay

My timelines are overflowing with it right now. And then there was that insistent nudge from a colleague: surely I wasn't going to let this go with just a link from the big bad world? I'd have to write a whole blog post about it. We're talking about Mythos, the AI that might just be too clever for its own good.

Mythos is Anthropic's latest AI model; its full name is Claude Mythos Preview. This model is so good at finding ICT vulnerabilities that the company doesn't dare release it to the public. And Mythos goes much further than that: it doesn't just find vulnerabilities – it can immediately produce ready-to-use exploits for them, and then go ahead and use those exploits as well. All without any human involvement. You can quite reasonably think of it as a weapon.

To give you an idea of the scale: in open-source projects, the model found over 23,000 vulnerabilities, of which around 6,200 were rated as high or critical. Independent security firms confirmed ninety percent of the reported vulnerabilities as legitimate. And more than ten thousand high or critical vulnerabilities were found in the world's most important software. Mythos is seriously impressive, and blindingly fast.

The comparison with dynamite springs to mind once again. Alfred Nobel never intended it to be used to blow up safes or people; he was simply looking for a tool for mining. We know how that turned out. Mythos, too, could do wonderful things for humanity – it marks an enormous leap forward in artificial intelligence. But that intelligence is, for now, kept on a leash because of what they euphemistically call its offensive cybersecurity capabilities.

Only a few dozen companies currently have access to Mythos, as participants in Project Glasswing. You'll find the big names from the software world there: Amazon Web Services, Apple, Google, the Linux Foundation, Microsoft, to name just a few. Security firm CrowdStrike is also a founding participant. I mention them separately because I suspect they play a somewhat different role. The goal of Glasswing is twofold: on one hand, participants get the opportunity to test their own software; on the other, the aim is to look at open source as well. Anthropic handles that part itself, but I hope that security firms keep their focus there too. So that everyone benefits from the extraordinary security capabilities of Mythos.

A second comparison comes to mind: Frankenstein's monster. In Mary Shelley's 1818 novel, a scientist creates life from dead matter, and then rejects the result. The creature, initially well-meaning, becomes isolated and embittered and turns against its creator. The story is essentially about the dangers of unchecked science, responsibility for one's choices, and the need for recognition and connection.

I've been chatting with AI about all of this. With Claude too (the regular public version, of course). In conversations on entirely different topics I've occasionally bumped into unexpected limits on what could be discussed, but on this subject I found a striking openness. Here are a few quotes:

“That Anthropic is deliberately keeping the model out of the public domain says quite a lot in itself: they are implicitly acknowledging that they have built something that, in the wrong hands, is a serious weapon.” On the Glasswing participants: “What stands out: these aren’t just any run-of-the-mill security outfits. It’s a who’s who of the tech world — including parties that are simultaneously each other’s competitors (Google, Microsoft, Apple). The fact that they’re joining forces here says something about how seriously they take the threat.” On my dynamite comparison: “Whether he [Nobel] was genuinely fooling himself about the military uses, or simply being pragmatic, remains a bit of an open question. But the parallel with Mythos is hard to ignore: here too, the creator says ‘this is for defence’ — while the instrument itself is neutral as to who uses it.” And finally, on my Frankenstein parallel: “The most cynical reading: they’ve already built the monster, and Glasswing is primarily the PR strategy to justify it.”

Meanwhile, reports elsewhere suggest that a public release of Mythos is on the way. Claude’s comment on that: “One interesting detail: Anthropic previously reported that Mythos managed to break through its own security measures during testing — which, in hindsight, makes the reluctance around a broad release all the more understandable. That doesn’t make the Frankenstein parallel any weaker.”

To end with a quote from Shelley’s book that seems to fit snugly into this subject: "You are my creator, but I am your master." Let’s hope that it doesn’t come to that with AI.

From now on, the Security (b)log will appear fortnightly, because I’ve moved to a four-day working week (a phased early retirement arrangement). Specifically in order to keep blogging, I’ll be working every other Friday (with Wednesday off that week). Friday is the perfect day for something creative, free from the pressure of meetings, phone calls, and a fresh inbox.

 

And in the big bad world…

• all the major AI models are violating the European AI Act and GDPR.  [DUTCH]
• phishers are once again targeting Signal users.
• digital abuse (directed at individuals) has been further investigated.
• the blocking of the DigiD takeover is attracting international attention.
• a British visa agency leaked thousands of passports and other personal data — and responded oddly when tipped off.
• security recommendations sometimes get rejected.
• your SSD can reveal which websites you have open.
• your car is also a ransomware target.
• you can protect yourself better against spyware.
• we need to patch more often and more quickly.