Mozilla has been working with the preview of Claude Mythos for several weeks and is sharing some insights with the world. The essence: Mythos delivers on its promises, provided it has the right framework.
Mozilla is opening up. Normally, the Firefox creator doesn’t share details about internally discovered bugs until at least several months after a patch is launched, but now the company is making an exception. After all, Mozilla wants to share the impact and added value of AI in the search for bugs.
271 vulnerabilities
In April, Mozilla already announced it had tracked down and patched 271 vulnerabilities, thanks to Anthropic’s Claude Mythos preview. As a refresher: in early April, Anthropic revealed the existence of that Large Language Model (LLM), specifically developed to discover software vulnerabilities. Mythos immediately caused a stir: the AI could detect vulnerabilities in no time in software that even the best researchers with the best tools hadn’t seen for years.
Anthropic knew what it had on its hands: Mythos wasn’t launched broadly, but access was provided very specifically to a select group of software companies. They could then get to work with the LLM to patch bugs before criminals inevitably got their hands on similar technology in the near future. After all, besides Anthropic, other AI companies are also working on AI for bugs, and no one is under the illusion that the technology will remain exclusively in the hands of the ‘good’ side.
Suspicion
As the creator of Firefox, Mozilla is one of the parties deploying Mythos. The goal is to make the Firefox browser more secure and stay ahead of any AI-driven criminals. However, the revelation of the 271 patched bugs was met with immediate skepticism, partly because Mozilla is not in the habit of broadly sharing internally discovered bugs or assigning them a CVE.
Furthermore, since the launch of the first version of ChatGPT, AI companies haven’t exactly built a reputation for nuance. Although the impact of AI on various sectors, including software development, is concrete and undeniable today, the valuations of the companies involved are primarily based on future promises, and immense investments stand in contrast to an equally immense lack of profit. A healthy suspicion regarding Mythos’s role at Mozilla is therefore understandable.
More details
That is why the company is now coming forward with more details. According to the company, the intention is not to praise Mythos itself, but to illustrate the added value of that LLM and similar systems.
First and foremost, Mozilla indicates what the situation was a few months ago. AI could already generate bug reports, but they often took the form of unwanted slop. The reports seemed plausible but were often hallucinated. Project maintainers had to respond to suspected problems and spent a lot of time on them, even if the suspected bug later turned out to be a hallucination.
Not just the LLM
In just a few months, that dynamic has completely changed. Mozilla points to two causes for this:
- AI models have suddenly become much more powerful (with Claude Mythos as an example),
- Mozilla has drastically overhauled its techniques for deploying LLMs.
According to Mozilla, the big gain lies in the combination of both, and not just the launch of a new LLM.
Model in a harness
Specifically, Mozilla has introduced what it calls a harness. This is essentially a very concrete definition of what the LLM is allowed to do, in what way, and with which tools. Mozilla built an entire system to integrate an AI model into its existing bug research pipeline.
In other words, the company didn’t just unleash AI on the code. The LLM received specific instructions to search for defined problems within a useful context. Engineers decided where the AI system should search based on their knowledge and automated signals.
The result of this scoping was that bug reports suddenly became very concrete: the LLM could find clear errors and demonstrate what was wrong in a way that was useful to programmers.
Concrete bug reports
Due to the scoped integration of the AI, false positives and hallucinations disappeared. Bug reports suddenly became very good, with concrete and reproducible results. As a result, the number of discovered bugs increased exponentially.

Mozilla points out that developing the harness was a complex task, requiring many iterations to achieve the desired end result. Mozilla built a feedback loop with human engineers to develop the approach.
Of course, the new LLM also plays a major role. Within the boundaries of the harness, Claude Mythos could work with complex logic across different processes to find previously unknown problems. However, the harness is LLM-agnostic. Mozilla can easily replace Claude Mythos with another LLM when a new or more performant version becomes available.
Mozilla advises software developers to start building a harness today that works within their own context, using any modern model. That harness will then be ready for the integration of new models as they become available.
