Researchers at Unit 42, the research team at Palo Alto Networks, are seeing a rapidly growing market for so-called dark LLMs. These customized AI models remove security restrictions and make it easier to develop and execute advanced cyberattacks.
Dark LLMs are Large Language Models that have been deliberately modified or built to no longer have security brakes. According to Unit 42, this represents a new chapter in the dual-use dilemma surrounding technology. The same models that help security teams with detection and response can be used by criminals to accelerate and scale up attacks.
Revenue model
Unit 42 sees a growing ecosystem of customized models such as WormGPT and FraudGPT. These are sold via Telegram channels and dark web forums. The models generate, among other things, phishing material, malware scaffolding and automated scripts for attacks.
The distribution follows a commercial model. Access is via subscriptions with promises of “uncensored” output and support, similar to SaaS services. In addition, open source variants are emerging, such as KawaiiGPT, which can be installed locally without extensive programming knowledge. This increases usability for less technically savvy criminals.
read also
Trend Micro: “Cybercrime Fully Automated by 2026”
According to Unit 42, the models investigated produce flawless phishing texts and usable code fragments for malware. This lowers the entry threshold and supports large-scale, automated campaigns.
Underground market
The researchers conclude that dark LLMs have evolved from an experiment to a fully-fledged underground market. LLM automation is therefore becoming central to the way attackers work.
For security teams, this means that they must not only look at classic indicators, but also at AI-generated artifacts. Think of very convincing phishing emails or rapidly changing malware variants.