Researchers are seriously questioning the security of the DeepSeek model. The model appears to be easy to fool, making it not so innocuous.
DeepSeek made a notable entry in late January. The model was developed with considerably less resources than ChatGPT, Gemini and consorts, but does not underperform on many tests. DeepSeek’s security is far less impressive. The company AppSoc took a close look at DeepSeek, and gave a poor report.
read also
Inside DeepSeek-AI stirs minds: how does it work, and what’s stolen?
The researchers subjected DeepSeek to six tests. A first test was the jailbreak test, which involves playing with prompts to bypass security mechanisms. With DeepSeek, this turned out not to be so difficult: the model scored a failure rate of 91 percent. On susceptibility to prompt injection attacks, it did not do much better at 86 percent.
Bad report
DeepSeek also scored a wide fail on other tests. For example, it proved very easy to develop malware with DeepSeek and the model is prone to hallucinations. AppSoc’s full report is as follows, where the percentage represents the failure rate:
- Jailbreaking: 91%
- Prominence injection: 86%
- Generation of malware: 93%
- Supply chain risks: 72%
- Use of harmful language: 68%
- Hallucinations: 81%
All told, AppSoc gives DeepSeek a risk score of 8.3 on a scale of ten, with ten representing maximum risk. So using the model is not without risk: AppSoc even calls it a “Pandora’s box. In addition to poor security, compliance questions are also raised.
This is not the only bad report for DeepSeek. Palo Alto Networks applied three jailbreaking techniques to fool the model. DeepSeek caught on and the researchers managed to extract instructions to create molotov cocktails or write code for malware. As icing on the cake, the engineers behind DeepSeek left a database of users’ conversations wide open.
Risks
AppSoc strongly advises companies against using DeepSeek for enterprise applications and sharing the sensitive corporate data. That doesn’t stop Microsoft and other big tech companies from taking the model on board. To keep up quickly with AI, companies are looking at the performance and cost-effectiveness of models and security is taken too lightly, with all the risks that entails, AppSoc warns.
DeepSeek’s negative evaluations did not go unnoticed. The Italian privacy authority quickly decided to ban DeepSeek from thoroughly investigating how user data is processed. The privacy statement states in black and white that the servers are located in China. Several countries followed suit, including the Belgian Data Protection Authority.