AI Chatbots are Being Used to Create Perfect Phishing Emails

Identifying phishing attempts used to be fairly straightforward for end users. The messages often contain grammatical errors and spelling mistakes that had been inadvertently included in the messages.  Phishing campaigns are often conducted by individuals who do not speak English as a first language, so errors will inevitably be made and it is those errors that make it fairly easy for people to spot a phishing attempt.

Those errors may soon become a thing of the past thanks to artificial intelligence tools such as ChatGPT. ChatGPT and other large language model AI tools can be used to create perfect English (or other languages) and therefore convincing text for use in phishing and social engineering attacks. Evidence is growing that these tools are being adopted by malicious actors to create phishing content that is indistinguishable from the content that a human could create, and in many cases, it is even better.

Europol has recently issued an alert about the malicious use of these AI tools for phishing and warned that the problem is likely to get worse. It is not just a case of being able to draft a grammatically correct email devoid of spelling mistakes, but that these AI chatbots can write emails in whatever style the threat actor wants, including in an authoritative tone as one would expect from an official government communication.

The biggest threat is likely to be highly targeted emails – spear phishing. Spear phishing has a far higher success rate than standard phishing attempts, as emails are carefully crafted to attack a very small number of individuals. That requires considerable research to ensure that the scam is convincing and the email will likely be opened and the request followed. The ability of AI tools to create spear phishing emails should not be underestimated. The messages these tools can generate can be exactly what a threat actor needs and the process can be largely automated, which means a higher success rate and more attacks.

These tools are significantly lowering the barrier of entry for conducting phishing attacks, and while there are restrictions in place to prevent the malicious use of these AI tools, they are being bypassed. You can ask ChatGPT to write a phishing email but it won’t, but you can create the phishing content if you are not so direct. The cybersecurity firm Darktrace says it has found evidence of phishing emails increasingly being written by chatbots, and not only does that make it easier for cybercriminals to create convincing messages, they also allow much longer messages to be created than was previously possible. The company reports that phishing email volume is down, which it suggests could be due to threat actors being able to write better, more linguistically complex emails and opt for quality over quantity. Chatbots have also been used to write malicious scripts that could be used as ransomware or for information-stealing malware. Researchers have created examples of both using the engine that powers ChatGPT.  Europol paints “a grim outlook” as phishing emails will become a lot harder for people to identify. Tools have been developed that are capable of detecting AI-written content but they are not reliable and as AI chatbots become more advanced, these tools will likely become even more unreliable.

So while the outlook may not be too good, the advances in AI technology mean businesses will need to up their game and that means ensuring that they provide security awareness training to the workforce and keep them abreast of the changing tactics used by threat actors. Training should also emphasize that employees should not implicitly trust any communication and should assume that it might be a scam. Training should cover security best practices and businesses will need to improve their technical defenses and implement further solutions to identify and block the various stages of a phishing attempt, such as advanced spam filtering (SpamTitan includes an AI-based component for detecting phishing attempts), a web filter, multi-factor authentication and to ensure that patches are applied promptly and all software is kept up to date.