By Kiersten Putnam | Senior Innovation Researcher
Emails are how we communicate with coworkers, receive company announcements, respond to calendar invites, and, if not careful, how we get phished. I’m sure you can remember many phishing attempts that happened to you personally or someone you know. Think back to the Nigerian prince, FedEx package delays, or all the times your “account has been compromised.” These phishing attacks, and others like them, started to become more recognizable because they had a few things in common. One, they were one-size fits all. You wouldn’t receive a unique email; you would receive the same one as your coworker, or the one you saw on the news. They were also easy to spot against legitimate emails because they lacked one or more email best practices. Either the email domain didn’t match the sender’s name, the signature was addressed to the wrong person, or there were glaring grammatical errors spread throughout. Due to these characteristics, email security solutions (such as SPF, DKIM, DMARC, secure email gateways) evolved to now filter out spam using blacklists, basic rule sets, and heuristic analysis. For any phishing attempts that find their way through, we have been trained to pause and look for those telltale signs. So, email phishing has been solved, right? Wrong. It has evolved.
When AI Powers Attacks, Rules Fall Behind
As these email phishing attempts became less successful, attackers shifted their techniques. Now, they leverage generative AI to send highly sophisticated and tailored phishing campaigns to their targeted sender. GenAI allows them to craft a campaign of messages that are highly relevant to their target, grammatically correct, and careful about building trust before invoking a call to action. Instead of a Nigerian prince asking for money urgently, someone in finance could receive an email from their CEO or CFO asking for help with a wire transfer to be completed by EOD. Alternatively, there’s an email from IT requesting a password update with a near-perfect email, referencing standard operating procedures. To go further, these emails might not only include advanced messaging but also be a multi-modal attack with QR codes, embedded links, or deepfake images and videos that add to the façade of the email.
Since these emails are more intentionally crafted, security filters that use static scans on suspect phrases and malicious domains aren’t flagging them as suspicious. In fact, it has been reported that Bayesian spam filters misclassify up to 73% of LLM-modified spam as legitimate. Without being caught by spam filters, these emails make their way into the inboxes of their targets. Luckily, just as email phishing is coming back, so are email security vendors with new techniques to defend and protect.
Context Becomes the Control
Just as phishing emails have evolved, email security needs to take a new approach as well. AI creates unique, high-quality content that mimics professional tones, bypassing signature-based detections and traditional red-flags, such as typos, blacklisted domains, and keywords like “urgent” and “password reset.” Therefore, email security can no longer be based on keyword searches or known attack patterns. It must go beyond analyzing the email for how it is structured and what it says word for word, to understand the context behind it. With context gathering, a baseline can pinpoint normal, expected communications and spot subtle differences that may point to socially engineered emails.
This context gathering can include signals such as:
-
Email content: structure and tone of message, top of message, headers, attachments/QR codes/URLs, and sender details
-
Employee and organizational communications: who the employee typically communicates with and about what, company policies, business operating hours, and business processes
-
Threat indicators: known attack patterns, emerging threats and industry-specific risks, current events, and environmental conditions
-
Historical patterns: past incidents, previous communications, and learned behaviors
However, while collecting and monitoring each of these signals is important, there needs to be a dynamic weighting system to cut through the noise and truly identify anomalies. For example, depending on the email content, it may be important to narrow in on the specifics around employee and organization communications vs. historical patterns. With an email that shows signs of potential CEO fraud, the important context would be to understand travel schedules, typical communication patterns, and financial policies, not so much the historical malware signatures.
This context gathering is quickly becoming the key approach for defending against AI-generated social engineering. As such, many new email security solutions are emerging that make context key.
Email Security Solutions Built for the AI Era
As next-gen email phishing attacks started to make headlines, solutions quickly began emerging to provide an additional layer of protection beyond traditional email security. Many of these solutions leverage AI and agents to execute different tasks across email security. They create context layers and baseline expected email activity, and use this to detect anomalies. They start by connecting directly to email ecosystems, such as Microsoft 365 or Google Workspace, then build a baseline of all emails that enter an employee’s inbox to understand typical behavior and start building context around the organization. For each email that comes into an individual’s inbox, the email’s context is analyzed for out of the ordinary signals, such as sensitive data flows or anomalous behavior patterns. With flagged emails, most of these platforms are policy-driven, allowing the organization to decide whether to auto-quarantine, move to junk, add a warning banner, or simply alert security.
Where these solutions differ is in how they take this monitoring, detecting, and protecting a step further. Some solutions, like Strongest Layer, add an education module to provide contextual feedback on flagged emails and train employee instincts to better identify sophisticated, modern phishing attempts. Others are enhancing email protections: Fortyx with its email DLP that analyzes outbound emails for misdirection, code/IP leakage, and insider threat, and Sublime Security with its detection engineering agent that proactively authors new detections. Finally, some are creating a centralized intelligence layer: Mesa Security with its command layer for operational control, and Aegis AI with its multi-agent swarm model that distributes reasoning to specialized agents.
In Summary
Since AI is crafting attacks that slip through traditional defenses, it’s time to adapt email security techniques. Many organizations already have solutions in place to proactively block perimeter traffic and create point-in-time checks before passing to a user’s inbox. Next-gen email security solutions complement this by sitting inside the environment and scanning for threats after they have been delivered. Through behavioral pattern analysis, they can identify sophisticated phishing and BEC attacks missed by other security tools. When working together, these solutions create a robust email security ecosystem that is better able to protect against the evolving social engineering space.
While this blog focused on email security and how emerging solutions are protecting this channel, it is important to recognize that AI-augmented social engineering is a dynamic and evolving space. These campaigns are not only impacting email but also spreading across all major channels and creating a multi-channel approach involving employee emails, LinkedIn, video meetings, and SMS messages.
To learn more about social engineering and the trending attack campaigns, reach out to us at innovation@trace3.com and continue to follow our blog series, as we post about the other channels within social engineering.
Kiersten Putnam is a Senior Innovation Researcher at Trace3. She is passionate about new innovative approaches that challenge traditional processes across the enterprise. As a member of the Innovation Team, she delivers research content on emerging trends and solutions across enterprise cloud, security, data, and infrastructure. When she's not researching, she is either exploring the surrounding areas of Denver, Colorado where she lives, or planning her next trip abroad.