The basic functionality provided by Microsoft Defender and its email protections was insufficient for our needs. While it effectively handled common spam and phishing attempts, we required a solution that could address more sophisticated attacks. After researching the market, we discovered Sublime Security. Our organization expressed considerable interest in tools that utilize LLM and AI agent technologies. After conducting a thorough review, we selected Sublime Security, and we have been using it in our environment ever since.

The autonomous security analyst agent takes user-submitted emails and conducts a thorough review of them. Out of the box, Sublime Security comes with a variety of security rules that it applies to incoming emails. However, when someone submits an email to a phishing mailbox, the autonomous security analyst performs an even more in-depth analysis of that message. This tool utilizes an underlying agentic component powered by a large language model (LLM), which, as I understand, currently uses Claude 3.7. The analyst thoroughly examines the email and provides not only the verdict it arrives at but also a detailed explanation of how it reached that conclusion. This includes all the logic and reasoning used during its evaluation, giving us a level of detail that is quite valuable.

I have found that the verdicts are generally accurate, although there can be unusual edge cases in any organization. The information returned from the analysis has helped me understand the rationale behind decisions, clarifying why something was classified as spam, gray mail, or malicious. While these classifications are typically reliable, specific contexts might require additional considerations. Since September of this year, we have been operating in a fully automated remediation mode with the autonomous security analyst. So far, I haven’t seen many instances of users disputing the verdicts. We communicate the results to those who report phishing emails, informing them of the decision made and the actions taken. Users have been generally receptive to this feedback, and we aim to be transparent about our reasoning.

We continuously evaluate the cases labeled as either malicious or benign to ensure that legitimate emails are not incorrectly flagged as dangerous. Our goal is to avoid mistakenly labeling a dangerous email as safe. Although I trust the tool, we are diligent about validating its performance regularly, especially since the email landscape is always changing. Overall, I have been very satisfied with the tool, and we are currently achieving about a 95% automation rate for user-submitted emails.

I felt comfortable moving forward with enabling auto-remediation due to the detailed information provided. As an analyst, if I encountered something that didn't seem quite right or if there was an edge case in the environment that required a different response, I could easily navigate and address these issues. The web interface offers a comprehensive layout for evaluating emails in depth, providing all the essential details one would need for decision-making. For instance, you have access to basic information such as SPF and DKIM, along with complete header information. You can also retrieve a copy of the email itself, including any attachments, which can be placed in a sandbox environment for further analysis. This allows you to utilize any other tools or assessment methods necessary to verify findings, giving you complete freedom to investigate. Overall, the auditability and the ability to evaluate the information in Sublime through various mechanisms made me very comfortable with setting these capabilities to auto-remediation. So far, we have been very pleased with the results.

It is a big deal for us that Sublime Security offers reduced vendor dependency in deploying new protections. Being able to utilize this platform regardless of the environment it monitors is crucial. We have Sublime Security deployed in both a Microsoft environment and a Google Workspace setting. The ability to enable rules through the same platform, regardless of the monitored environment, and to do this via API calls makes the process much cleaner. It eliminates the need to rely on other tools to perform its functions. In the future, we aim to extract information from the Sublime platform to integrate with a Security Orchestration and Automated Response (SOAR) platform, like Simplify or Demisto Cortex. We’re keen on leveraging data from Sublime Security, but for now, we’re very satisfied with performing most tasks directly within the portal. We are also considering other automation opportunities moving forward. Overall, the ability to apply this functionality without initially worrying about integration with other tools speeds up onboarding and enhances our value from the system right from the start.

Based on a 30-day period, using the Microsoft Defender toolset, around 240,000 phishing messages are blocked before delivery each month. Sublime Security's control set operates after this initial filtering. The basic phishing attempts—those that are well-known and have malware attached—are typically blocked by Microsoft Defender. However, of the emails that do get past Microsoft Defender, we still triage and remediate about 186,000 emails a month within Sublime Security. This number represents emails that successfully bypassed Microsoft Defender's capabilities. Having Sublime Security as an additional layer of defense significantly enhances our security. To put it into perspective, we are blocking about two-thirds more unwanted emails with Sublime Security compared to what Microsoft Defender blocks. So, we are noticing a substantial reduction in unwanted emails, even after other security controls have had their chance to filter them out but failed to do so.

What I appreciate most about Sublime Security is the amount of detail that's provided. I've used Microsoft Defender and, in the past, Proofpoint for similar purposes. Both offer valuable insights, but what stands out about Sublime Security is how accessible the detailed analyses are for the messages that have been evaluated. The tools clearly explain why a specific verdict was assigned to a message. This level of detail is essential for analysts and anyone conducting evaluations, as it helps them understand the specific circumstances of their environment. What may be clearly malicious in one organization could be viewed as legitimate or benign in another. The information provided about flagged emails and the subsequent evaluations offers a comprehensive breakdown of how a particular conclusion was reached, which has been incredibly beneficial.

I know that a lot of time has been invested in improving the efficacy of the platform, and it shows; it performs very well. Moving forward, I think our focus should be on how to achieve better integration with other systems. While they do provide API-level access and web hooks, I believe more out-of-the-box integrations with SOAR platforms and SIEM tools would enhance Sublime's value. This would allow it to be integrated more closely with the workflows of various teams and could potentially increase its market appeal. From my perspective, the tool itself functions exceptionally well, which gives me confidence in the system. I want to see this functionality extend to other tools that I use, enabling faster automation and improved workflows for the team, particularly from a security operations standpoint. I have no critiques regarding the tool itself. They've done an outstanding job and are maintaining high quality throughout their development process. They have a great product, and it's essential that they continue to uphold that standard, even though it requires significant effort.

I have been using it since 2024.

My thoughts on the stability of Sublime Security are positive. I believe the effectiveness of the system is largely due to its API-based development and its scalable infrastructure on the back end. I haven't experienced any slowness when using their portal or during the actions they take to remove malicious content from the environment. For example, our CISO contacted me after receiving a notification on their phone about an email that Microsoft initially thought was valid. However, by the time they opened their Outlook client, the email had already been removed because it was deemed malicious. Sublime Security acted quickly; once Microsoft alerted Outlook about the incoming email, Sublime Security managed to pull it from the stream before it had a chance to appear in the user's inbox. This happened so swiftly that the email was deleted before the intended recipient ever received it. We were able to conduct a thorough evaluation and provide the CISO with all the information they needed, which made them very satisfied. If we can meet the expectations of such a demanding customer in terms of email security, it's a strong indication of the system's reliability.

Regarding scalability, we initially protected around 20,000 mailboxes daily, and that number has now increased to about 35,000, with no signs of lag or slowness. When we first onboarded the tool, there were close to 600 active detection rules. That number has now risen to 734. Despite nearly doubling the number of mailboxes and adding significantly more detection rules, the service remains just as fast as it was when we first implemented it. Overall, it appears to be highly scalable.

We have a Teams channel that we use regularly with them. They provide updates on new capabilities being enabled, as well as information that allows us to open cases if we have questions. If there's an issue in the platform that they want us to investigate further, this communication channel allows for free-flowing dialogue.

I haven't been disappointed so far; their response is very fast. If there's something more complex, they can connect you with the appropriate engineering resources for a more in-depth conversation. We have our own internal AI review panel, and we were able to get in touch with the person who constructed the LLM and its agentic aspects relatively quickly to address our questions. They offer a wide range of expertise to their customer base on relatively short notice. Overall, they have been highly responsive and very helpful.

It's very simple. You just need to gain API-level access to a Microsoft 365 tenant or a Google Workspace account. As long as someone can provide an API key with the necessary permissions, that's all it takes. The entire process of setting it up is straightforward and easy to implement.

It could take about a week to get the right approvals because of the large organization, but the actual technical implementation takes 30 minutes.

Maintenance is taken care of by Sublime. They do auto updates and addition of new rules and all of that. It's completely hosted by them. It's a full SaaS model.

Sublime Security has proven to be a significant advantage for us. We transitioned from relying on a third-party service provided by Cofense for handling user-reported phishing emails to managing everything in-house. This means that our previous relationship with Cofense has ended, and we have fully replaced their services with Sublime, which automates many remediation actions.

Importantly, we didn’t need to hire any additional staff; we were able to absorb this responsibility using our existing team. As we activated more functionalities of the tool, our team learned to manage the process effectively. Now, we handle everything internally, utilizing both the tool and our team's processes.

It's very reasonable. It's competitive with its peers, especially for the number of mailboxes we have.

We have not yet fully enabled ADÉ, but I am working on getting it activated because we have confidence in the rest of the toolset and its available functions. Currently, it is undergoing evaluation and remains in public beta. For any components that we activate, particularly those based on AI, we have an internal review board. This board focuses on determining whether a large language model (LLM) or AI component will be used to facilitate model learning in other environments or if it will be restricted solely to our tenant. We want to ensure that our internal organizational messages are not used to train external models. This review process will be conducted on a function-by-function basis, even for tools we've previously assessed. Although we haven't completed this review for the component yet, it is on our agenda. I would like to have it turned on before the end of the calendar year. We are moving forward with this, but it must pass our internal review first. If the review results are positive, we will aim to enable it before the end of December.

I would rate Sublime Security a nine out of ten.