MITRE Engenuity ATT&CK Examination for Managed Services (menuPass + ALPHV BlackCat)
The latest findings from the recent round of ATT&CK® Reviews for Managed Services conducted by MITRE Engenuity™ have been unveiled. They evaluated 11 vendors on their capacity to recognize, analyze, and precisely portray real-world adversary actions.
This assessment marked the second edition of ATT&CK Reviews for Managed Services, which was initially introduced in 2022 to enhance organizations’ understanding of how products like Sophos MDR can shield them against intricate, multi-phase assaults.
If you wish to get a quick overview of the evaluation, you can view this short video:
What was the extent of the ATT&CK Reviews?
The ATT&CK Evaluations by MITRE Engenuity were structured to mimic a representative scenario of how a managed service provider should engage with organizations during a sophisticated attack.
During this round, the MITRE Engenuity team replicated the actions of recognized threat actors. A ‘black box’ methodology was adopted, where MITRE revealed neither the simulated threat actor(s) nor the scope of techniques until the evaluation was concluded.
This evaluation imitated the tactics and techniques employed by two established threat groups – menuPass and ALPHV/BlackCat – and appraised each vendor’s capability to spot and describe particular adversarial activities.
The overall evaluation encompassed 172 adversary activities (sub-steps) across 15 primary steps. However, only 43 sub-steps, which MITRE Engenuity deemed crucial for the success of the attack sequence, were included in the results.
The evaluation was exclusively concentrated on detection and reporting. The evaluation did not scrutinize the ability to obstruct, react to, or resolve threats. It is essential to note that adversary actions emulated in this evaluation might have been hindered by protective technologies (e.g., advanced endpoint tools) which vendors needed to deactivate during the evaluation.
Participants in the Evaluation
In this evaluation phase, eleven managed security service providers took part:
| Bitdefender | BlackBerry | CrowdStrike | Field Effect |
| Microsoft | Palo Alto Networks | SecurityHQ | Secureworks |
| SentinelOne | Sophos | Trend Micro |
Results of Sophos
The outcome of MITRE ATT&CK Evaluations can be interpreted in various ways, with MITRE Engenuity refraining from ranking or designating any vendor as a “champion” or “leading” entity. The manner in which each vendor presents information about their managed service is as crucial as the results themselves, considering the unique needs and preferences of each organization.
Sophos admirably “Reported” and accurately delineated 84% of the 43 adversarial activities (sub-steps) selected by MITRE Engenuity – a figure higher than the average of the participating vendors. A significant majority (75%) of Sophos’ detections were classified as “Actionable”. In this context, “Reported” signifies that the adversarial activity was successfully identified and adequately contextualized. Moreover, when the reported information effectively encapsulates the “5 W’s” (Who, What, When, Where, and Why), the activity is further labeled as “Actionable”.
The results also feature the count of alert emails disseminated by each vendor.
To ensure an efficient, comprehensible, and proactive response, Sophos MDR emphasizes on furnishing high-value, human-written notifications containing critical details and context that customers require.
During the 5-day MITRE ATT&CK Evaluation for Managed Services, Sophos MDR dispatched 24 emails. In contrast, the average among other participants exceeded 120 emails, with certain vendors sending over 300 emails. Alert fatigue, arising from an overwhelming influx of notifications from security solutions, stands as a considerable concern in the realm of cybersecurity. Sophos recognizes the significance of your organization’s time and values quality over quantity especially when resources are limited.
Utilizing MITRE Engenuity ATT&CK Evaluation Outcomes
ATT&CK Evaluations currently rank among the most esteemed autonomous security assessments globally, owed largely to the meticulous reconstruction and emulation of actual attack scenarios, transparency in results, and the abundance of participant information.
When contemplating a Managed Detection and Response (MDR) service, it is advisable to review the findings from MITRE Engenuity ATT&CK Evaluations in conjunction with other reputable corroborative evidence from third parties, including authenticated customer reviews and analyst assessments.
While analyzing the data accessible on MITRE Engenuity’s evaluation portal, go beyond the statistics and ponder over the following aspects, acknowledging that some queries about managed security services cannot be resolved through ATT&CK Reviews, such as:
- Is the service presenting information to you in the manner you desire, with valuable communications containing the essential information you need?
- Does the service presume that you possess an in-house security operations team, or can they furnish a complete ‘instant SOC’ equipped to take actions for threat eradication on your behalf?
- Who will be liaising with the managed service provider on a day-to-day basis? IT Administrators, seasoned security analysts, or perhaps both?
- Can the service seamlessly integrate with other technologies in your environment to detect and respond to multi-tier threats extending beyond endpoints (such as firewall, email, cloud, identity, network, backup and recovery, etc.)?
- Does the service provide comprehensive remote incident response, and are the inclusive IR services capped at a fixed number of hours, or unlimited?
Reasons for Our Participation
Sophos is dedicated to participating in MITRE Engenuity ATT&CK Evaluations alongside some of the premier security vendors in the field. As a collective, we stand united against a common adversary. These assessments aid in the enhancement of our individual and collective capabilities, ultimately benefiting the organizations we safeguard.
Our involvement in the recent evaluation serves to fortify Sophos’ position as a top-tier Managed Detection and Response (MDR) provider and a reliable cybersecurity ally to over 22,000 clients.
Don’t rely solely on our assurances
Sophos Managed Detection and Response emerges as the most favored MDR solution globally. We secure a larger number of organizations compared to any other MDR provider, boasting extensive expertise across diverse industries and sectors. Recent external validation encompasses:
To delve deeper into Sophos MDR and how it can cater to your requirements, explore our website or engage with a security specialist today.
About Author
