Understanding the CyberRatings SSE Report and Why Zscaler is Named A Top Provider

CyberRatings, the leading non-profit security testing organization, recently tested several vendors’ security service edge (SSE) offerings with the goal of deriving empirical data that would answer the question, “Are SSE products secure by default?”  

The emphasis is aligned to a shift in the industry towards “secure by default” approaches to develop and deploy software products. The trend is an outcome of a 2023 report authored by CISA and its international partners that emphasizes that the burden of security should be shifted away from the end-user and back towards software vendors.

In CISA’s words:

Secure-by-Default means products are resilient against prevalent exploitation techniques out of the box without additional charge. These products protect against the most prevalent threats and vulnerabilities without end-users having to take additional steps to secure them.

Scope of the test: a “quick look” at default security posture

CyberRating labeled this as a “mini-test” because their analysts did not intentionally run through a comprehensive evaluation of the full capabilities of the vendors’ SSE platforms. Instead, with the mini-test, CyberRating’s analysts sought to provide a data-driven “quick look” at the default security posture vendors can deliver with minimal to no additional security configuration of their SSE.

In short, CyberRatings wanted to test the initial functional working state of these SSE offerings. For a more complete evaluation of Zscaler SSE, see the June 2024 CyberRatings SSE Report in which Zscaler achieved an “AAA” rating. 

SSE security posture test methodology

The SSE mini-test was designed to provide insight into the default security posture across these platforms using a small subset of malware samples (using ~3,000 samples vs. the 100,000+ samples in CyberRating’s more all-inclusive tests). CyberRatings ran the following tests using Windows 11 clients configured with each vendor’s SSE client software:

• Test 1: Download ~1,000 benign samples over HTTP designed to be susceptible to being classified as malware despite being innocuous (e.g., the solution’s propensity for triggering false positives).
   
• Test 2: Download ~3,000 active malware samples over HTTP, current to within 30 days of the test (e.g., the SSE’s ability to detect and block basic malware downloads). No evasions were applied.

Zscaler SSE found “secure-by-default” with zero false positives

In its default configuration, Zscaler stands above our competitors in the mini-test of SSE platforms: Zscaler blocked about 97% of the malware test samples with no false positives. Zscaler also includes Cloud Sandbox capability out of the gate, unlike several of our competitors. 

Here’s what's notable in the evaluation results in relation to our competitors:
 

• Zscaler blocked the most malware samples when compared to other vendors’ SSE platforms running in their default configurations. Zscaler also did not produce any false positives. With our SSE, the most important security controls needed are automatically enabled to protect enterprises from malicious cyber actors.
   
• Cisco’s, Checkpoint’s, and Versa Networks’ SSE platforms did not detect any malware samples when their default configuration remained unchanged. In other words, they do not provide security by default. Their ability to inspect traffic for malware is dependent on making configuration changes. Yet, security teams are frequently overloaded with security and operational responsibilities, resulting in limited time to dig through documentation to understand and implement robust cybersecurity posture.
   
• Cisco still drew false-positives, even when the CyberRatings evaluators made changes to their default configuration.

The keys to effective AI-driven threat detection are a quality dataset and Zscaler’s AI models training on the telemetry collected by operating the world’s largest security cloud. This cloud continuously collects and analyzes over 500 trillion daily signals and 500 billion daily transactions.
 

An effective SSE offering also reduces complexity

The results of this mini-test are clear: customers need ease-of-use combined with high efficacy to defeat today’s advanced cyberthreats—and Zscaler was found to deliver:  

• Security by default—there is no configuration option hidden somewhere in the administrative console that must be enabled.  Zscaler automatically enables the most important security controls needed to protect enterprises from malicious threat actors. In short: we deliver security value upon deployment and thus deliver fast time to value. 
   
• Effective blocking of malware without introducing false positives. False positives can consume a security team’s valuable time that could have been spent on higher-value work. 
   
• Frictionless configuration that eliminates complexity. Security teams are frequently overloaded with security and operational responsibilities, resulting in limited time to understand and implement robust cybersecurity posture.
   

Find out more about the CyberRating’s SSE mini-test: Get your copy now