Report | Miercom NGFW Security Benchmark, 2024

NGFW Firewall Security Benchmark 2024 Firewall Security Efficacy Competitive Assessment Summary Lab Report for Check Point Software January 2024 SR240117C miercom.com/checkpoint

https://miercom.com/checkpoint

Table of Contents

1.0 Executive Summary 2.0 Testing Summary Results 2.1 Malware Prevention and Detection Summary 2.1.1 Malware Prevention vs Detection-Only Zero+1 Day Malware 2.1.2 Malware Prevention Efficacy Zero+1 Day Malware 2.2 Malicious Phishing URLs Prevention and Detection Summary 2.2.1 Phishing and Malicious URL Prevention 3.0 False Positive Detection 3.1 False Positive Testing Summary 3.1.1 False Positive Rate for Malware Detection 4.0 Products Tested 5.0 Test Setup 5.1 Miercom Advanced Offensive Threat Detection 5.2 VirusTotal 5.3 Testing Environment 6.0 About Miercom 7.0 Use of This Report

1.0 Executive Summary

Miercom was engaged by Check Point to conduct competitive security effectiveness testing of the Check Point Next Generation Firewalls (NGFW) as compared to products from Cisco, Fortinet, and Palo Alto Networks. Testing with Zscaler involved their SWG (Secure Web Gateway). Testing included verifying the effectiveness of anti-virus, anti-malware, Intrusion Prevention System (IPS), anti-bot, URL Filtering (URLF), sandboxing, machine learning, and phishing protection. We conducted tests with all security services enabled and challenged each solution’s ability to detect and block modern-day malware.

Modern threats like web-based malware attacks, targeted phishing attacks, application-layer attacks, and others increase the threat level against organizations globally. The majority of new malware and intrusion attempts to exploit weaknesses in applications, as opposed to networking components and services. NGFWs with advanced threat prevention offer the best protection against the latest generation of cyberattacks. Our testing specifically focused on the ability to detect and prevent new malware variants within the first 24 hours of their discovery as well as detecting and preventing new phishing sites.

In this report, Zero+1 Day Malware (one day past Zero-Day discovery) means newly discovered malware on the first day of discovery. These malware samples are less likely to be known by any vendor’s signature detection mechanisms in the first 24 hours.

Key Findings

• Critical Prevention Rate in the first 24 hours: Check Point led in the group test for immediate prevention of the total malware samples. The first 24 hours of a malware campaign are the most dangerous, and this is the critical time to stop an attack before it quickly spreads and creates widespread damage. A security system with a higher block rate in the first 24 hours means an enterprise will spend less time, money, and energy responding to and remediating infected servers and endpoints.
• Zero+1 Day Malware Prevent vs. Detect Tests: Check Point prevented over 99.8% of new malware from a comprehensive set of files and file types, including executables, documents, and archived files that were no more than one day old. Check Point led with the highest score preventing 99.8% of malware downloads. Fortinet had 84.0% prevention and 9.4% detect-only; Zscaler had 75.4% prevention; Palo Alto Networks had 69.4% prevention and 8.7% detect-only; Cisco had 47.8% prevention and 37.2% detect-only.

2.1 Malware Prevention and Detection Summary

• Summary of NGFW Test Results: Blocking and Detection Efficacy comparing test results from Zero+1 Day recently discovered malware between products.
• New Variant Malware Prevention success rate: In our Zero+1 Day Malware test, Check Point prevented over 99.8% of malware from a large set of files and file types including executables, documents, and archives. The chart above reflects how each vendor's firewall performed in Prevention vs. Detection-Only in the first 24 hours of an attack.

2.2 Malicious Phishing URLs Prevention and Detection Summary

• Missed malicious URLs, less is better: Check Point demonstrated not only static detection ability but could also detect phishing websites dynamically with AI-based phishing protection, based on analysis of web page content.

3.0 False Positive Detection

• False Positive Testing Summary: False positive occurrences are non-malicious (or benign) files that are misidentified as malicious. Check Point scored the best, with lowest false positive detection compared to Palo Alto Networks, Fortinet, Cisco, and Zscaler.

4.0 Products Tested

• Check Point: Version: R82 and R81.20 Data sheet and specifications
• Palo Alto Networks: Version: PAN-OS 11.1.1 Data sheet and specifications
• Fortinet: Version: FortiGate/FortiOS 7.4.2 Data sheet and specifications
• Cisco Systems: Version: 7.4.1 Data sheet and specifications
• Zscaler: Platform version: 6.2 Client Version: 3.9.0.156 Data sheet and specifications

5.0 Test Setup

The testing conducted was designed to determine the strengths and weaknesses of each NGFW product using verified malicious samples for a customized, open-source approach. Over the course of 90 days, we assessed each NGFW solution using AV + Anti-Malware, IPS, anti-bot, URLF, sandboxing, and machine learning inline detection mechanisms.

6.0 About Miercom

Miercom has published hundreds of network product analyzes in leading trade periodicals. Miercom’s reputation as the leading, independent product test center is undisputed.

7.0 Use of This Report

This document is provided “as is,” by Miercom and gives no warranty, representation, or undertaking. All trademarks used in the document are owned by their respective owners.