Miercom 2024 NGFW Security Benchmark | Check Point Software

Miercom 2024 NGFW Security Benchmark

NGFW Firewall Security Benchmark 2024 Firewall Security Efficacy Competitive Assessment Summary Lab Report for Check Point Software
January 2024 SR240117C miercom.com/checkpoint

Table of Contents

1.0 Executive Summary
2.0 Testing Summary Results
2.1 Malware Prevention and Detection Summary
2.1.1 Malware Prevention vs Detection-Only Zero+1 Day Malware
2.1.2 Malware Prevention Efficacy Zero+1 Day Malware
2.2 Malicious Phishing URLs Prevention and Detection Summary
2.2.1 Phishing and Malicious URL Prevention
3.0 False Positive Detection
3.1 False Positive Testing Summary
3.1.1 False Positive Rate for Malware Detection
4.0 Products Tested
5.0 Test Setup
5.1 Miercom Advanced Offensive Threat Detection
5.2 VirusTotal
5.3 Testing Environment
6.0 About Miercom
7.0 Use of This Report

1.0 Executive Summary

Miercom was engaged by Check Point to conduct competitive security effectiveness testing of the Check Point Next Generation Firewalls (NGFW) as compared to products from Cisco, Fortinet, and Palo Alto Networks. Testing with Zscaler involved their SWG (Secure Web Gateway). Testing included verifying the effectiveness of anti-virus, anti-malware, Intrusion Prevention System (IPS), anti-bot, URL Filtering (URLF), sandboxing, machine learning, and phishing protection. We conducted tests with all security services enabled and challenged each solution’s ability to detect and block modern-day malware. Modern threats like web-based malware attacks, targeted phishing attacks, application-layer attacks, and others increase the threat level against organizations globally. The majority of new malware and intrusion attempts to exploit weaknesses in applications, as opposed to networking components and services. NGFWs with advanced threat prevention offer the best protection against the latest generation of cyberattacks. Our testing specifically focused on the ability to detect and prevent new malware variants within the first 24 hours of their discovery as well as detecting and preventing new phishing sites.

In this report, Zero+1 Day Malware (one day past Zero-Day discovery) means newly discovered malware on the first day of discovery. These malware samples are less likely to be known by any vendor’s signature detection mechanisms in the first 24 hours.

Key Findings

Critical Prevention Rate in the first 24 hours: Check Point led in the group test for immediate prevention of the total malware samples. The first 24 hours of a malware campaign are the most dangerous, and this is the critical time to stop an attack before it quickly spreads and creates widespread damage. A security system with a higher block rate in the first 24 hours means an enterprise will spend less time, money, and energy responding to and remediating infected servers and endpoints.

2.1 Malware Prevention and Detection Summary

2.1.1 Malware Prevention vs Detection-Only Zero+1 Day Malware

New Variant Malware Prevention success rate: In our Zero+1 Day Malware test, Check Point prevented over 99.8% of malware from a large set of files and file types including executables, documents, and archives. Palo Alto Networks, Fortinet, Cisco, and Zscaler had prevention rates of 69.4%, 84.0%, 47.8%, and 75.4% respectively.

2.1.2 Malware Prevention Efficacy Zero+1 Day Malware

The chart above reflects how each vendor's firewall performed prevention in the first 24 hours of an attack.

2.2 Malicious Phishing URLs Prevention and Detection Summary

2.2.1 Phishing and Malicious URL Prevention

Check Point demonstrated not only static detection ability but could also detect phishing websites dynamically with AI-based phishing protection, based on analysis of web page content such as corporate logos/icons, suspicious fields, irregular spellings, redirection, and many other obscured maleficent components of these websites.

3.0 False Positive Detection

3.1 False Positive Testing Summary

False positive occurrences are non-malicious (or benign) files that are misidentified as malicious.

3.1.1 False Positive Rate for Malware Detection

Check Point scored the best, with the lowest false positive detection compared to Palo Alto Networks, Fortinet, Cisco, and Zscaler.

4.0 Products Tested

5.0 Test Setup

The testing conducted was designed to determine the strengths and weaknesses of each NGFW product. Over the course of 90 days, we repeatedly downloaded sets of 500 malicious files from VirusTotal. Testing included review of thousands of samples over the previous 90 days.

6.0 About Miercom

Miercom has published hundreds of network product analyzes in leading trade periodicals and other publications. Miercom’s reputation as the leading, independent product test center is undisputed.

7.0 Use of This Report

This document is provided "as is", by Miercom and gives no warranty, representation, or undertaking, whether express or implied, and accepts no legal responsibility, whether direct or indirect, for the accuracy, completeness, usefulness, or suitability of any information contained in this report.