HyperFlow

Overview

Elephant flows are large (in total number of bytes) continuous connections that the TCP or UDP establishes.

For example, a download of a large file (such as a Linux ISO file) over the HTTP, HTTPS, FTP, or NFS protocol.

These large continuous connections consume the network capacity significantly in comparison to other types of data sessions.

Without the HyperFlow feature, a Security Gateway uses only one CPU core (one CoreXLFirewall instance) to inspect one elephant connection. In addition, traffic throughput decreases gradually as the CPU utilization increases on the Security Gateway.

The HyperFlow feature on Security Gateways R81.20 and higher handles such elephant connections on more than one CPU core in parallel.

The HyperFlow feature breaks the whole inspection task into smaller tasks and dispatches these smaller tasks to the available CPU cores:

As a result, the HyperFlow feature:

• Increases throughput of elephant connections when Threat Prevention Software Blades are enabled (the Security Gateway takes less time to inspect elephant connections).

• Automatically detects and dynamically allocates the CPU cores between main tasks on a Security Gateway.

• Improves response time from the CoreXL FWK processes while they inspect elephant connections (the idle time of the corresponding CPU cores increases).

For additional information, see sk178070.

Requirements

1. Check Point Appliance models with at least 8 CPU logical cores.

For the list of supported models, see sk178070.

2. Firewall in User Mode (USFW). See sk167052.

3. Enable the CoreXL Dynamic Balancing (see Dynamic Balancing of CoreXL Instances):

4. Configure SecureXL to work in Kernel Mode (KPPAK) (see Configuring SecureXL).

5. Enable the applicable Software Blades from one of these categories:

   • NGFW

   • NGTP

   • NGTP with HTTPS Inspection

   • NGTX

Glossary

• Processing incoming traffic from the network interfaces
• Securely accelerating authorized packets (when SecureXL is running)
• Distributing non-accelerated packets among FW instances | CoreXL_FW_RESERVED | A logical sibling of a CoreXL Firewall instance (FW worker) that handles a heavy connection.
  When a logical CPU core is utilized at a high level because it handles heavy connections, its logical sibling can be stopped to decrease its utilization of resources from the physical CPU core.
  Chain of events:
  1. CoreXL assigns a CoreXL Firewall instance to inspect an elephant connection.
  
  2. This CoreXL Firewall instance runs on a logical CPU core of a physical CPU.
  
  3. To improve the internal performance of the physical CPU, the CoreXL Dynamic Balancing feature can stop the CoreXL Firewall instances on the sibling logical CPU cores of the original logical CPU core. | | PPE | Parallel Processing Engine architecture.
  This is the HyperFlow dynamic infrastructure that allocates CPU cores as required to increase the throughput of Elephant connections.
  This is the thread that polls the interface queues and retrieves packets.
  Also known as Dual Mode Dispatcher (DMD). | | PPE_MGR | Parallel Processing Engine Manager. Receives packet payload and dispatches jobs to PPE. Works on a complete physical CPU core (if there are 32 or more CPU cores). | | PPE_MGR_RESERVED | A sibling of PPE_MGR. Because PPE_MGR works on a complete physical CPU core (if there are 32 or more CPU cores), it is not possible to use other logical CPU cores on that physical CPU core. The logical siblings have the status "PPE_MGR_RESERVED" | | PPE_WT | Parallel Processing Engine Worker Thread. Receives packet handling jobs from the Parallel Processing Engine Manager. Dispatches jobs to Worker Threads (WTs.) Works on a complete physical CPU core (if there are 32 or more CPU cores). | | DPDK | Data Plane Development Kit. | | PMD | Poll Mode Driver. | | WT | Worker Thread.
  The thread that executes the packet handling logic.
  In plural: WTs |

Syntax

Parameters

• Status of connection pipelining: <Status>
  - Flow mode: <Mode>
  - Status of PPE_MGR and PPE_WT: <Status> | | sync | Configures the synchronous flow mode. In this mode, CoreXL Firewall instances do not send jobs to the PPE. | | wake_up | Wakes up the Job Dispatcher (PPE) and Working Threads (WTs) from sleep.
  The PPE gets CPU cores available for allocation. |

Monitoring in CPView

You can monitor how the HyperFlow performs on the Security Gateway.

1. Go to CPU > Advanced

2. If the tab Hyperflow appears, it means HyperFlow is enabled

3. Go to CPU > Advanced > Hyperflow > Overview

4. Examine the field PPE_MGR state

5. Go to CPU > Overview > Host

6. Examine the last section CPU (pay attention to the column Idle)

7. On the Security Gateway, examine the elephant connections in the past 24 hours and which CoreXL Firewall instance inspects them (see the "[fw_<Number>]" in the beginning of each line):

Example:

2. In CPView, go to CPU > Top-Connections > Instances- (for example, Instances0-5) > Instance (for example, Instance5)
3. In the last section Top Connections, examine the columns "% out of CPU" and "% out of WT CPU"

Example (this is only the applicable part of the output):

1. Go to Advanced > HyperFlow > Overview > PPE_0
2. Click the applicable tab
3. Examine these sections:

1. Go to Advanced > HyperFlow > Firewall-messages
2. Examine these sections:

1. Go to Advanced > HyperFlow > Jobs > PPE

2. Examine these sections:

   • Sent to firewall (count)
   • Average execution time (Cycles)

3. Go to Advanced > HyperFlow > Comm > PPE

4. Examine these sections:

   • Enqueue
   • Dequeue

5. Go to Advanced > HyperFlow > WT

6. Examine these rows:

   • Number of WTs
   • Worker ID State

Limitations

• Firewall in Kernel Mode (KFW) is not supported.
• Open Servers and Virtual Machines are not supported.
• Cloud platforms are not supported.
• HyperFlow is not supported when the Security Gateway / Cluster is configured to work as an HTTP/HTTPS Proxy.
• When an elephant connection triggers the HyperFlow, the outputs of the "top" and "ps" commands may show that the HyperFlow user space processes consume some CPU cores at 99-100%.

This happens because HyperFlow is constantly polling its queues to handle the incoming jobs. After the elephant connection closes, the output of these commands shows that the user space "us" consumption goes back to regular levels because HyperFlow stops processing the jobs.

To see the actual load on the CPU, use CPView (CPU > Overview > Host), SNMP, or SmartConsole.

This does not trigger inspection bypass because of a high CPU load.

• When the HyperFlow feature is enabled, the speed of an elephant flow may not increase in these cases:
  • This is a complex connection that requires special parsers – FTP, SCP, VoIP
  • This connection undergoes the Content Awareness or a Data Loss Prevention policy
  • This connection has dynamic content and goes through the Pattern Matcher 1st tier
  • Strict Hold is enabled (the $FWDIR/conf/malware_config file on the Security Gateway contains "strict_hold_enable=1")
  • This connection is HTTPS and the HTTPS Inspection is disabled
• When you enable only the Firewall Software Blade in the Security Gateway object, the HyperFlow feature does not improve the performance.

Troubleshooting

• Log files (the main file is rotated every 10 megabytes):
  • $FWDIR/log/connection_pipelining.elg
  • $FWDIR/log/dmd.elg
  • $FWDIR/log/dmd_controller.elg
  • $FWDIR/log/dsd.elg
• Internal configuration files (you must not edit these files):
  • $FWDIR/conf/connection_pipelining.conf
  • $FWDIR/conf/connection_pipelining_params.conf

For additional information, see sk178070.