Troubleshooting Firewall Reboots due to tWebMain process in Gen 6 devices

03/26/2020 100 13304

DESCRIPTION:

This article outlines the steps in troubleshooting when the firewall is rebooting due to suspension of tWebMain. This is a process responsible for displaying events/information in the GUI of the firewall.

Usually when this happens, the CPU or Core 1 (in GUI) spikes up to a 100%.

CAUSE:

There are several factors that can keep tWebMain busy:

• GUI logging: This depends on the number of log events generated. The logging on the firewall, when enabled for regular events (eg: DNS Allow, ICMP allowed, Connection NAT Mapping), will generate a large number of events. This overwhelms the tWebMain process if we have GUI Alerts enabled.
• App Control Logging / App Rule Logging
• Log Name Resolution: When this is set to DNS Then NetBIOS or just NetBIOS, requests from the machines to public DNS servers requesting PTR records, will result in errors
• Site to Site VPNs: These when configured incorrectly, even though the tunnel might work, will repeatedly generate errors which are logged.

The CPU Utilization can be found in the TSR of the firewall by doing a search for "CPU Utilization" or "Utilization per process".

The following is an excerpt from a TSR: it shows that the overall CPU Utilization is 65% (Which is above normal, but not alarming). The value 50 denotes the priority the process gets. In other words, if there is a higher priority process coming in, it will preempt one of the lower priority processes.

--CPU Monitor--
CPU Monitor:
Current 1s CPU Utilization: 100.00%
Current 10s CPU Utilization: 100.00%

Total Average CPU Utilization: 65.15%

CPU Utilization Per Process:
# Name PC PRI Total% (secs) Curr% (secs)
--- ----------------- ---------- --- ------------- -------------
1. pass_to_stack 0x81203774 50 16.24 (1224.62) 28.57 (0.30)
2. tWebListen 0x81203774 50 9.89 (746.07) 17.46 (0.18)
3. tWebMain06 0x81200cb0 50 1.50 (113.23) 6.35 (0.07)

RESOLUTION:

The following changes can be made in the firewall settings to prevent tWebMain from causing issues:

• GUI Logging: This change should be done under Log | Settings: disable GUI alerts for regular events, or if they have to be logged, send them to a syslog server (if one is configured), or change the Log filter value to Non zero (typically 60 seconds). Please see: Reduce CPU usage reviewing logs configuration
• App control/ App rule logging: This change should be done under Firewall | App Rules or Firewall | App control Advanced
  • For App Rules, change the Global Log Redundancy filter value to non zero value.
  • For App Control, change the filter value for the categories (27 in total). This eliminates redundant logs for the specified period of time.

• Log Name Resolution: Under Log | Name Resolution, set the value to None or just DNS.

• In the diag.html page
  • Enable the "Main Log Process Reschedule Interval" and set the value to 100 entries.
  • Change the "Log Output Limit:" value to "1000" entries (Available for SuperMassive10000 series).
  • Disable the "Exempt Redundancy Filter intervals of unfiltered events from global, category-level and group-level changes"
• If AppFlow is enabled, please disable it from AppFlow | Flow Reporting by disabling "Enable Real-Time Data Collection" and "Enable AppFlow To Local Collector"

NOTE: If the issue persists even after making the changes, collect stack trace information, TSR, settings files and Console logs if necessary and create a Collaboration Request with SEEG. Click on this link to read more about How to obtain diagnostic Logs/data from console cable connection when Firewall is freezing/locking.

NOTE: When change values in the diag.html page on SuperMassive10000 series running firmware version 6.0.5.2-72o or 6.0.5.2-73o, the firmware needs to be upgraded to 6.0.5.2-74o or later due to any button in the diag.html page are not responsive.