Logger Service
See AlsoNetsweeper logging makes use of the Netsweeper LogMod5 logging framework, which is a graph-based framework that provides a high level of flexibility.
During normal operation, this is how the process flows:
- Log entries are queued up as they are generated. This queue is periodically emptied and the entries in the queue enter the logmod5 graph as a packet (bundle) of entries.
- The packet starts at the lm5_entry node and moves through the graphs via connections.
- The packet reaches an end point and is processed in some way.
For example:
- Written to nslogger_request.log log files
- Sent to a remote logger
- Stored temporarily to be processed later
This diagram explains in a little more detail:
The default configuration is a simple configuration that logs all entries to the local disk. By default, we will use a maximum of 5 gigabytes of disk space utilizing many log files that are required until this maximum is reached.
For further information on configuring LogMod5, please see: 'Configuration - LogMod5'.
The Logger Service
Inputs Logging
Inbound Ports Logging
Below is a list of inbound ports that the logging service uses.
|
Port |
Protocol |
Comments |
|---|---|---|
|
3000 |
TCP |
Remote logging packets received on this port. |
|
3434 |
TCP |
Remote administration commands. |
Logging and the Policy Service (nsd.conf)
The LogMod5 framework is configured through the 'Policy Server Settings' (nsd.conf) found under Administration > Services. It uses the Policy Server Settings file because the logging framework is a component of the policy service.
Using the nsd.conf file, the logging framework can be configured in many ways to suit your logging needs. This does, however, add complexity to the configuration of the logging framework. A view of the default configuration for LogMod5, as seen in the WebAdmin Policy Server Settings, and a diagram of the resulting LogMod5 graph, is shown below.
Incoming Logging Packets
The LogMod5 service, a component of the policy service, receives two distinct types of logging packets.
- Local logging packets
- These are logging packets received from the local policy service, in which LogMod5 is instantiated.
Remote logging packets
- These are remote logging packets received from a remote policy service.
LogMod5 can be configured to handle these two types of packets separately.
Outputs Logging
Outbound Ports Logging
Below is a list of outbound ports that the logging service uses.
|
Port |
Protocol |
Comments |
|---|---|---|
|
3000 |
TCP |
Remote logging packets sent to this port. |
|
3161 |
UDP |
SNMP. |
Request Logs Viewer
The Request Logs are typically the final destination for logging packets within the LogMod5 framework. They track all request log activity and become inputs for the Netsweeper Reporting Service.
By default, the Request Logs are located at the following location on the Netsweeper server (but can be located elsewhere if non-default configurations are used):
/usr/local/netsweeper/logs
Go to Logs > Request Logs to view the live logs being generated by the local logging service. You can filter log entries using the ‘Search’ box.
Logger Service SNMP MIBs
Any Netsweeper server provides a comprehensive set of SNMP monitoring MIBs that can be collected by an external system. MIBs provide specific operational metrics about each service that is operating on a Netsweeper service.
For logging, the following MIBs are available.
|
Option |
Description |
|---|---|
|
nsdLoggerPktsReceived |
Logger Input Packets Counter |
|
nsdLoggerPktsParseErrors |
Logger Input Packet Parse Errors Counter |
|
nsdLoggerPktsSent |
Logger Sent Packets Counter |
|
nsdLoggerPktsSendErrors |
Logger Unsuccessful Send Attempts Counter |
|
nsdLoggerRequestsLogged |
Logger Written Requests Counter |
|
nsdLoggerFileWriteErrors |
Logger File Write Errors Counter |
|
nsdLoggerRequestLogged rates |
Current Logged Requests Rate per Minute |
|
nsdLoggerRequestsLogged1MinuteRateMin |
Min Requests Logged Rate per Minute |
|
nsdLoggerRequestsLogged1MinuteRateMax |
Max Requests Logged Rate per Minute |
|
nsdLoggerRequestsLogged1MinuteRateAverage |
Average Logged Requests Rate per Minute |
|
nsdLoggerRequestsLogged5MinutesRateCurrent |
Current Logged Requests Rate per 5 Minutes |
|
nsdLoggerRequestsLogged5MinutesRateMin |
Min Logged Requests Rate per Minute |
|
nsdLoggerRequestsLogged5MinutesRateMax |
Max Logged Requests Rate per 5 Minutes |
|
nsdLoggerRequestsLogged5MinutesRateAverage |
Average Logged Requests Rate per 5 Minutes |
|
nsdLoggerRequestsLogged15MinutesRateCurrent |
Current Logged Requests Rate per 15 Minutes |
|
nsdLoggerRequestsLogged15MinutesRateMin |
Min Logged Requests Rate per 15 Minutes |
|
nsdLoggerRequestsLogged15MinutesRateMax |
Max Logged Requests Rate per 15 Minutes |
|
nsdLoggerRequestsLogged15MinutesRateAverage |
Average Logged Requests Rate per 15 Minutes |
|
nsdLoggerRequestsLogged1HourRateCurrent |
Current Logged Requests Rate per Hour |
|
nsdLoggerRequestsLogged1HourRateMin |
Min Logged Requests Rate per Hour |
|
nsdLoggerRequestsLogged1HourRateMax |
Max Logged Requests Rate per Hour |
|
nsdLoggerRequestsLogged1HourRateAverage |
Average Logged Requests Rate per Hour |
|
nsdLoggerRequestsLogged1DayRateCurrent |
Current Logged Requests Rate per Day |
|
nsdLoggerRequestsLogged1DayRateMin |
Min Logged Requests Rate per Day |
|
nsdLoggerRequestsLogged1DayRateMax |
Max Logged Requests Rate per Day |
|
nsdLoggerRequestsLogged1DayRateAverage |
Average Logged Requests Rate per Day |
|
nsdLoggerQueueAddOkayCount |
Number of log entries we added to the log processing queue |
|
nsdLoggerQueueAddFailedCount |
Number of log entries we failed to add to the log processing queue |
|
nsdLoggerQueueSoftLimitHitCount |
Number of log entries that triggers the soft limit of the queue. This means we had to signal the logging processor to process the queue |
|
nsdLoggerQueueHardLimitHitCount |
Number of log entries that triggered the hard limit. This means we had to wait for the queue to be processed |
|
nsdLoggerQueueSize |
The current size of the log processing queue |
|
nsdLoggerQueueAddStoppedDiscardCount |
Number of log records that were discarded because the logger was stopped |
|
nsdLoggerQueueAddHardLimitDiscardCount |
Number of log records that were discarded because the logger queue hit hard limit |
|
nsdLoggerQueueAddStoppedCount |
Number of times the logger was stopped |
|
nsdLoggerQueueAddDenyPageCount |
Number of requests that were server with deny page because logger had unrecoverable error |
|
nsdLoggerBytesReceived |
Logger Input Bytes Counter |
|
nsdLoggerBytesSent |
Logger Output Bytes Counter |
|
nsdLoggerConnectAttempts |
Number of attempts to connect to remote logging server(s) |
|
nsdLoggerConnectSuccess |
Number of successful attempts to connect to remote logging server(s) |
|
nsdLoggerConnectFailed |
Number of unsuccessful attempts to connect to remote logging server(s) |
|
nsdLoggerLostInSndBuf |
Number of bytes in TCP send buffer when remote closes the socket |
|
nsdLoggerDiskQueueLostBytes |
Number of bytes Lost in Disk Queue file |
|
nsdLoggerDiskQueueLostFiles |
Number of files Lost in Disk Queue |
To acquire this list of available MiBs for the Netsweeper Policy Service, including the Logger, execute the following command:
snmpwalk -c public -v2c localhost:3161 NSD-MIB::nsd
Outgoing Logging Packets
As outlined in the “Inputs” section of this document, a logging service can be configured to log remotely, meaning that it will send logging packets to another logging service on a separate machine. This is configured as part of the LogMod5 framework.
For further information on configuring LogMod5, please see ‘Configuration - LogMod5’.
Monitoring Logging
SNMP MiBs
SNMP MiBs are available for you to monitor the current state of the logging service.
For specific MiBs, please see the Outputs section of this document.
Monitoring Graphs
Within the Netsweeper WebAdmin interface are the monitoring graphs, allowing a visual view of SNMP data that is generated by the Netsweeper systems in your deployment. These can be accessed through Monitoring > Monitoring Graphs.
The available SNMP MIBs that can be monitored using the monitoring graphs are listed above, in the SNMP MIBs section.
Services Page
The ‘Services’ page allows for a quick, at-a-glance, view of the services running on your Netsweeper deployment. You can reach this page by browsing to Administration > Services.
Status Page
The nsd.conf (policy service configuration file) contains settings for remote administration and is used for authentication and monitoring of system status in the WebAdmin. You must list the servers by IP address or subnets that can connect. If you add the IP address of 0.0.0.0 all servers will be allowed to connect.
In WebAdmin, to view the status and statistics for the remote systems and services, go to Administration> Status and click on the Services tab.
