NXLog User Guide

103. Debugging Data Processing

This section explains how to resolve issues with complex processing rules and configurations while using NXLog.

The examples in this section show how the event log stream can be debugged. Using these methods, you can see:

the data that has been received or read by the inputs;
whether a required field exists, and what its value is;
whether the parser is working correctly, and whether it is populating the fields correctly; and
the contents of all fields, and their values after NXLog parses them.

Note	A remote device may be sending invalid data to NXLog. To troubleshoot in this case, use a network traffic analyzer such as Wireshark or tcpdump.

Example 428. Writing Fields and Values to an External File

In this example configuration, the file_write() procedure (from the xm_fileop module) is used to dump information to an external file.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

  <Extension _fileop>
    Module  xm_fileop
</Extension>

<Extension _syslog>
    Module  xm_syslog
</Extension>

<Input in>
    Module  im_tcp
    Host    0.0.0.0
    Port    1514
    <Exec>
        parse_syslog_bsd();

        # Debug $SyslogSeverity and $Hostname fields
        file_write("/tmp/debug.txt",
                   "Severity: " + $SyslogSeverity +
                   ", Hostname: " + $Hostname + "\n");
    </Exec>
</Input>

Example 429. Writing Specific Fields and Values to the Internal Log

This configuration uses the log_info() procedure to send values to the internal log.

The generated messages will be visible:

in the file defined in the LogFile global directive,
in the input from the im_internal module, and
on standard output when running NXLog in the foreground with the -f command line switch.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

  <Extension _syslog>
    Module  xm_syslog
</Extension>

<Input in>
    Module  im_tcp
    Host    0.0.0.0
    Port    1514
    <Exec>
        parse_syslog_bsd();

        # Debug $SyslogSeverity and $Hostname fields
        log_info("Severity: " + $SyslogSeverity + ", Hostname: " + $Hostname);
    </Exec>
</Input>

Example 430. Dumping All Fields to the Internal Log

In this example, the to_json() procedure (from the xm_json module) is used to dump all the fields to the internal log.

Note	Alternatively, the to_xml() procedure (from the xm_xml module) could be used.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

  <Extension _syslog>
    Module  xm_syslog
</Extension>

<Extension _json>
    Module  xm_json
</Extension>

<Input in>
    Module  im_tcp
    Host    0.0.0.0
    Port    1514
    <Exec>
        parse_syslog_bsd();

        # Dump $raw_event
        log_info("raw_event is: " + $raw_event);

        # Dump fields in JSON
        log_info("Other fields are: " + to_json());
    </Exec>
</Input>

Output Sample

2012-05-18 13:11:35 INFO raw_event is: <27>2010-10-12 12:49:06 host app[12345]: test message
2012-05-18 13:11:35 INFO Other fields are: {"MessageSourceAddress":"127.0.0.1","EventReceivedTime":"2012-05-18 13:11:35","SourceModuleName":"in","SourceModuleType":"im_tcp","SyslogFacilityValue":3,"SyslogFacility":"DAEMON","SyslogSeverityValue":3,"SyslogSeverity":"ERR","SeverityValue":4,"Severity":"ERROR","Hostname":"host","EventTime":"2010-10-12 12:49:06","SourceName":"app","ProcessID":"12345","Message":"test message"}