Return to
Portfolio

64. IBM QRadar SIEM

IBM QRadar Security Information and Event Management (SIEM) collects event data and uses analytics, correlation, and threat intelligence features to identify known or potential threats, provide alerting and reports, and aid in incident investigations. For more information, see IBM QRadar SIEM on IBM.com.

NXLog can be configured to collect events and forward them to QRadar SIEM. This chapter provides information about setting up this integration, both for generic structured logs and for several specific log types. The last section shows output examples for forwarding the processed logs to QRadar.

Note
 The instructions and examples in this chapter were tested with QRadar 7.3.1.

64.1. Setting up the QRadar Appliance

Several tasks may be required to prepare IBM QRadar for receiving events from NXLog.

64.1.1. QRadar Dependencies and System Configuration

64.1.2. Adding a TLS Syslog Log Source

Events can be sent to QRadar securely with TLS. With these instructions, the NXLog agent(s) will verify the authenticity of the QRadar receiver and encrypt event data in transit. This requires that appropriate certificates be created and a separate TLS Syslog "listener" log source be added on QRadar.

This log source will act as a gateway, passing each event on to another matching log source. Only one TLS listener is required per port; see Configuring multiple log sources over TLS syslog on IBM Knowledge Center.

First, prepare the TLS certificate and key files (for more information, see OpenSSL Certificate Creation):

  1. Locate a certificate authority (CA) certificate and private key, or generate and sign a new one. The CA certificate (for example, rootCA.pem) will be used by the NXLog agent to authenticate the QRadar receiver in Forwarding Logs below.

  2. Create a certificate and private key for QRadar TLS Syslog (for example, server.crt and server.key).

  3. Convert the QRadar private key to a DER-encoded PKCS8 key (see QRadar: TLS Syslog support of DER-encoded PKCS8 custom certificates):

    $ openssl pkcs8 -topk8 -inform PEM -outform DER -in server.key \
      -out server.key.der -nocrypt

  4. Copy the private key and certificate files to QRadar (the steps below assume the files are copied to /root/server.*).

Then add the log source on QRadar:

  1. In the QRadar web interface, go to Menu > Admin > Data Sources > Events > Log Sources.

    1 log sources icon

  2. Click Add to add a new log source. The Add a log source window appears.

    tls 1 add log source

  3. Enter a Log Source Name and, optionally, a Log Source Description.

  4. For the Log Source Type, select Universal DSM.

  5. For the Protocol Configuration, select TLS Syslog.

  6. As the Log Source Identifier, enter the source device IP address or hostname. For multiple log sources, any identifier can be used here.

  7. For Certificate Type, select Provide Certificate.

  8. Set Provided Server Certificate Path to the path of the server certificate (for example, /root/server.crt).

  9. Set Provided Private Key Path to the path of the DER-encoded server key (for example, /root/server.key.der).

  10. Select the Target Event Collector. Use this to poll for and process events using the specified event collector, rather than on the Console appliance.

  11. Make any other changes required, and then click Save.

  12. Go to Menu > Admin and click Advanced > Deploy Full Configuration after making all required log source changes.

64.1.3. Adding a QRadar Log Source

Follow these steps to add a new log source to QRadar SIEM. This will need to be done once for each log source, using the correct Log Source Type for each.

  1. In the QRadar web interface, go to Menu > Admin > Data Sources > Events > Log Sources.

    1 log sources icon

  2. Click Add to add a new log source. The Add a log source window appears.

    2 add log source

  3. Enter a Log Source Name and, optionally, a Log Source Description.

  4. Select a Log Source Type. Consult the sections below for the correct log type to use for each source.

  5. For the Protocol Configuration, select Syslog.

  6. As the Log Source Identifier, enter the source system’s IP address.

    Note
    		 The Syslog hostname field is used by QRadar as the log source identifier to associate events with a particular log source when received. This value can be adjusted by changing the $Hostname = host_ip(); line in the examples below: keep the line as-is to use the system’s first non-loopback IP address, remove the line to use the system hostname, or set the line to a custom value (for example, $Hostname = "myhostname";).

  7. Select the Target Event Collector. Use this to poll for and process events using the specified event collector, rather than on the Console appliance.

  8. Make any other changes required, and then click Save.

  9. Go to Menu > Admin and click Advanced > Deploy Full Configuration after making all required log source changes.

64.2. Sending Generic Structured Logs to QRadar

NXLog can be configured to send generic structured logs to QRadar using Log Event Extended Format (LEEF). The xm_leef to_leef() procedure will generate LEEF events using certain NXLog fields for the event header and all remaining fields as event attributes.

LEEF has several predefined event attributes that should be used where applicable—see LEEF event components and Predefined LEEF event attributes on IBM Knowledge Center. These fields can be set during parsing, set to static values manually ($usrName = "john";), renamed using the rename_field() directive, or renamed using the xm_rewrite Rename directive (NXLog Enterprise Edition only). Additionally, to_leef() will set several predefined attributes automatically.

Use Universal LEEF as QRadar’s Log Source Type. Once LEEF events have been received by QRadar, specific fields can be selected for extraction as described in Writing an expression for structured data in LEEF format (in the QRadar Security Intelligence Platform documentation). LEEF events can also be mapped to QRadar Identifiers (QIDs). For more information, see the Universal LEEF section in the QRadar DSM Guide.

Example 272. Sending LEEF Logs to QRadar

This example reads Syslog messages from file, parses them, and sets some additional fields. Then the xm_leef to_leef() procedure is used to convert the event to LEEF (and write it to the $raw_event field). Because the event is converted in the scope of this input instance, it is not necessary to do additional processing in the corresponding output instance—see Forwarding Logs for output examples that could be used to send the events to QRadar.

Note
 This example is intended as a starting point for a configuration that provides a specific set of fields to QRadar. For logs that are already structured, it may only be necessary to rename a few fields according to the predefined LEEF attribute names.
Input Sample (auth.log)
Jul 31 07:17:01 debian CRON[968]: pam_unix(cron:session): session opened for user root by (uid=0)
Aug 11 22:43:26 debian sshd[5584]: Invalid user baduser from 10.80.0.1 port 33122
nxlog.conf [Download file]

  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42


  
<Extension _leef>
    Module  xm_leef
</Extension>

<Extension _syslog>
    Module  xm_syslog
</Extension>

<Input auth>
    Module  im_file
    File    '/var/log/auth.log'
    <Exec>
        # Parse Syslog event and set fields in the event record
        parse_syslog();

        # Set event category and event ID (for QID mapping)
        if $Message =~ /^Invalid/
        {
            $Category = "Failed";
            $EventID = "Logon Failure";
        }
        else
        {
            $Category = "Success";
            $EventID = "Logon Success";
        }

        # Extract user name for "usrName" event attribute
        $Message =~ /user (?<usrName>\S+)/;

        # Set $AccountType based on whether the message mentions the root user;
        # this is mapped to the "role" event attribute
        if $usrName == "root" $AccountType = "Administrator";
        else $AccountType = "User";

        # Use the first non-loopback IP address as the log source identifier
        $Hostname = host_ip();

        # Convert to LEEF
        to_leef();
    </Exec>
</Input>
Output Sample
<13>Jul 31 07:17:01 10.80.1.49 CRON[968]: LEEF:1.0|NXLog|CRON|4.4.4347|Logon Success|EventReceivedTime=2019-08-11 22:48:59⇥SourceModuleName=file⇥SourceModuleType=im_file⇥SyslogFacilityValue=1⇥SyslogFacility=USER⇥SyslogSeverityValue=5⇥SyslogSeverity=NOTICE⇥sev=2⇥Severity=INFO⇥identHostName=debian⇥devTime=2019-07-31 07:17:01⇥vSrcName=CRON⇥ProcessID=968⇥Message=pam_unix(cron:session): session opened for user root by (uid=0)⇥cat=Success⇥EventID=Logon Success⇥usrName=root⇥role=Administrator⇥devTimeFormat=yyyy-MM-dd HH:mm:ss
<13>Aug 11 22:43:26 10.80.1.49 sshd[5584]: LEEF:1.0|NXLog|sshd|4.4.4347|Logon Failure|EventReceivedTime=2019-08-11 22:48:59⇥SourceModuleName=file⇥SourceModuleType=im_file⇥SyslogFacilityValue=1⇥SyslogFacility=USER⇥SyslogSeverityValue=5⇥SyslogSeverity=NOTICE⇥sev=2⇥Severity=INFO⇥identHostName=debian⇥devTime=2019-08-11 22:43:26⇥vSrcName=sshd⇥ProcessID=5584⇥Message=Invalid user baduser from 10.80.0.1 port 33122⇥cat=Failed⇥EventID=Logon Failure⇥usrName=baduser⇥role=User⇥devTimeFormat=yyyy-MM-dd HH:mm:ss

64.3. Sending Specific Log Types for QRadar to Parse

To take full advantage of QRadar’s parsing of specific log types, NXLog can be configured to send logs using the specific format expected by the corresponding QRadar DSM. In each case, events are collected, parsed, and converted to a tab-delimited key-value pair format that QRadar expects.

64.3.1. DHCP Server

To send DHCP Server audit log events to QRadar SIEM, set up DHCP Audit Logging and use the NXLog configuration shown below. If QRadar does not auto-discover the log source, add one manually. The Log Source Type should be set to Microsoft DHCP Server and the Protocol Configuration should be set to Syslog—see Adding a QRadar Log Source.

For more information, see DHCP Server Audit Logging and the Microsoft DHCP Server page in the QRadar DSM Guide.

Example 273. Sending Windows DHCP Events to QRadar

In this example, NXLog is configured to read logs from the following paths:

  • C:\Windows\System32\dhcp\DhcpSrvLog-*.log

  • C:\Windows\System32\dhcp\DhcpV6SrvLog-*.log

NXLog parses the events and converts the structured data for forwarding to QRadar.

Input Sample (DhcpSrvLog)
13,07/31/19,07:18:29,Conflict,10.80.2.1,BAD_ADDRESS,,,0,6,,,,,,,,,0
Input Sample (DhcpV6SrvLog)
11004,07/31/19,07:32:34,DHCPV6 Renew,2001:db8::667a:1521:96ab:5f50,QRADARWIN.nxlog.org,,14,00010001244AC14F5254005DF4CC,,,,,
nxlog.conf [Download file]

  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81


  
<Extension _syslog>
    Module  xm_syslog
</Extension>

<Extension dhcp_csv_parser>
    Module  xm_csv
    Fields  ID, Date, Time, Description, IPAddress, LogHostname, MACAddress, \
            UserName, TransactionID, QResult, ProbationTime, CorrelationID, \
            DHCID, VendorClassHex, VendorClassASCII, UserClassHex, \
            UserClassASCII, RelayAgentInformation, DnsRegError
</Extension>

<Extension dhcpv6_csv_parser>
    Module  xm_csv
    Fields  ID, Date, Time, Description, IPAddress, LogHostname, MACAddress, \
            UserName, TransactionID, QResult, ProbationTime, CorrelationID, \
            DHCID, VendorClassHex
</Extension>

<Input dhcp>
    Module  im_file
    File    'C:\Windows\System32\dhcp\DhcpSrvLog-*.log'
    File    'C:\Windows\System32\dhcp\DhcpV6SrvLog-*.log'
    <Exec>
        # Only process lines that begin with an event ID
        if $raw_event =~ /^\d+,/
        {
            if file_name() =~ /^.*\\(.*)$/ $FileName = $1;
            if $FileName =~ /DhcpSrvLog-/
            {
                dhcp_csv_parser->parse_csv();
                $Message = "AgentDevice=WindowsDHCP" +
                    "\tAgentLogFile=" + $FileName +
                    "\tID=" + $ID +
                    "\tDate=" + $Date +
                    "\tTime=" + $Time +
                    "\tDescription=" + $Description +
                    "\tIP Address=" + $IPAddress +
                    "\tHost Name=" + $LogHostname +
                    "\tMAC Address=" + $MACAddress +
                    "\tUser Name=" + $UserName +
                    "\tTransactionID=" + $TransactionID +
                    "\tQResult=" + $QResult +
                    "\tProbationtime=" + $ProbationTime +
                    "\tCorrelationID=" + $CorrelationID +
                    "\tDhcid=" + $DHCID +
                    "\tVendorClass(Hex)=" + $VendorClassHex +
                    "\tVendorClass(ASCII)=" + $VendorClassASCII +
                    "\tUserClass(Hex)=" + $UserClassHex +
                    "\tUserClass(ASCII)=" + $UserClassASCII +
                    "\tRelayAgentInformation=" + $RelayAgentInformation +
                    "\tDnsRegError=" + $DnsRegError;
            }
            else
            {
                dhcpv6_csv_parser->parse_csv();
                $Message = "AgentDevice=WindowsDHCP" +
                    "\tAgentLogFile=" + $FileName +
                    "\tID=" + $ID +
                    "\tDate=" + $Date +
                    "\tTime=" + $Time +
                    "\tDescription=" + $Description +
                    "\tIP Address=" + $IPAddress +
                    "\tHost Name=" + $LogHostname +
                    "\tMAC Address=" + $MACAddress +
                    "\tUser Name=" + $UserName +
                    "\tTransactionID=" + $TransactionID +
                    "\tQResult=" + $QResult +
                    "\tProbationtime=" + $ProbationTime +
                    "\tCorrelationID=" + $CorrelationID +
                    "\tDhcid=" + $DHCID +
                    "\tVendorClass(Hex)=" + $VendorClassHex;
            }
            $EventTime = strptime($Date + ' ' + $Time, '%m/%d/%y %H:%M:%S');
            $Hostname = host_ip();
            to_syslog_bsd();
        }
        # Discard header lines (which do not begin with an event ID)
        else drop();
    </Exec>
</Input>
Output Sample (DhcpSrvLog)
<13>Jul 31 07:18:29 10.80.1.49 AgentDevice=WindowsDHCP⇥AgentLogFile=DhcpSrvLog-Wed.log⇥ID=13⇥Date=07/31/19⇥Time=07:18:29⇥Description=Conflict⇥IP Address=10.80.2.1⇥Host Name=BAD_ADDRESS⇥MAC Address=⇥User Name=⇥TransactionID=0⇥QResult=6⇥Probationtime=⇥CorrelationID=⇥Dhcid=⇥VendorClass(Hex)=⇥VendorClass(ASCII)=⇥UserClass(Hex)=⇥UserClass(ASCII)=⇥RelayAgentInformation=⇥DnsRegError=0

64.3.2. DNS Debug Log

To send DNS debug log events to QRadar, enable debug logging and use the NXLog configuration shown below.

Warning
 Do not enable Details in the DNS Server Debug Logging dialog.

If QRadar does not auto-discover the log source, add one manually. The Log Source Type should be set to Microsoft DNS Debug and the Protocol Configuration should be set to Syslog—see Adding a QRadar Log Source. If the Microsoft DNS Debug log source type is not available, see Setting up the QRadar Appliance above.

For more information, see Windows DNS Server and the Microsoft DNS Debug page in the QRadar DSM Guide.

Example 274. Sending DNS Debug Logs to QRadar

This configuration uses the xm_msdns extension module to parse the Windows DNS debug log.

nxlog.conf [Download file]

  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69


  
<Extension _syslog>
    Module  xm_syslog
</Extension>

<Extension dns_parser>
    Module  xm_msdns
</Extension>

<Input dns>
    Module      im_file
    File        'C:\logs\dns.log'
    InputType   dns_parser
    <Exec>
        $raw_event =~ /(?x)^(?<Date>\d+\/\d+\/\d+)\s(?<Time>\d+:\d+:\d+\s+\w{2})/;
        if file_name() =~ /^.*\\(.*)$/ $FileName = $1;
        $Message = "AgentDevice=WindowsDNS" +
            "\tAgentLogFile=" + $FileName +
            "\tDate=" + $Date +
            "\tTime=" + $Time +
            "\tThread ID=" + $ThreadID;
        if $Context == "EVENT"
        {
            $EventDescription =~ s/,//g;
            $Message = $Message +
                "\tContext=EVENT" +
                "\tMessage=" + $EventDescription;
        }
        else if $Context == "Note"
        {
            $Note =~ s/^Note: //;
            $Note =~ s/, / /g;
            $Message = $Message +
                "\tContext=NOTE" +
                "\tMessage=" + $Note;
        }
        else
        # $Context == "PACKET"
        {
            $FlagsChar = "";
            if $AuthoritativeAnswer $FlagsChar = "A";
            if $TruncatedResponse $FlagsChar = $FlagsChar + "T";
            if $RecursionDesired $FlagsChar = $FlagsChar + "D";
            if $RecursionAvailable $FlagsChar = $FlagsChar + "R";
            if $QueryResponseIndicator == "Query" $QueryResponse = "Q";
            else if $QueryResponseIndicator == "Response" $QueryResponse = "R";
            if $Opcode == "Standard Query" $OpcodeStr = "Q";
            else if $Opcode == "Notify" $OpcodeStr = "N";
            else if $Opcode == "Update" $OpcodeStr = "U";
            else if $Opcode == "Unknown" $OpcodeStr = "?";
            $Message = $Message +
                "\tContext=PACKET" +
                "\tMessage=" +
                "\tInternal packet identifier=" + $InternalPacketIdentifier +
                "\tUDP/TCP indicator=" + $Protocol +
                "\tSend/Receive indicator=" + $SendReceiveIndicator +
                "\tRemote IP=" + $RemoteIP +
                "\tXid (hex)=" + $Xid +
                "\tQuery/Response=" + $QueryResponse +
                "\tOpcode=" + $OpcodeStr +
                "\tFlags (hex)=" + $FlagsHex +
                "\tFlags (char codes)=" + $FlagsChar +
                "\tResponseCode=" + $ResponseCode +
                "\tQuestion Type=" + $QuestionType +
                "\tQuestion Name=" + $QuestionName;
        }
        $Hostname = host_ip();
        to_syslog_bsd();
    </Exec>
</Input>
Output Sample
<13>Jul 20 08:42:07 10.80.1.49 AgentDevice=WindowsDNS⇥AgentLogFile=debug.log⇥Date=7/20/2019⇥Time=8:42:07 AM⇥Thread ID=0710⇥Context=EVENT⇥Message=The DNS server has finished the background loading of zones. All zones are now available for DNS updates and zone transfers as allowed by their individual zone configuration.

64.3.3. Microsoft Exchange Server

Microsoft Exchange Server logs can be collected and sent to QRadar SIEM as shown below.

QRadar does not support auto-discovery for Exchange Server logs, so it is necessary to add a log source manually. The Log Source Type should be set to Microsoft Exchange Server and the Protocol Configuration should be set to Syslog—see Adding a QRadar Log Source.

For more information, see the Microsoft Exchange chapter and the Microsoft Exchange Server pages in the QRadar DSM Guide.

Example 275. Sending Exchange Server Logs to QRadar

The following configuration uses the im_file module to read message tracking, Outlook web access (OWA), and SMTP logs from various paths. The logs are parsed and converted for forwarding to QRadar.

Note
 Make sure to use the correct ID for the Exchange Back End site. This can be verified using the Internet Information Services (IIS) Manager. The following example collects logs from the site with ID 2 (W3SVC2/).
nxlog.conf [Download file]

  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125


  
<Extension _syslog>
    Module  xm_syslog
</Extension>

<Extension w3c_parser>
    Module  xm_w3c
</Extension>

<Extension w3c_comma_parser>
    Module      xm_w3c
    Delimiter   ,
</Extension>

<Input exchange_OWA>
    Module      im_file
    File        'C:\inetpub\logs\LogFiles\W3SVC2\u_ex*.log'
    InputType   w3c_parser
    <Exec>
        if file_name() =~ /^.*\\(.*)$/ $FileName = $1;
        if ${cs-uri-query} == undef ${cs-uri-query} = "-";
        if ${cs-username} == undef ${cs-username} = "-";
        if ${cs(Referer)} == undef ${cs(Referer)} = "-";
        $Message = "AgentDevice=MicrosoftExchange" +
            "\tAgentLogFile=" + $FileName +
            "\tAgentLogFormat=W3C" +
            "\tAgentLogProtocol=OWA" +
            "\tdate=" + $date +
            "\ttime=" + $time +
            "\ts-ip=" + ${s-ip} +
            "\tcs-method=" + ${cs-method} +
            "\tcs-uri-stem=" + ${cs-uri-stem} +
            "\tcs-uri-query=" + ${cs-uri-query} +
            "\ts-port=" + ${s-port} +
            "\tcs-username=" + ${cs-username} +
            "\tc-ip=" + ${c-ip} +
            "\tcs(User-Agent)=" + ${cs(User-Agent)} +
            "\tcs(Referer)=" + ${cs(Referer)} +
            "\tsc-status=" + ${sc-status} +
            "\tsc-substatus=" + ${sc-substatus} +
            "\tsc-win32-status=" + ${sc-win32-status} +
            "\ttime-taken=" + ${time-taken};
        $EventTime = parsedate($date + " " + $time);
        $Hostname = host_ip();
        delete($SourceName);
        to_syslog_bsd();
    </Exec>
</Input>

define BASEDIR C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles

<Input exchange_MessageTracking>
    Module      im_file
    File        '%BASEDIR%\Logs\MessageTracking\MSGTRK*.LOG'
    InputType   w3c_comma_parser
    <Exec>
        if file_name() =~ /^.*\\(.*)$/ $FileName = $1;
        ${message-info} =~ s/\s+$//g;
        $Message = "AgentDevice=MicrosoftExchange" +
            "\tAgentLogFile=" + $FileName +
            "\tAgentLogFormat=MSGTRK" +
            "\tAgentLogProtocol=MT" +
            "\tdate-time=" + ${date-time} +
            "\tclient-ip=" + ${client-ip} +
            "\tclient-hostname=" + ${client-hostname} +
            "\tserver-ip=" + ${server-ip} +
            "\tserver-hostname=" + ${server-hostname} +
            "\tsource-context=" + ${source-context} +
            "\tconnector-id=" + ${connector-id} +
            "\tsource=" + $source +
            "\tevent-id=" + ${event-id} +
            "\tinternal-message-id=" + ${internal-message-id} +
            "\tmessage-id=" + ${message-id} +
            "\tnetwork-message-id=" + ${network-message-id} +
            "\trecipient-address=" + ${recipient-address} +
            "\trecipient-status=" + ${recipient-status} +
            "\ttotal-bytes=" + ${total-bytes} +
            "\trecipient-count=" + ${recipient-count} +
            "\trelated-recipient-address=" + ${related-recipient-address} +
            "\treference=" + $reference +
            "\tmessage-subject=" + ${message-subject} +
            "\tsender-address=" + ${sender-address} +
            "\treturn-path=" + ${return-path} +
            "\tmessage-info=" + ${message-info} +
            "\tdirectionality=" + $directionality +
            "\ttenant-id=" + ${tenant-id} +
            "\toriginal-client-ip=" + ${original-client-ip} +
            "\toriginal-server-ip=" + ${original-server-ip} +
            "\tcustom-data=" + ${custom-data} +
            "\ttransport-traffic-type=" + ${transport-traffic-type} +
            "\tlog-id=" + ${log-id} +
            "\tschema-version=" + ${schema-version};
        $EventTime = parsedate(${date-time});
        $Hostname = host_ip();
        delete($SourceName);
        to_syslog_bsd();
    </Exec>
</Input>

<Input exchange_SMTP>
    Module      im_file
    File        '%BASEDIR%\Logs\Hub\ProtocolLog\SmtpReceive\RECV*.LOG'
    File        '%BASEDIR%\Logs\Hub\ProtocolLog\SmtpSend\SEND*.LOG'
    InputType   w3c_comma_parser
    <Exec>
        if file_name() =~ /^.*\\(.*)$/ $FileName = $1;
        if $event == undef $event = "-";
        $Message = "AgentDevice=MicrosoftExchange" +
            "\tAgentLogFile=" + $FileName +
            "\tAgentLogFormat=SMTP" +
            "\tAgentLogProtocol=SMTP" +
            "\tdate-time=" + ${date-time} +
            "\tconnector-id=" + ${connector-id} +
            "\tsession-id=" + ${session-id} +
            "\tsequence-number=" + ${sequence-number} +
            "\tlocal-endpoint=" + ${local-endpoint} +
            "\tremote-endpoint=" + ${remote-endpoint} +
            "\tevent=" + $event +
            "\tdata=" + $data +
            "\tcontext=" + $context;
        $EventTime = parsedate(${date-time});
        $Hostname = host_ip();
        delete($SourceName);
        to_syslog_bsd();
    </Exec>
</Input>
Output Sample (SMTPReceive)
<13>Jul 27 23:35:09 10.80.1.49 AgentDevice=MicrosoftExchange⇥AgentLogFile=RECV2019072723-1.LOG⇥AgentLogFormat=SMTP⇥AgentLogProtocol=SMTP⇥date-time=2019-07-27T23:35:09.647Z⇥connector-id=QRADARWIN\Default QRADARWIN⇥session-id=08D7122B7BADF0F4⇥sequence-number=1⇥local-endpoint=10.80.1.49:2525⇥remote-endpoint=10.80.1.49:21408⇥event=>⇥data=220 QRADARWIN.nxlog.org Microsoft ESMTP MAIL Service ready at Sat, 27 Jul 2019 23:35:08 +0000⇥context=

64.3.4. Microsoft IIS

Microsoft IIS logs can be collected using the W3C Extended Log File Format. The W3C logging should be configured as described in the Configuring Microsoft IIS by using the IIS Protocol page of the QRadar DSM Guide.

Note
 For NXLog Community Edition, the xm_csv module can be used instead of xm_w3c.

If QRadar does not auto-discover the log source, add one manually. The Log Source Type should be set to Microsoft IIS and the Protocol Configuration should be set to Syslog—see Adding a QRadar Log Source.

For more information, see the Microsoft IIS chapter and the QRadar DSM Guide Microsoft IIS Server pages.

Example 276. Sending Windows IIS Events to QRadar

This configuration uses the xm_w3c extension module to parse the IIS log, and converts the events to a tab-delimited format for QRadar.

Input Sample
2019-07-24 09:21:55 127.0.0.1 POST /OWA/auth.owa &CorrelationID=<empty>;&cafeReqId=4b9353b7-e17b-4bc5-9e54-bc6b4733d6dd;&encoding=; 443 HealthMailboxa733ff32a90d44bb970f7a147fb3f328@nxlog.org 127.0.0.1 AMProbe/Local/ClientAccess - 302 0 0 10171
nxlog.conf [Download file]

  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42


  
<Extension _syslog>
    Module  xm_syslog
</Extension>

<Extension w3c_parser>
    Module  xm_w3c
</Extension>

<Input iis>
    Module      im_file
    File        'C:\inetpub\logs\LogFiles\W3SVC1\u_ex*.log'
    InputType   w3c_parser
    <Exec>
        if file_name() =~ /^.*\\(.*)$/ $FileName = $1;
        if ${cs-uri-query} == undef ${cs-uri-query} = "-";
        if ${cs-username} == undef ${cs-username} = "-";
        if ${cs(Referer)} == undef ${cs(Referer)} = "-";
        $Message = "AgentDevice=MSIIS" +
            "\tAgentLogFile=" + $FileName +
            "\tAgentLogFormat=W3C" +
            "\tAgentLogProtocol=W3C" +
            "\tdate=" + $date +
            "\ttime=" + $time +
            "\ts-ip=" + ${s-ip} +
            "\tcs-method=" + ${cs-method} +
            "\tcs-uri-stem=" + ${cs-uri-stem} +
            "\tcs-uri-query=" + ${cs-uri-query} +
            "\ts-port=" + ${s-port} +
            "\tcs-username=" + ${cs-username} +
            "\tc-ip=" + ${c-ip} +
            "\tcs(User-Agent)=" + ${cs(User-Agent)} +
            "\tcs(Referer)=" + ${cs(Referer)} +
            "\tsc-status=" + ${sc-status} +
            "\tsc-substatus=" + ${sc-substatus} +
            "\tsc-win32-status=" + ${sc-win32-status} +
            "\ttime-taken=" + ${time-taken};
        $EventTime = parsedate($date + " " + $time);
        $Hostname = host_ip();
        delete($SourceName);
        to_syslog_bsd();
    </Exec>
</Input>
Output Sample
<13>Jul 24 09:21:55 10.80.1.49 AgentDevice=MSIIS⇥AgentLogFile=u_ex190724.log⇥AgentLogFormat=W3C⇥AgentLogProtocol=W3C⇥date=2019-07-24⇥time=09:21:55⇥s-ip=127.0.0.1⇥cs-method=POST⇥cs-uri-stem=/OWA/auth.owa⇥cs-uri-query=&CorrelationID=<empty>;&cafeReqId=4b9353b7-e17b-4bc5-9e54-bc6b4733d6dd;&encoding=;⇥s-port=443⇥cs-username=HealthMailboxa733ff32a90d44bb970f7a147fb3f328@nxlog.org⇥c-ip=127.0.0.1⇥cs(User-Agent)=AMProbe/Local/ClientAccess⇥cs(Referer)=-⇥sc-status=302⇥sc-substatus=0⇥sc-win32-status=0⇥time-taken=10171

64.3.5. Microsoft SQL

Microsoft SQL logs can be collected using the xm_charconv and im_file modules.

If QRadar does not auto-discover the log source, add one manually. The Log Source Type should be set to Microsoft SQL Server and the Protocol Configuration should be set to Syslog—see Adding a QRadar Log Source.

For configuration information, see the Microsoft SQL Server section in the QRadar DSM Guide.

Example 277. Sending Microsoft SQL Logs to QRadar

This example reads and parses events from the SQL Server log file, then converts the events to a tab-delimited format for QRadar.

nxlog.conf [Download file]

  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61


  
<Extension _syslog>
    Module      xm_syslog
</Extension>

<Extension charconv>
    Module      xm_charconv
    LineReader  UTF-16LE
</Extension>

define ERRORLOG_EVENT /(?x)(?<Date>\d+-\d+-\d+)\s \
                           (?<Time>\d+:\d+:\d+.\d+)\s \
                           (?<Source>\S+)\s+ \
                           (?<Payload>.+)$/s

<Input sql>
    Module      im_file
    File        'C:\Program Files\Microsoft SQL Server\' + \
                    'MSSQL14.MSSQLSERVER\MSSQL\Log\ERRORLOG'
    InputType   charconv
    <Exec>
        # Attempt to match regular expression
        if $raw_event =~ %ERRORLOG_EVENT%
        {
            # Check if previous lines were saved
            if defined(get_var('saved'))
            {
                $tmp = $raw_event;
                $raw_event = get_var('saved');
                set_var('saved', $tmp);
                delete($tmp);

                # Process and send previous event
                $raw_event =~ %ERRORLOG_EVENT%;
                if file_name() =~ /^.*\\(.*)$/ $FileName = $1;
                $Payload =~ s/\t/ /g;
                $Payload =~ s/\s*$//;
                $Message = "AgentDevice=MSSQL" +
                    "\tAgentLogFile=" + $FileName +
                    "\tDate=" + $Date +
                    "\tTime=" + $Time +
                    "\tSource=" + $Source +
                    "\tMessage=" + $Payload;
                $EventTime = parsedate($Date + " " + $Time);
                $Hostname = host_ip();
                to_syslog_bsd();
            }
            # Save this line to module variable until the next event
            else
            {
                set_var('saved', $raw_event);
                drop();
            }
        }
        # Not the first line of the event; save to module variable
        else
        {
            set_var('saved', get_var('saved') + " " + $raw_event);
            drop();
        }
    </Exec>
</Input>
Output Sample
<13>Aug 21 22:55:36 10.80.1.49 AgentDevice=MSSQL⇥AgentLogFile=ERRORLOG⇥Date=2019-08-21⇥Time=22:55:36.23⇥Source=spid16s⇥Message=The Service Broker endpoint is in disabled or stopped state.

64.3.6. Windows EventLog

To send Windows EventLog data to QRadar, use the im_msvistalog module and convert the events to a tab-delimited key-value pair format supported by the corresponding QRadar DSM.

Note
 This format is recommended instead of Snare or Log Event Extended Format (LEEF) in order to take full advantage of the parsing provided by the QRadar DSM. Otherwise additional parsing and/or mappings would be required to translate Windows EventLog fields to QRadar fields.

If QRadar does not auto-discover the log source, add one manually. The Log Source Type should be set to Microsoft Windows Security Event Log and the Protocol Configuration should be set to Syslog—see Adding a QRadar Log Source.

For more information, see the Windows Event Log chapter and the Microsoft Windows Security Event Log section in the QRadar DSM Guide.

Example 278. Sending Windows EventLog to QRadar

This configuration will collect from the Windows EventLog using im_msvistalog, convert the $Message field to a specific tab-delimited format, and add a BSD Syslog header with xm_syslog.

Note
 This example does not filter events, but forwards all events to QRadar. Only a subset of those events will be recognized and parsed by the QRadar DSM. For more information about using EventLog queries to limit collected events, see Windows Event Log.
nxlog.conf [Download file]

  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67


  
<Extension syslog>
    Module  xm_syslog
</Extension>

<Input eventlog>
    Module  im_msvistalog
    <Exec>
        if $Category == undef $Category = 0;
        $EventTimeStr = strftime($EventTime, "YYYY-MM-DDThh:mm:ss.sUTC");
        if $EventType == 'CRITICAL'
        {
            $EventTypeNum = 1;
            $EventTypeStr = "Critical";
        }
        else if $EventType == 'ERROR'
        {
            $EventTypeNum = 2;
            $EventTypeStr = "Error";
        }
        else if $EventType == 'INFO'
        {
            $EventTypeNum = 4;
            $EventTypeStr = "Informational";
        }
        else if $EventType == 'WARNING'
        {
            $EventTypeNum = 3;
            $EventTypeStr = "Warning";
        }
        else if $EventType == 'VERBOSE'
        {
            $EventTypeNum = 5;
            $EventTypeStr = "Verbose";
        }
        else
        {
            $EventTypeNum = 0;
            $EventTypeStr = "Audit";
        }
        if $OpcodeValue == 0 $Opcode = "Info";
        if $TaskValue == 0 $TaskValue = "None";

        $Message = "AgentDevice=WindowsLog" +
            "\tAgentLogFile=" + $Channel +
            "\tSource=" + $SourceName +
            "\tComputer=" + hostname_fqdn() +
            "\tOriginatingComputer=" + $Hostname +
            "\tUser=" + $AccountName +
            "\tDomain=" + $Domain +
            "\tEventIDCode=" + $EventID +
            "\tEventType=" + $EventTypeNum +
            "\tEventCategory=" + $Category +
            "\tRecordNumber=" + $RecordNumber +
            "\tTimeGenerated=" + $EventTimeStr +
            "\tTimeWritten=" + $EventTimeStr +
            "\tLevel=" + $EventTypeStr +
            "\tKeywords=" + $Keywords +
            "\tTask=" + $TaskValue +
            "\tOpcode=" + $Opcode +
            "\tMessage=" + $Message;
        $Hostname = host_ip();
        delete($SourceName);
        delete($Severity);
        delete($SeverityValue);
        to_syslog_bsd();
    </Exec>
</Input>
Output Sample
<13>Jul 15 20:24:43 10.80.1.49 AgentDevice=WindowsLog⇥AgentLogFile=System⇥Source=Service Control Manager⇥Computer=QRW.nxlog.org⇥OriginatingComputer=10.80.1.49⇥User=⇥Domain=⇥EventIDCode=7036⇥EventType=4⇥EventCategory=0⇥RecordNumber=9830⇥TimeGenerated=2019-07-15T20:24:43.296533Z⇥TimeWritten=2019-07-15T20:24:43.296533Z⇥Level=Informational⇥Keywords=9259400833873739776⇥Task=None⇥Opcode=Info⇥Message=The WinCollect service entered the stopped state.

64.4. Forwarding Logs

Use an output instance to forward the processed logs to QRadar SIEM. The configurations shown here can be used with any of the above input instances. Because all event formatting is done in the input instances above, the output instances here do not require any Exec directives (the $raw_event field is passed without any further modification).

Example 279. Forwarding Logs via TCP

This om_tcp instance sends logs to QRadar via TCP. In this example, events are sent from the Microsoft IIS and Windows EventLog sources.

nxlog.conf [Download file]

  
1
2
3
4
5
6
7
8
9


  
<Output qradar>
    Module  om_tcp
    Host    10.0.0.2
    Port    514
</Output>

<Route r>
    Path    iis, eventlog => qradar
</Route>

Forwarding logs with TLS requires adding a TLS Syslog listener, as described in Adding a TLS Syslog Log Source above. The root certificate authority (CA) certificate, which is used to verify the authenticity of the QRadar receiver’s certificate, should be provided to om_ssl with either CADir or CAFile.

Example 280. Forwarding Logs With TLS

In this example, the om_ssl module is used to send logs to QRadar securely, with TLS encryption.

nxlog.conf [Download file]

  
1
2
3
4
5
6


  
<Output qradar>
    Module  om_ssl
    Host    10.0.0.2
    Port    6514
    CAFile  C:\Program Files\cert\rootCA.pem
</Output>

← Previous: HP ProCurve | ↑ Up: Integration | ⌂ Home: NXLog User Guide | Next: Linux Audit System →