NXLog User Guide

71. Microsoft Exchange

Microsoft Exchange is a widely used enterprise level email server running on Windows Server operating systems. The following sections describe various logs generated by Exchange and provide solutions for collecting logs from these sources with NXLog.

Exchange stores most of its operational logs in a comma-delimited format similar to W3C. These files can be read with im_file and the xm_w3c extension module. For NXLog Community Edition, the xm_csv extension module can be used instead, with the fields listed explicitly and the header lines skipped. In some of the log files, the W3C header is prepended by an additional CSV header line enumerating the same fields as the #Fields directive; NXLog must be configured to skip that line also. See the sections under Transport Logs for examples.

The information provided here is not intended to be comprehensive, but rather provides a general overview of NXLog integration with some of the major log mechanisms used by Exchange. Other logs generated by Exchange can be found in the Logging and other subdirectories of the installation directory.

Note	This Guide focuses on Exchange Server 2010 SP1 and later versions. Older versions are either not supported by Microsoft or are being decomissioned. Apart from passing their end of life date, these versions also lack the audit logging feature.

71.1. Transport Logs

Exchange Server writes various transport logs. Three of those logs are covered in the following sections. For more information about additional Exchange transport logs, see the Transport logs in Exchange 2016 TechNet article.

71.1.1. Configuring Transport Logs

Message tracking, connectivity, and protocol logs are enabled by default and written to comma-delimited log files, in a format similar to W3C. The logs can be enabled or disabled, and the log file locations modified, through the Exchange Admin Center (EAC).

Log in to the Exchange Admin Center (at https://server/ecp).
Click servers in the list on the left.
Select the server and click the Edit icon.
Click transport logs in the list on the left.
Modify the logging configuration as required, then click Save.

71.1.2. Message Tracking Logs

Message tracking logs provide a detailed record of message activity as mail flows through the transport pipeline on an Exchange server.

Log Sample

#Software: Microsoft Exchange Server
#Version: 15.01.1034.026
#Log-type: Message Tracking Log
#Date: 2017-09-15T20:01:45.863Z
#Fields: date-time,client-ip,client-hostname,server-ip,server-hostname,source-context,connector-id,source,event-id,internal-message-id,message-id,network-message-id,recipient-address,recipient-status,total-bytes,recipient-count,related-recipient-address,reference,message-subject,sender-address,return-path,message-info,directionality,tenant-id,original-client-ip,original-server-ip,custom-data,transport-traffic-type,log-id,schema-version
2017-09-15T20:01:45.863Z,,,,WINEXC,No suitable shadow servers,,SMTP,HAREDIRECTFAIL,34359738369,<49b4b9a2781a45cba555008075f7bffa@test.com>,8e1061b7-a376-497c-3172-08d4fc7497bf,test1@test.com,,6533,1,,,test,Administrator@test.com,Administrator@test.com,,Originating,,,,S:DeliveryPriority=Normal;S:AccountForest=test.com,Email,63dc9d79-5b4e-4f6c-1358-08d4fc7497c3,15.01.1034.026

NXLog can be configured to collect these logs with the im_file module, and to parse them with xm_w3c.

Example 309. Collecting Message Tracking Logs With xm_w3c

This configuration collects message tracking logs from the defined BASEDIR and parses them using the xm_w3c module. The logs are then converted to JSON format and forwarded via TCP.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

  define BASEDIR C:\Program Files\Microsoft\Exchange Server\V15

<Extension _json>
    Module      xm_json
</Extension>

<Extension w3c_parser>
    Module      xm_w3c
    Delimiter   ,
</Extension>

<Input messagetracking>
    Module      im_file
    File        '%BASEDIR%\TransportRoles\Logs\MessageTracking\MSGTRK*.LOG'
    InputType   w3c_parser
</Input>

<Output tcp>
    Module      om_tcp
    Host        10.0.0.1
    Port        1514
    Exec        to_json();
</Output>

For NXLog Community Edition, the xm_csv module can be configured to parse these files.

Example 310. Using xm_csv for Message Tracking Logs

This configuration uses the xm_csv module to parse the message tracking logs.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

  define BASEDIR C:\Program Files\Microsoft\Exchange Server\V15

<Extension csv_parser>
    Module      xm_csv
    Fields      date-time, client-ip, client-hostname, server-ip, server-hostname, \
                source-context, connector-id, source, event-id, \
                internal-message-id, message-id, network-message-id, \
                recipient-address, recipient-status, total-bytes, recipient-count, \
                related-recipient-address, reference, message-subject, \
                sender-address, return-path, message-info, directionality, \
                tenant-id, original-client-ip, original-server-ip, custom-data, \
                transport-traffic-type, log-id, schema-version
</Extension>

<Input messagetracking>
    Module      im_file
    File        '%BASEDIR%\TransportRoles\Logs\MessageTracking\MSGTRK*.LOG'
    <Exec>
        if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop();
        else
        {
            csv_parser->parse_csv();
            $EventTime = parsedate(${date-time});
        }
    </Exec>
</Input>

71.1.3. Connectivity Logs

Connectivity logging records outbound message transmission activity by the transport services on the Exchange server.

Log Sample

#Software: Microsoft Exchange Server
#Version: 15.0.0.0
#Log-type: Transport Connectivity Log
#Date: 2017-09-15T03:09:34.541Z
#Fields: date-time,session,source,Destination,direction,description
2017-09-15T03:09:33.526Z,,Transport,,*,service started; #MaxConcurrentSubmissions=20; MaxConcurrentDeliveries=20; MaxSmtpOutConnections=Unlimited

NXLog can be configured to collect these logs with the im_file module, and to parse them with xm_w3c.

Example 311. Collecting Connectivity Logs With xm_w3c

This configuration collects connectivity logs from the defined BASEDIR and parses them using the xm_w3c module.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12

  define BASEDIR C:\Program Files\Microsoft\Exchange Server\V15

<Extension w3c_parser>
    Module      xm_w3c
    Delimiter   ,
</Extension>

<Input connectivity>
    Module      im_file
    File        '%BASEDIR%\TransportRoles\Logs\Hub\Connectivity\CONNECTLOG*.LOG'
    InputType   w3c_parser
</Input>

For NXLog Community Edition, the xm_csv module can be configured to parse these files.

Example 312. Using xm_csv for Connectivity Logs

This configuration uses the xm_csv module to parse the connectivity logs.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

  define BASEDIR C:\Program Files\Microsoft\Exchange Server\V15

<Extension csv_parser>
    Module      xm_csv
    Fields      date-time, session, source, Destination, direction, description
</Extension>

<Input connectivity>
    Module      im_file
    File        '%BASEDIR%\TransportRoles\Logs\Hub\Connectivity\CONNECTLOG*.LOG'
    <Exec>
        if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop();
        else
        {
            csv_parser->parse_csv();
            $EventTime = parsedate(${date-time});
        }
    </Exec>
</Input>

71.1.4. Protocol/SMTP Logs

Protocol logging records the SMTP conversations that occur on Send and Receive connectors during message delivery.

Log Sample

#Software: Microsoft Exchange Server
#Version: 15.0.0.0
#Log-type: SMTP Send Protocol Log
#Date: 2017-09-20T21:00:47.866Z
#Fields: date-time,connector-id,session-id,sequence-number,local-endpoint,remote-endpoint,event,data,context
2017-09-20T21:00:47.167Z,internet,08D5006A392BE443,0,,64.8.70.48:25,*,,attempting to connect

NXLog can be configured to collect these logs with the im_file module, and to parse them with xm_w3c.

Example 313. Collecting Protocol Logs With xm_w3c

This configuration collects protocol logs from the defined BASEDIR and parses them using the xm_w3c module.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

  define BASEDIR C:\Program Files\Microsoft\Exchange Server\V15

<Extension w3c_parser>
    Module      xm_w3c
    Delimiter   ,
</Extension>

<Input smtp_receive>
    Module     im_file
    File       '%BASEDIR%\TransportRoles\Logs\Hub\ProtocolLog\SmtpReceive\RECV*.LOG'
    InputType  w3c_parser
</Input>

<Input smtp_send>
    Module      im_file
    File        '%BASEDIR%\TransportRoles\Logs\Hub\ProtocolLog\SmtpSend\SEND*.LOG'
    InputType   w3c_parser
</Input>

For NXLog Community Edition, the xm_csv module can be configured to parse these files.

Example 314. Using xm_csv for Protocol Logs

This configuration uses the xm_csv module to parse the protocol logs.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

  define BASEDIR C:\Program Files\Microsoft\Exchange Server\V15

<Extension csv_parser>
    Module      xm_csv
    Fields      date-time, connector-id, session-id, sequence-number, \
                local-endpoint, remote-endpoint, event, data, context
</Extension>

<Input smtp_receive>
    Module  im_file
    File    '%BASEDIR%\TransportRoles\Logs\Hub\ProtocolLog\SmtpReceive\RECV*.LOG'
    <Exec>
        if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop();
        else
        {
            csv_parser->parse_csv();
            $EventTime = parsedate(${date-time});
        }
    </Exec>
</Input>

<Input smtp_send>
    Module      im_file
    File        '%BASEDIR%\TransportRoles\Logs\Hub\ProtocolLog\SmtpSend\SEND*.LOG'
    <Exec>
        if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop();
        else
        {
            csv_parser->parse_csv();
            $EventTime = parsedate(${date-time});
        }
    </Exec>
</Input>

71.2. EventLog

Exchange Server also logs events to Windows EventLog. Events are logged to the Application and Systems channels, as well as multiple Exchange-specific crimson channels (see your server’s Event Viewer). For more information about events generated by Exchange, see the following TechNet articles.

See also Windows Event Log for more information about using NXLog to collect logs from Windows EventLog.

Example 315. Collecting Exchange Events From the EventLog

With this configuration, NXLog will use the im_msvistalog module to subscribe to the Application and System channels (Critical, Error, and Warning event levels only) and the MSExchange Management crimson channel (all event levels). Note that the Application and System channels will include other non-Exchange events.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14

  <Input eventlog>
    Module  im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0" Path="Application">
                <Select Path="Application">
                    *[System[(Level=1 or Level=2 or Level=3)]]</Select>
                <Select Path="System">
                    *[System[(Level=1 or Level=2 or Level=3)]]</Select>
                <Select Path="MSExchange Management">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>

71.3. IIS Logs

Exchange is closely integrated with the Internet Information Server (IIS), which itself logs Outlook Web Access (OWA) and Exchange Admin Center (EAC) events.

See the Microsoft IIS chapter for more information about collecting events from IIS with NXLog.

71.4. Audit Logs (nxlog-xchg)

Exchange also provides two types of audit logs: administrator audit logs and mailbox audit logs. For more information, see Administrator audit logging in Exchange 2016 and Mailbox audit logging in Exchange 2016 on TechNet.

The nxlog-xchg utility can be used to retrieve these logs. See the Exchange (nxlog-xchg) add-on documentation.