Return to
Portfolio

57.2. Windows DNS Server

DNS logging can be set up on Windows Server by using either ETW tracing or DNS Debug Logging.

57.2.1. Using ETW Trace Providers

DNS logs can also be captured by enabling DNS Analytical Logging and using Event Tracing for Windows (ETW). For more information, see DNS Logging and Diagnostics on Microsoft TechNet.

With Analytical Logging enabled, NXLog can use the im_etw module to collect DNS logs from the Microsoft-Windows-DNSServer ETW provider. This is the preferred method for collecting logs from Windows Server versions 2012 R2 and later, but this method is not available on earlier versions of Windows Server.

Note
On Windows Server 2012 R2, this feature is provided by hotfix 2956577.
Example 238. Using im_etw

The following configuration collects DNS logs via ETW using the im_etw module.

nxlog.conf [Download file]
1
2
3
4
<Input etw>
    Module      im_etw
    Provider    Microsoft-Windows-DNSServer
</Input>
Event Sample
{
  "EventTime": "2017-03-10 09:51:03",
  "Provider": "Microsoft-Windows-DNSServer",
  "TCP": "0",
  "InterfaceIP": "10.2.0.162",
  "Source": "10.2.0.198",
  "RD": "1",
  "QNAME": "nickelfreesolutions.com.",
  "QTYPE": "1",
  "XID": "11675",
  "Port": "22416",
  "Flags": "256",
  "BufferSize": "41",
  "PacketData": "0x2D9B01000001000000000000136E69636B656C66726565736F6C7574696F6E7303636F6D0000010001",
  "EventReceivedTime": "2017-03-10 09:51:04",
  "SourceModuleName": "etw_in",
  "SourceModuleType": "im_etw"
}

57.2.2. DNS Debug Logging

DNS logging can be enabled with debug logging mode. Queries are logged one per line.

To enable DNS debug logging, perform the following actions.

  1. Open the DNS Management console (dnsmgmt.msc).

  2. Right-click on the DNS server and choose Properties from the context menu.

  3. Under the Debug Logging tab, enable Log packets for debugging.

    DNS Server Properties window
  4. Mark the check boxes corresponding to the data that should be logged.

    Note
    The Details option will produce multi-line logs. To parse this detailed format, refer to Parsing Detailed DNS Logs With Regular Expressions below.
  5. Set the File path and name to the desired log file location.

    Warning
    The Windows DNS service may not recreate the debug log file after a rollover. If you encounter this issue, be sure to use the C: drive for the debug log path. See the following post on the NXLog website: "The disappearing Windows DNS debug log".
Log Sample (Standard Debug Mode)
4/21/2017 7:52:03 AM 06B0 PACKET  00000000028657F0 UDP Snd 10.2.0.1        6590 R Q [8081   DR  NOERROR] A      (7)example(3)com(0)

See the following sections for information about parsing the logs.

57.2.2.1. Parsing Non-Detailed Logs With xm_msdns

The xm_msdns module, available in NXLog Enterprise Edition, can be used for parsing Windows Server DNS logs.

Warning
This module does not support parsing of DNS Debugging logs generated with the Details option enabled.
Note
This module has been tested under Windows Server versions 2008 R2, 2012 R2, and 2016.
Example 239. Using xm_msdns

This configuration uses the im_file and xm_msdns modules to read and parse the log file. Output is written to file in JSON format for this example.

nxlog.conf [Download file]
1
2
3
4
5
6
7
8
9
10
11
12
<Extension dns_parser>
    Module      xm_msdns
    EventLine   TRUE
    PacketLine  TRUE
    NoteLine    TRUE
</Extension>

<Input in>
    Module      im_file
    File        'C:\Server\dns.log'
    InputType   dns_parser
</Input>
Event Sample
{
  "EventTime": "2017-04-21 07:52:03",
  "ThreadId": "06B0",
  "Context": "PACKET",
  "InternalPacketIdentifier": "00000000028657F0",
  "Protocol": "UDP",
  "SendReceiveIndicator": "Snd",
  "RemoteIP": "10.2.0.1",
  "Xid": "6590",
  "QueryResponseIndicator": "Response",
  "Opcode": "Standard Query",
  "FlagsHex": "8081",
  "RecursionDesired": true,
  "RecursionAvailable": true,
  "ResponseCode": "NOERROR",
  "QuestionType": "A",
  "QuestionName": "example.com",
  "EventReceivedTime": "2017-04-21 7:52:03",
  "SourceModuleName": "in",
  "SourceModuleType": "im_file"
}
57.2.2.2. Parsing Non-Detailed Logs With Regular Expressions

While the xm_msdns module is the preferred method for parsing DNS logs, and is about three times faster, regular expressions can also be used.

Warning
This example does not parse DNS Debugging logs generated with the Details option enabled.
Note
This has been tested under Windows Server versions 2008 R2, 2012 R2, and 2016.
Example 240. Parsing DNS Logs With Regular Expressions

This example parses the DNS Debugging logs from file and then writes the output to file in JSON format.

nxlog.conf [Download file]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
define EVENT_REGEX /(?x)(?<Date>\d+(?:\/\d+){2})\s \
                    (?<Time>\d+(?:\:\d+){2}\s\w+)\s \
                    (?<ThreadId>\w+)\s+ \
                    (?<Context>\w+)\s+ \
                    (?<InternalPacketIdentifier>[[:xdigit:]]+)\s+ \
                    (?<Protocol>\w+)\s+ \
                    (?<SendReceiveIndicator>\w+)\s \
                    (?<RemoteIP>[[:xdigit:].:]+)\s+ \
                    (?<Xid>[[:xdigit:]]+)\s \
                    (?<QueryType>\s|R)\s \
                    (?<Opcode>[A-Z]|\?)\s \
                    (?<QFlags>\[(.*?)\])\s+ \
                    (?<QuestionType>\w)\s+ \
                    (?<QuestionName>.*)/
define EMPTY_EVENT_REGEX /(^$|^\s+$)/
define DOMAIN_REGEX /\(\d+\)([\w-]+)\(\d+\)([\w-]+)/
define SUBDOMAIN_REGEX /\(\d+\)([\w-]+)\(\d+\)([\w-]+)\(\d+\)(\w+)/
define NOT_STARTING_WITH_DATE_REGEX /^(?!\d+\/\d+\/\d+).+/
define QFLAGS_REGEX /(?x)(?<FlagsHex>\d+)\s+ \
                     (?<FlagsCharCodes>\s+|([A-Z]{2}|[A-Z]))\s+ \
                     (?<ResponseCode>\w+)/

<Extension _json>
    Module  xm_json
</Extension>

<Input in>
    Module  im_file
    File    'C:\Server\dns.log'
    <Exec>
        # Drop entries that have empty lines
        if $raw_event =~ %EMPTY_EVENT_REGEX% drop();
        # Drop entries not starting with date
        if $raw_event =~ %NOT_STARTING_WITH_DATE_REGEX% drop();
        # Split entries into fields & define regular entries
        if $raw_event =~ %EVENT_REGEX%
        {
            $Regular = TRUE;
            $EventTime  = parsedate($Date + " " + $Time);
            $Raw = $raw_event;
            delete($date);
            delete($time);
            if $FlagsCharCodes =~ /^\s+$/ delete($FlagsCharCodes );
            # Convert domains from (8)mydomain(1)com to mydomain.com
            if $QuestionName =~ %DOMAIN_REGEX% $QuestionName = $1 + "." + $2;
            # Convert domains from (8)sub(8)mydomain(1)com to sub.mydomain.com
            if $QuestionName =~ %SUBDOMAIN_REGEX%
                $QuestionName = $1 + "." + $2 + "." +$3;

            # Set query flags
            if $QFlags =~ %QFLAGS_REGEX% delete($QFlags);

            # Set the query type
            if $QueryType =~ %EMPTY_EVENT_REGEX% $QueryType = "query";
            else $QueryType = "response";
        }
        else
        {
            $Regular = FALSE;
            $Raw = $raw_event;
        }
    </Exec>
</Input>

<Output out>
    Module  om_file
    Exec    to_json();
    File    'C:\output-dns-traffic.json'
</Output>
Output Sample
{
  "EventReceivedTime": "2017-04-21 07:52:16",
  "SourceModuleName": "in",
  "SourceModuleType": "im_file",
  "Context": "PACKET",
  "InternalPacketIdentifier": "00000000028657F0",
  "Opcode": "Q",
  "Protocol": "UDP",
  "QueryType": "response",
  "QuestionName": "notabilus.com",
  "QuestionType": "A",
  "RemoteIP": "10.2.0.1",
  "SendReceiveIndicator": "Snd",
  "ThreadId": "06B0",
  "Xid": "6590",
  "Regular": true,
  "EventTime": "2017-04-21 07:52:03",
  "Raw": "4/21/2017 7:52:03 AM 06B0 PACKET  00000000028657F0 UDP Snd 10.2.0.1        6590 R Q [8081   DR  NOERROR] A      (9)notabilus(3)com(0)",
  "FlagsCharCodes": "DR",
  "FlagsHex": "8081",
  "ResponseCode": "NOERROR"
}
57.2.2.3. Parsing Detailed DNS Logs With Regular Expressions

Detailed DNS logging uses a multi-line format that can be parsed with xm_multiline and regular expressions.

Example 241. Parsing Multiple Line Detailed Debug DNS Logs

In this example, the xm_multiline module joins lines that belong to the same event, by using a regular expression to match the header line. Then a regular expression is used to parse the content into fields.

Input Sample
6/1/2017 8:33:36 PM 09B8 PACKET  0000022041EED460 UDP Rcv 192.168.56.1    edaa   Q [2001   D   NOERROR] A      (6)google(3)com(0)
UDP question info at 0000022041EED460
  Socket = 680
  Remote addr 192.168.56.1, port 48210
  Time Query=6941, Queued=0, Expire=0
  Buf length = 0x0fa0 (4000)
  Msg length = 0x0027 (39)
  Message:
    XID       0xedaa
    Flags     0x0120
      QR        0 (QUESTION)
      OPCODE    0 (QUERY)
      AA        0
      TC        0
      RD        1
      RA        0
      Z         0
      CD        0
      AD        1
      RCODE     0 (NOERROR)
nxlog.conf [Download file]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
define EVENT_REGEX /(?x)(?<Date>\d+(?:\/\d+){2})\s \
                   (?<Time>\d+(?:\:\d+){2}\s\w+)\s \
                   (?<ThreadId>\w+)\s+ \
                   (?<Context>\w+)\s+ \
                   (?<InternalPacketIdentifier>[[:xdigit:]]+)\s+ \
                   (?<Protocol>\w+)\s+ \
                   (?<SendReceiveIndicator>\w+)\s \
                   (?<RemoteIP>[[:xdigit:].:]+)\s+ \
                   (?<Xid>[[:xdigit:]]+)\s \
                   (?<QueryType>\s|R)\s \
                   (?<Opcode>[A-Z]|\?)\s \
                   (?<QFlags>\[(.*?)\])\s+ \
                   (?<QuestionType>\w+)\s+ \
                   (?<QuestionName>.*)\s+  \
                   (?<LogInfo>.+)\s+.+=\s \
                   (?<Socket>\d+)\s+ Remote\s+ addr\s \
                   (?<RemoteAddr>.+),\sport\s \
                   (?<PortNum>\d+)\s+Time\sQuery= \
                   (?<TimeQuery>\d+),\sQueued= \
                   (?<Queued>\d+),\sExpire= \
                   (?<Expire>\d+)\s+.+\( \
                   (?<BufLen>\d+)\)\s+.+\( \
                   (?<MsgLen>\d+)\)\s+Message:\s+ \
                   (?<Message>(?s).*)/

define HEADER_REGEX /(?x)(?<Date>\d+(?:\/\d+){2})\s \
                    (?<Time>\d+(?:\:\d+){2}\s\w+)\s \
                    (?<ThreadId>\w+)\s+ \
                    (?<Context>\w+)\s+ \
                    (?<InternalPacketIdentifier>[[:xdigit:]]+)\s+ \
                    (?<Protocol>\w+)\s+ \
                    (?<SendReceiveIndicator>\w+)\s \
                    (?<RemoteIP>[[:xdigit:].:]+)\s+ \
                    (?<Xid>[[:xdigit:]]+)\s \
                    (?<QueryType>\s|R)\s \
                    (?<Opcode>[A-Z]|\?)\s \
                    (?<QFlags>\[(.*?)\])\s+ \
                    (?<QuestionType>\w+)\s+ \
                    (?<QuestionName>.*)/

<Extension multiline>
    Module      xm_multiline
    HeaderLine  %HEADER_REGEX%
</Extension>

<Input filein>
    Module      im_file
    File        'C:\Server\dns.log'
    InputType   multiline
    <Exec>
        if $raw_event =~ %EVENT_REGEX%
        {
            $EventTime  = parsedate($Date + " " + $Time);
            delete($Date);
            delete($Time);
        }
    </Exec>
</Input>
Output Sample
{
  "EventReceivedTime": "2018-11-30T04:33:38.660127+01:00",
  "SourceModuleName": "filein",
  "SourceModuleType": "im_file",
  "BufLen": "512",
  "Context": "PACKET",
  "Expire": "0",
  "InternalPacketIdentifier": "000000D58F45A560",
  "LogInfo": "UDP response info at 000000D58F45A560",
  "Message": "XID       0x000d\r\n    Flags     0x8180\r\n      QR        1 (RESPONSE)\r\n      OPCODE    0 (QUERY)\r\n      AA        0\r\n      TC        0\r\n      RD        1\r\n      RA        1\r\n      Z         0\r\n      CD        0\r\n      AD        0\r\n      RCODE     0 (NOERROR)\r\n    QCOUNT    1\r\n    ACOUNT    1\r\n    NSCOUNT   0\r\n    ARCOUNT   0\r\n    QUESTION SECTION:\r\n    Offset = 0x000c, RR count = 0\r\n    Name      \"(6)google(3)com(0)\"\r\n      QTYPE   AAAA (28)\r\n      QCLASS  1\r\n    ANSWER SECTION:\r\n    Offset = 0x001c, RR count = 0\r\n    Name      \"[C00C](6)google(3)com(0)\"\r\n      TYPE   AAAA  (28)\r\n      CLASS  1\r\n      TTL    26\r\n      DLEN   16\r\n      DATA   2a00:1450:400d:805::200e\r\n    AUTHORITY SECTION:\r\n      empty\r\n    ADDITIONAL SECTION:\r\n      empty\r\n",
  "MsgLen": "56",
  "Opcode": "Q",
  "PortNum": "60010",
  "Protocol": "UDP",
  "QFlags": "[8081   DR  NOERROR]",
  "QueryType": "R",
  "QuestionName": "(6)google(3)com(0)",
  "QuestionType": "AAAA",
  "Queued": "0",
  "RemoteAddr": "::1",
  "RemoteIP": "::1",
  "SendReceiveIndicator": "Snd",
  "Socket": "512",
  "ThreadId": "044C",
  "TimeQuery": "12131",
  "Xid": "000d",
  "EventTime": "2018-11-30T04:32:43.000000+01:00"
}

57.2.3. Collecting DNS logs via Sysmon

Sysmon Event ID 22 DNSEvent (DNS query) is generated when a process executes a DNS query, whether the result is successful or fails, cached or not. The telemetry for this event was added for Windows 8.1 so it is not available on Windows 7 and earlier. See the Sysmon section for more information.

Warning
To collect DNS events Sysmon creates an ETW trace session and writes the data into the Windows Event Log which can then be collected with the im_msvistalog module. To avoid this performance overhead it is recommended to use the im_etw module to collect event data directly from the DNS ETW providers for greater efficiency.
Example 242. Collecting DNSEvent Logs with Sysmon

Environments that already utilize Sysmon monitoring only need to use the im_msvistalog module and add the relevant Sysmon filtering rules for DNS monitoring.

nxlog.conf [Download file]
1
2
3
4
5
6
7
8
9
10
11
12
<Input in>
    Module  im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Microsoft-Windows-Sysmon/Operational">
                    *[System[(EventID='22')]]
                </Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>
Event Sample in JSON
{
  "EventTime": "2019-10-29T15:47:43.685222+00:00",
  "Hostname": "HOST1",
  "Keywords": "9223372036854775808",
  "EventType": "INFO",
  "SeverityValue": 2,
  "Severity": "INFO",
  "EventID": 22,
  "SourceName": "Microsoft-Windows-Sysmon",
  "ProviderGuid": "{5770385F-C22A-43E0-BF4C-06F5698FFBD9}",
  "Version": 5,
  "TaskValue": 22,
  "OpcodeValue": 0,
  "RecordNumber": 9152,
  "ExecutionProcessID": 3880,
  "ExecutionThreadID": 868,
  "Channel": "Microsoft-Windows-Sysmon/Operational",
  "Domain": "NT AUTHORITY",
  "AccountName": "SYSTEM",
  "UserID": "S-1-5-18",
  "AccountType": "User",
  "Message": "Dns query:\r\nRuleName: \r\nUtcTime: 2019-10-29 15:47:43.274\r\nProcessGuid: {b3c285a4-5f1e-5db8-0000-0010c24d1d00}\r\nProcessId: 5696\r\nQueryName: example.com\r\nQueryStatus: 0\r\nQueryResults: ::ffff:93.184.216.34;\r\nImage: C:\\Windows\\System32\\PING.EXE",
  "Category": "Dns query (rule: DnsQuery)",
  "Opcode": "Info",
  "UtcTime": "2019-10-29 15:47:43.274",
  "ProcessGuid": "{b3c285a4-5f1e-5db8-0000-0010c24d1d00}",
  "ProcessId": "5696",
  "QueryName": "example.com",
  "QueryStatus": "0",
  "QueryResults": "::ffff:93.184.216.34;",
  "Image": "C:\\Windows\\System32\\PING.EXE",
  "EventReceivedTime": "2019-10-29T15:47:44.949924+00:00",
  "SourceModuleName": "in",
  "SourceModuleType": "im_msvistalog"
}