NXLog User Guide

92. Syslog

NXLog can be configured to collect or generate log entries in the various Syslog formats. This section describes the various Syslog protocols and discusses how to use them with NXLog.

92.1. BSD Syslog (RFC 3164)

The original Syslog was written for the Sendmail project in the 1980s. It was later adopted by many other applications and implemented across many operating systems, and became the standard logging system for Unix-style systems. There was no authoritative publication about Syslog until 2001, when the Internet Engineering Task Force (IETF) published informational RFC 3164, which described the "observed behavior" among implementations. Today, modern implementations follow RFC 3164, which attempts to accommodate the many older implementations; still it must be acknowledged that many undocumented variations exist among the BSD Syslog protocols as implemented by various applications and devices.

Log Sample

<30>Nov 21 11:40:27 myserver sshd[26459]: Accepted publickey for john from 192.168.1.1 port 41193 ssh2

BSD Syslog defines both the log entry format and the transport. The message format is free-form, allowing for the payload to be JSON or another structured data format.

92.1.1. BSD Syslog Format

BSD Syslog uses a simple format, comprised of three parts.

Base BSD Syslog Format

<PRI>HEADER MSG

Note	While this is the common and recommended format for a BSD Syslog message, there are no set requirements and a device may send a BSD Syslog log message containing only a free-form message, without PRI or HEADER parts.

The PRI part, or "priority", is calculated from the facility and severity codes. The facility code indicates the type of program that generated the message, and the severity code indicates the severity of the message (see the Syslog Facilities and Syslog Severities tables below). The priority code is calculated by multiplying the facility code by eight and then adding the severity code.

Note	The PRI part is not written to file by many Syslog loggers. In that case, each log entry begins with the HEADER.

The HEADER part contains two fields: TIMESTAMP and HOSTNAME. The TIMESTAMP provides the local time when the message was generated in Mmm dd hh:mm:ss format, with no year or time zone specified; the HOSTNAME is the name of the host where the message was generated.

The MSG part contains two fields: TAG and CONTENT. The TAG is the name of the program or process that generated the message, and contains only alphanumeric characters. Any other character will represent the beginning of the CONTENT field. The CONTENT field often contains the process ID enclosed by brackets ([]), a colon (:), a space, and then the actual message. In the log sample above, the MSG part begins with myserver sshd[26459]: Accepted publickey; in this case, the TAG is ssh and the CONTENT field begins with [26459]. The CONTENT field can contain only ASCII printable characters (32-126).

Fields Commonly Used in the BSD Syslog Format

<PRI>TIMESTAMP HOSTNAME TAG[PID]: MESSAGE

Table 33. Syslog Facilities
Facility Code	Description
0	kernel messages
1	user-level messages
2	mail system
3	system daemons
4	security/authorization messages
5	messages generated internally by syslogd
6	line printer subsystem
7	network news subsystem
8	UUCP subsystem
9	clock daemon
10	security/authorization messages
11	FTP daemon
12	NTP subsystem
13	log audit
14	log alert
15	scheduling daemon
16	local use 0 (local0)
17	local use 1 (local1)
18	local use 2 (local2)
19	local use 3 (local3)
20	local use 4 (local4)
21	local use 5 (local5)
22	local use 6 (local6)
23	local use 7 (local7)

Table 34. Syslog Severities
Severity Code	Description
0	Emergency: system is unusable
1	Alert: action must be taken immediately
2	Critical: critical conditions
3	Error: error conditions
4	Warning: warning conditions
5	Notice: normal but significant condition
6	Informational :informational messages
7	Debug: debug-level messages

92.1.2. BSD Syslog Transport

According to RFC 3164, the BSD Syslog protocol uses UDP as its transport layer. Each UDP packet carries a single log entry. BSD Syslog implementations often also support plain TCP and TLS transports, though these are not covered by RFC 3164.

92.1.3. Disadvantages of BSD Syslog

There are several disadvantages associated with the BSD Syslog protocol.

The transport defined by RFC 3164 uses UDP and provides no mechanism to ensure reliable delivery, integrity, or confidentiality of log messages.
Many undocumented variations exist among implementations.
The timestamp indicates neither the year nor the timezone, and does not provide precision greater than the second.
The PRI field (and therefore the facility and severity codes) are not retained by many Syslog loggers when writing to log files.
The entire length of the log entry is limited to 1024 bytes.
Only ASCII characters 32-126 are allowed, no Unicode or line breaks.

92.2. IETF Syslog (RFCs 5424-5426)

In 2009, the IETF released RFCs 5424, 5425, and 5426 as "Proposed Standard"s intended to replace the "legacy" BSD Syslog. RFC 5425 includes a timestamp with year, timezone, and fractional seconds; provides a "structured data" field for key-value pairs; and offers UTF-8 encoding. RFC 5425 defines the use of TLS transport and supports multi-line log messages. RFC 5426 describes the use of UDP transport.

Log Sample

<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] An application event log entry...

92.2.1. IETF Syslog Format

IETF Syslog uses a base format similar to that of BSD Syslog.

Base IETF Syslog Format

HEADER STRUCTURED-DATA MSG

The HEADER part contains seven fields.

PRI: message priority (same as BSD Syslog)
VERSION: Syslog format version (always "1" for RFC 5424 logs)
TIMESTAMP: derived from RFC 3339 (YYYY-MM-DDTHH:MM:SS.000000Z, or with the time zone specified)
HOSTNAME
APP-NAME: device or application that generated the message
PROCID: ID of the process that generated the message
MSGID: message type (for example, "TCPIN" for incoming TCP traffic and "TCPOUT" for outgoing)

Note	The PRI field is not written to file by many Syslog loggers. In that case, each log entry begins with the VERSION field.

The STRUCTURED-DATA part is optional. If it is omitted, then a hyphen acts as a placeholder. Otherwise, it is surrounded by brackets. It contains an ID of the block and a list of space-separated "key=value" pairs.

The MSG part is optional and contains a free-form, single-line message. If the message is encoded in UTF-8, then it may be preceded by a Unicode byte order mark (BOM).

Fields in the IETF Syslog Format

<PRI>VERSION TIMESTAMP HOSTNAME APP-NAME PROCID MSGID [SD-ID STRUCTURED-DATA] MESSAGE

92.2.2. IETF Syslog Transport

IETF Syslog can use UDP, plain TCP, or TLS transport. UDP transport is described by RFC 5426, TLS transport by RFC 5425.

RFC 5425 also documents the octet-framing method that is used for TLS transport and provides support for multi-line messages. Octet-framing can be used with plain TCP also, TLS is not required. The message length is pre-pended as in the following example which shows the raw data that is sent over TCP/TLS.

Log Sample With Octet Framing

101 <13>1 2012-01-01T17:15:52.873750+01:00 myhost - - - [NXLOG@14506 TestField="test value"] test message

In practice IETF Syslog is commonly transferred without octet-framing over TCP or TLS. In this case the newline (\n) character is used as the record separator, similarly to how BSD Syslog is transferred over TCP or TLS.

92.3. Collecting and Parsing Syslog

NXLog can be configured to collect Syslog logs by:

reading Syslog files written by another local Syslog agent,
accepting Syslog via the local /dev/log Unix domain socket, or
accepting Syslog over the network (via UDP, TCP, or TLS).

92.3.1. Reading Syslog Log Files

Configuring NXLog to read Syslog from file allows another local Syslog agent to continue its logging operations as before. Note that NXLog will likely not have access to the facility and severity codes because most Syslog loggers do not write the PRI field to log files.

Make sure NXLog has permission to read log files in /var/log. See Reading Rsyslog Log Files for more information.

Example 378. Reading Syslog From File

This configuration reads log messages from file and parses them using the parse_syslog() procedure.

nxlog.conf


  1
2
3
4
5
6
7
8
9

  <Extension _syslog>
    Module  xm_syslog
</Extension>

<Input in>
    Module  im_file
    File    '/var/log/messages'
    Exec    parse_syslog();
</Input>

Note	The parse_syslog() procedure parses the log entry as either BSD or IETF format (the parse_syslog_bsd() and parse_syslog_ietf() procedures can be used alternatively).

92.3.2. Accepting Syslog via /dev/log

Many applications support logging by sending log messages to the /dev/log Unix domain socket. It is the responsibility of the system logger to accept these messages and then store them as configured. NXLog can be configured to directly accept logs that are sent to the /dev/log Unix domain socket, in place of the stock Syslog logger.

Configure NXLog (see the example below).
Disable the stock Syslog agent’s collection of /dev/log messages, if necessary. See also Replacing Rsyslog. Either
- disable the service entirely (for example, systemctl --now disable rsyslogd) or
- modify the configuration to disable reading from /dev/log (for example, remove $ModLoad imuxsock from /etc/rsyslog.conf and restart Rsyslog).
Restart NXLog.

Example 379. Reading From /dev/log

With this configuration, NXLog uses the im_uds module to read messages from /dev/log, and the parse_syslog() procedure to parse them.

Warning

FlowControl should be disabled for collecting from /dev/log. Otherwise the syslog() system call will block if the Output queue becomes full, resulting in an unresponsive system.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10

  <Extension _syslog>
    Module      xm_syslog
</Extension>

<Input in>
    Module      im_uds
    UDS         /dev/log
    FlowControl FALSE
    Exec        parse_syslog();
</Input>

92.3.3. Accepting Syslog via UDP, TCP, or TLS

NXLog can be configured to listen on a port and collect Syslog over the network. A port can be used to receive messages with UDP, TCP, or TLS transport. The local Syslog agent may already by configured to listen on port 514 for UDP log messages from local applications.

Configure NXLog with im_udp, im_tcp, or im_ssl. See the examples below.
For NXLog to listen for messages on port 514, the local Syslog agent must not be listening on that port. It may be necessary to either
- disable the service entirely (for example, systemctl disable rsyslogd) or
- modify the configuration to disable listening on port 514 (for example, remove input(type="imudp" port="514") from /etc/rsyslog.conf and restart Rsyslog).
Restart NXLog.

Example 380. Receiving Syslog via UDP

This configuration accepts either BSD or IETF Syslog from the local system only, via UDP.

Warning

The UDP transport can lose log entries and is therefore not recommended for receiving logs over the network.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10

  <Extension _syslog>
    Module  xm_syslog
</Extension>

<Input in>
    Module  im_udp
    Host    localhost
    Port    514
    Exec    parse_syslog();
</Input>

Example 381. Receiving Syslog via TCP

This configuration accepts either BSD or IETF Syslog via TCP, without supporting octet-framing.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10

  <Extension _syslog>
    Module  xm_syslog
</Extension>

<Input in>
    Module  im_tcp
    Host    0.0.0.0
    Port    1514
    Exec    parse_syslog();
</Input>

Example 382. Receiving IETF Syslog via TCP With Octet-Framing

This configuration accepts IETF Syslog via TCP, with support for octet-framing.

Note	Though this is for plain TCP, the Syslog_TLS directive is required because it refers to the octet-framing method described by RFC 5425.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11

  <Extension _syslog>
    Module      xm_syslog
</Extension>

<Input in>
    Module      im_tcp
    Host        0.0.0.0
    Port        1514
    InputType   Syslog_TLS
    Exec        parse_syslog_ietf();
</Input>

Example 383. Receiving IETF Syslog via TLS

This configuration accepts IETF Syslog via TLS, with support for octet-framing.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14

  <Extension _syslog>
    Module      xm_syslog
</Extension>

<Input in>
    Module      im_ssl
    Host        0.0.0.0
    Port        6514
    CAFile      %CERTDIR%/ca.pem
    CertFile    %CERTDIR%/client-cert.pem
    CertKeyFile %CERTDIR%/client-key.pem
    InputType   Syslog_TLS
    Exec        parse_syslog_ietf();
</Input>

92.4. Generating Syslog

NXLog can be configured to generate BSD or IETF Syslog and:

write it to file,
send it to the local syslog daemon via the /dev/log Unix domain socket, or
forward it to another destination over the network (via UDP, TCP, or TLS).

In each case, the to_syslog_bsd() and to_syslog_ietf() procedures are used to generate the $raw_event field from the corresponding fields in the event record.

92.4.1. Writing Syslog to File

The om_file module is used to write logs to file.

Example 384. Writing BSD Syslog to File

This configuration write logs to the specified file in the BSD Syslog format.

nxlog.conf


  1
2
3
4
5
6
7
8
9

  <Extension _syslog>
    Module  xm_syslog
</Extension>

<Output out>
    Module  om_file
    File    "/var/log/syslog"
    Exec    to_syslog_bsd();
</Output>

NXLog can be configured to write BSD Syslog to a file without the PRI part, emulating traditional Syslog implementations.

Example 385. Writing BSD Syslog Without the PRI

This configuration includes a regular expression for removing the PRI part from the $raw_event field after it is generated by the to_syslog_bsd() procedure.

nxlog.conf


  1
2
3
4
5
6
7
8
9

  <Extension _syslog>
    Module  xm_syslog
</Extension>

<Output out>
    Module  om_file
    File    "/var/log/syslog"
    Exec    to_syslog_bsd(); $raw_event =~ s/^\<\d+\>//;
</Output>

92.4.2. Sending Syslog to the Local Syslog Daemon via /dev/log

The om_uds module can be used for sending logs to a Unix domain socket.

Example 386. Sending Syslog to /dev/log

This configuration sends BSD Syslog to the Syslog daemon via /dev/log.

nxlog.conf


  1
2
3
4
5
6
7
8
9

  <Extension _syslog>
    Module  xm_syslog
</Extension>

<Output out>
    Module  om_uds
    UDS     /dev/log
    Exec    to_syslog_bsd();
</Output>

92.4.3. Sending Syslog to a Remote Logger via UDP, TCP, or TLS

The om_udp, om_tcp, and om_ssl modules can be used for sending Syslog over the network.

Example 387. Forwarding BSD Syslog via UDP

This configuration sends logs in BSD Syslog format to the specified host, via UDP port 514.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10

  <Extension _syslog>
    Module  xm_syslog
</Extension>

<Output out>
    Module  om_udp
    Host    192.168.1.1
    Port    514
    Exec    to_syslog_bsd();
</Output>

Example 388. Forwarding BSD Syslog via TCP

This configuration sends logs in BSD format to the specified host, via TCP port 1514.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10

  <Extension _syslog>
    Module  xm_syslog
</Extension>

<Output out>
    Module  om_tcp
    Host    192.168.1.1
    Port    1514
    Exec    to_syslog_bsd();
</Output>

Example 389. Forwarding IETF Syslog via TLS

With this configuration, NXLog sends logs in IETF format to the specified host, via port 6514.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14

  <Extension _syslog>
    Module      xm_syslog
</Extension>

<Output out>
    Module      om_ssl
    Host        192.168.1.1
    Port        6514
    CAFile      %CERTDIR%/ca.pem
    CertFile    %CERTDIR%/client-cert.pem
    CertKeyFile %CERTDIR%/client-key.pem
    OutputType  Syslog_TLS
    Exec        to_syslog_ietf();
</Output>

Note	The `OutputType Syslog_TLS` directive is necessary if octet-framing is required. The name was chosen to refer to the octet-framing method described by RFC 5425.

92.5. Extending Syslog

BSD Syslog uses a free-form message field, and does not provide a standard way to include key-value pairs in log messages. This section documents ways that structured data has been implemented using BSD Syslog as transport.

92.5.1. IETF Syslog Structured-Data

The Structured-Data part of the IETF Syslog format, as documented above, provides a syntax for key-value pairs.

Log Sample

<13>1 2016-10-13T14:23:11.000000-06:00 myserver - - - [NXLOG@14506 Purpose="test"] This is a test message.

NXLog can parse IETF Syslog with the parse_syslog() procedure provided by the xm_syslog extension module.

Example 390. Parsing IETF Syslog With Structured-Data

With this configuration, NXLog will parse the input IETF Syslog format from file, convert it to JSON, and output the result to file.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

  <Extension _json>
    Module  xm_json
</Extension>

<Extension _syslog>
    Module  xm_syslog
</Extension>

<Input in>
    Module  im_file
    File    '/var/log/messages'
    Exec    parse_syslog();
</Input>

<Output out>
    Module  om_file
    File    '/var/log/json'
    Exec    to_json();
</Output>

<Route r>
    Path    in => out
</Route>

Output Sample

{
  "EventReceivedTime": "2016-10-13 15:23:12",
  "SourceModuleName": "in",
  "SourceModuleType": "im_file",
  "SyslogFacilityValue": 1,
  "SyslogFacility": "USER",
  "SyslogSeverityValue": 5,
  "SyslogSeverity": "NOTICE",
  "SeverityValue": 2,
  "Severity": "INFO",
  "EventTime": "2016-10-13 15:23:11",
  "Hostname": "myserver",
  "Purpose": "test",
  "Message": "This is a test log message."
}

NXLog can also generate IETF Syslog with a Structured-Data part, using the to_syslog_ietf() procedure provided by the xm_syslog extension module.

Example 391. Generating IETF Syslog With Structured-Data

With the following configuration, NXLog will parse the input JSON from file, convert it to IETF Syslog format, and output the result to file.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

  <Extension _json>
    Module  xm_json
</Extension>

<Extension _syslog>
    Module  xm_syslog
</Extension>

<Input in>
    Module  im_file
    File    '/var/log/json'
    Exec    parse_json();
</Input>

<Output out>
    Module  om_file
    File    '/var/log/ietf'
    Exec    to_syslog_ietf();
</Output>

<Route r>
    Path    in => out
</Route>

Input Sample

{
  "EventTime": "2016-09-13 11:23:11",
  "Hostname": "myserver",
  "Purpose": "test",
  "Message": "This is a test log message."
}

Output Sample

<13>1 2016-09-13T11:23:11.000000-05:00 myserver - - - [NXLOG@14506 EventReceivedTime="2016-09-13 11:23:12" SourceModuleName="in" SourceModuleType="im_file" Purpose="test] This is a test log message.

92.5.2. JSON over Syslog

JSON has become popular recently to transfer structured data. For compatibility with Syslog devices it is common practice to encapsulate JSON in Syslog. NXLog can generate JSON with the to_json() procedure function provided by the xm_json extension module.

Example 392. Generating JSON with Syslog Header

With the following configuration, NXLog will read the Windows Event Log, convert it to JSON format, add a Syslog header, and send the logs via UDP to a Syslog agent. NXLog log messages are also included (via the im_internal module).

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

  <Extension _json>
    Module      xm_json
</Extension>

<Extension _syslog>
    Module      xm_syslog
</Extension>

<Input internal>
    Module      im_internal
</Input>

<Input eventlog>
    Module      im_msvistalog
</Input>

<Output out>
    Module      om_udp
    Host        192.168.1.1
    Port        514
    Exec        $Message = to_json(); to_syslog_bsd();
</Output>

<Route r>
    Path        internal, eventlog => out
</Route>

Note	If Syslog compatibility is not a concern, JSON can be transported without the Syslog header (omit the to_syslog_bsd() procedure).

92.5.3. Other Syslog Extensions

See also these formats, which extend BSD Syslog by using the free-form message field to contain key-value pairs.