NXLog User Guide

58. Docker

Docker is a containerization technology that enables the creation and use of Linux containers. Containers allow a developer to package an application with all of its dependencies and distribute it as a single package. The Docker container technology is widely used in modern, micro-service architectures.

By concept, Docker images should be lightweight; usually only one application is present and running in the container. Therefore, logs are written to the standard out and standard error streams and logging must be performed from outside the image.

58.1. Configuring Logging in Docker

By default, Docker writes logs from each container to a separate JSON file, stored under the container’s directory on the host machine. The logging of containers can be configured in two ways: by modifying the default logging configuration of the Docker daemon, or by changing it in the runtime options for a specific container. For more details about Docker’s logging drivers, see Configure logging drivers on Docker.com.

The default logging driver can be set in the daemon.json configuration file. This file is located in /etc/docker/ on Linux hosts or C:\ProgramData\docker\config\ on Windows Server hosts. The default logging driver is json-file.
The default logging driver can be overridden at the container level. To accomplish this, the log driver and its configuration options must be provided as parameters at container startup with the help of the docker run command. The configuration options are the same as setting up logging options for the Docker daemon. See the docker run command reference on Docker.com for more information.

58.2. Receiving Logs From Docker

Collecting logs from a Docker daemon or container is supported in four ways depending on the log driver in use.

To find the current logging driver for a running container, run the following docker inspect command, substituting the container name or ID for <CONTAINER>.

$ docker inspect -f '{{.HostConfig.LogConfig.Type}}' <CONTAINER>

58.2.1. JSON

With the json-file log driver, Docker produces a line-based log file in JSON format for each container. See the JSON File logging driver guide on Docker.com for more information.

Note	Because im_file recursively watches for log files in the containers directory, this may cause reduced performance in very large installations.

Example 243. Collecting Docker Logs in JSON Format

This example configuration reads from the JSON log files of all containers. The JSON fields are parsed and added to the event record with the xm_json parse_json() procedure. A $HostID field, with the container ID, is also added.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

  <Extension _fileop>
    Module  xm_fileop
</Extension>

<Extension _json>
    Module  xm_json
</Extension>

<Input in>
    Module  im_file
    File    '/var/lib/docker/containers/*/*-json.log'
    <Exec>
        parse_json();
        $HostID = file_basename(file_name());
        $HostID =~ s/-json.log//;
    </Exec>
</Input>

58.2.2. GELF

The gelf logging driver is a convenient format that is understood by a number of tools such as NXLog. In GELF, every log message is a dictionary with fields such as version, host, timestamp, short and long version of the message, and any custom fields that have been configured. See the Graylog Extended Format logging driver guide on Docker.com for more information.

Example 244. Collecting Docker Logs in GELF Format

In this example, NXLog accepts and parses logs in GELF format on TCP port 12201 with the im_tcp and xm_gelf modules.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10

  <Extension _gelf>
    Module      xm_gelf
</Extension>

<Input in>
    Module      im_tcp
    Host        0.0.0.0
    Port        12201
    InputType   GELF_TCP
</Input>

58.2.3. Syslog

The syslog logging driver routes logs to a Syslog server, such as NXLog, via UDP, TCP, SSL/TLS, or a Unix domain socket. See the Syslog logging driver guide on Docker.com for more information.

Example 245. Collecting Docker Logs in Syslog Format

Here, NXLog accepts logs on TCP port 1514 with the im_tcp module and parses the logs with the xm_syslog parse_syslog() procedure.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10

  <Extension _syslog>
    Module  xm_syslog
</Extension>

<Input in>
    Module  im_tcp
    Host    0.0.0.0
    Port    1514
    Exec    parse_syslog();
</Input>

58.2.4. ETW

On Windows-based systems, the etwlogs logging driver forwards container logs to the Event Tracing for Windows (ETW) system. Each ETW event contains a message with both the log and its context information. See the ETW logging driver guide on Docker.com for more information.

Example 246. Collecting Docker Logs in ETW Format

This example collects logs from the DockerContainerLogs Event Tracing provider using the im_etw module.

nxlog.conf


  1
2
3
4

  <Input in>
    Module      im_etw
    Provider    DockerContainerLogs
</Input>