NXLog User Guide

22. Configuration Overview

NXLog uses Apache style configuration files. The configuration file is loaded from its default location, or it can be explicitly specified with the -c command line argument.

The configuration file is made up of blocks and directives. Blocks are similar to XML tags containing multiple directives. Directive names are case insensitive but arguments are case sensitive in some cases. A directive and its argument must be specified on the same line. Values spanning multiple lines must have the newline escaped with the backslash (\). A typical case for this is the Exec directive. Blank lines and lines starting with the hash mark (#) are ignored. Configuration directives referring to a file or a path can be quoted with double quotes (") or single quotes ('). This applies to both global and module directives.

The configuration file can be logically divided into three parts: global parameters, module instances, and route instances.

Example 10. Configuration File Structure

Here are the three sections of a configuration file. This example contains two global directives, Input and Output blocks, and a Route block.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

  # Global section
User        nxlog
Group       nxlog

# Modules section
<Input in>
    Module  im_null
</Input>

<Output out>
    Module  om_null
</Output>

# Route section
<Route r>
    Path    in => out
</Route>

22.1. Global Directives

The global section contains directives that control the overall behavior of NXLog.

The LogFile directive sets a destination file for NXLog internal logs. If this directive is unset, the log file is disabled and internal NXLog logs are not written to file (unless configured via the im_internal module). See also Rotating the Internal Log File.

With the User and Group directives set, NXLog will drop root privileges after starting and run under the specified user and group. These directives are ignored if running on the Windows platform.

After starting, NXLog will change its working directory to the directory specified by the SpoolDir directive. Non-absolute paths in the configuration will be relative to this directory.

See the Reference Manual for a complete list of available global directives.

22.2. Modules

NXLog will only load modules which are specified in the configuration file and used in an active route. A module instance is specified according to its corresponding module type (Extension, Input, Processor, or Output). Each module instance must have a unique name and a Module directive. The following is a skeleton configuration block for an input module.

nxlog.conf


  1
2
3
4

  <Input instancename>
    Module  im_module
    ...
</Input>

For more details about module instance names, see Configuration in the Reference Manual.

22.3. Routes

Routes define the flow and processing order of the log messages. Each route instance must have a unique name and a Path.

Example 11. Route Block

This Route instance, named example, takes logs from Input module instances named in1 and in2, processes the logs with the proc Processor module instance, and sends the resulting logs to both Output module instances out1 and out2. These named module instances must be defined elsewhere in the configuration file.

nxlog.conf


  1
2
3

  <Route example>
    Path    in1, in2 => proc => out1, out2
</Route>

For more details about route instance names, see Configuration in the Reference Manual.

If no Route block is specified in the configuration, NXLog will automatically generate a route, with all the Input and Output instances specified in a single path.

Example 12. An Automatic Route Block

NXLog can use a configuration with no Route block, such as the following.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

  <Input in1>
    Module  im_null
</Input>

<Input in2>
    Module  im_null
</Input>

<Output out1>
    Module  om_null
</Output>

<Output out2>
    Module  om_null
</Output>

The following Route block will be generated automatically.

nxlog.conf (Generated Route)


  1
2
3

  <Route r>
    Path    in1, in2 => out1, out2
</Route>

22.4. Constant and Macro Definitions

A define is useful if there are many instances in the configuration where the same value must be used. Typically, defines are used for directories and hostnames. In such cases the value can be configured with a single definition. In addition to constants, other strings like code snippets or parser rules can be defined in this way.

An NXLog define works in a similar way to the C language, where the pre-processor substitutes the value in places where the macro is used. The NXLog configuration parser replaces all occurrences of the defined name with its value, and then after this substitution the configuration check occurs.

Example 13. Using Defines

This example shows the use of two defines: BASEDIR and IGNORE_DEBUG. The first is a simple constant, and its value is used in two File directives. The second is an NXLog language statement, it is used in an Exec directive.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13

  define BASEDIR /var/log
define IGNORE_DEBUG if $raw_event =~ /debug/ drop();

<Input messages>
    Module  im_file
    File    '%BASEDIR%/messages'
</Input>

<Input proftpd>
    Module  im_file
    File    '%BASEDIR%/proftpd.log'
    Exec    %IGNORE_DEBUG%
</Input>

The define directive can be used for statements as shown above, but multiple statements should be specified using a code block, with curly braces ({}), to result in the expected behavior.

Example 14. Incorrect Use of a Define

The following example shows an incorrect use of the define directive. After substitution, the drop() procedure will always be executed; only the warning message will be logged conditionally.

nxlog.conf (incorrect)


  1
2
3
4
5
6
7

  define ACTION log_warning("dropping message"); drop();

<Input in>
    Module  im_file
    File    '/var/log/messages'
    Exec    if $raw_event =~ /dropme/ %ACTION%
</Input>

To avoid this problem, the action should be defined using a code block.


  1

  define ACTION { log_warning("dropping message"); drop(); }

22.5. Environment Variables

The envvar directive works like define except that the value is retrieved from the environment. This makes it possible to reference the environment variable as if it was a define. This directive is only available in NXLog Enterprise Edition.

Example 15. Using Environment Variables

This is similar to the previous example using a define, but here the value is fetched from the environment.

nxlog.conf


  1
2
3
4
5
6

  envvar BASEDIR

<Input in>
    Module  im_file
    File    '%BASEDIR%/messages'
</Input>

22.6. File Inclusion

NXLog provides several features for including configuration directives and blocks from separate files or from executables.

Note

The SpoolDir directive does not take effect until after the configuration has been parsed, so relative paths specified with these directives are relative to the working directory where NXLog was started from. Generally, it is recommended to use absolute paths. If desired, define directives can be used to simulate relative paths (see Using Defines to Include a Configuration File).

With the include directive it is possible to specify a file or set of files to be included in the current configuration file.

Example 16. Including a Configuration File

This example includes the contents of the /opt/nxlog/etc/syslog.conf file in the current configuration.

nxlog.conf


  1

  include /opt/nxlog/etc/syslog.conf

Example 17. Using Defines to Include a Configuration File

In this example for Windows, two define directives are used to include an eventlog.conf configuration file.

nxlog.conf


  1
2
3

  define ROOT C:\Program Files (x86)\nxlog
define CONFDIR %ROOT%\conf
include %CONFDIR%\eventlog.conf

The include directive also supports wildcarded filenames. A set of files in a directory can be included without the need to explicitly list each one. This could be used to specify a drop-in directory for OS-specific configuration snippets (like windows2003.conf) or application-specific snippets (such as syslog.conf).

Example 18. Including a Configuration Directory

This example includes all .conf files in /opt/nxlog/etc/nxlog.d.

nxlog.conf


  1

  include /opt/nxlog/etc/nxlog.d/*.conf

Because the backslash (\) is used as a path separator on Windows, an additional backslash (\) must be used to prevent the \* from being treated as a literal asterisk (*). The same wildcard rules apply as for the im_file File directive.

nxlog.conf (Windows)


  1

  include C:\Program Files\nxlog\conf\nxlog.d\\*.conf

With the include_stdout directive, an external command can be used to provide configuration content. There are many ways this could be used, including fetching, decrypting, and validating a signed configuration from a remote host, or generating configuration content dynamically.

Example 19. Using an Executable to Generate Configuration

Here, a separate script is responsible for fetching the NXLog configuration.

nxlog.conf


  1

  include_stdout /opt/nxlog/etc/fetch_conf.sh