NXLog User Guide

44. Bro Network Security Monitor

NXLog can be configured to collect events generated by the Bro Network Security Monitor, a powerful open source Intrusion Detection System (IDS) and network traffic analysis framework. The Bro engine captures traffic and converts it to a series of high-level events. These events are then analyzed according to customizable policies. Bro supports real-time alerts, data logging for further investigation, and automatic program execution for detected anomalies. Bro is able to analyze different protocols, including HTTP, FTP, SMTP, and DNS; as well as run host and port scans, detect signatures, and discover syn-floods.

44.1. About Bro Logs

Bro creates different log files in order to record network activities such as files transferred over the network, SSL sessions, and HTTP requests. By default, BRO provides 60 different log files.

Table 31. A Few of Bro’s Default Log Files
File	Description
conn.log	TCP/UDP/ICMP connections
dhcp.log	DHCP leases
dns.log	DNS activity
files.log	Summaries of files transferred over the network
ftp.log	FTP activity
http.log	HTTP requests and replies
smtp.log	SMTP transactions
ssl.log	SSL/TLS handshake information
weird.log	Unexpected network-level activity

Bro produces human-readable logs in a format similar to W3C. Each log file uses a different set of fields.

dns.log Sample

#separator \x09
#set_separator	,
#empty_field	(empty)
#unset_field	-
#path	dns
#open	2018-01-03-11-44-20
#fields	ts	uid	id.orig_h	id.orig_p	id.resp_h	id.resp_p	proto	trans_id	rtt	query	qclass	qclass_name	qtype	qtype_name	rcode	rcode_name	AA	TC	RD	RA	Z	answers	TTLs	rejected
#types	time	string	addr	port	addr	port	enum	count	interval	string	count	string	count	string	count	string	bool	bool	bool	bool	count	vector[string]	vector[interval]	bool
1514951060.504593	CWKugb3ukgbFv1yhRf	192.168.56.101	59572	192.168.56.1	137	udp	23266	-	WORKGROUP	1	C_INTERNET	33	NBSTAT	0	NOERROR	F	F	F	F	0	-	-	F
1514951060.504593	CQINoQ18vBSIBaLiO9	192.168.56.101	49062	192.168.56.1	137	udp	7805	-	-	-	-	-	-	0	NOERROR	F	F	F	F	0	-	-	F
1514951060.504593	CthpDaR4iKLHtl3Qi	192.168.56.101	49062	192.168.56.255	137	udp	7805	-	WORKGROUP	1	C_INTERNET	32	NB	-	-	F	F	T	F	1	-	-	F

For more information about Bro logging, see the Bro Manual.

44.2. Parsing Bro Logs

NXLog Enterprise Edition can parse Bro logs with the xm_w3c module.

Note	The following configurations have been tested with Bro version 2.5.2.

Example 202. Using xm_w3c to Parse Bro Logs

This configuration reads Bro logs from a directory, parses with xm_w3c, and writes out events in JSON format.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

  <Extension _json>
    Module      xm_json
</Extension>

<Extension w3c_parser>
    Module      xm_w3c
</Extension>

<Input bro_in>
    Module      im_file
    File        '/usr/local/bro/logs/current/*.log'
    InputType   w3c_parser
</Input>

<Output bro_file>
    Module      om_file
    File        '/tmp/bro_logs'
    Exec        to_json();
</Output>

The xm_w3c module is recommended because it supports reading the field list from the W3C-style log file header. For NXLog Community Edition, the xm_csv module could be used instead to parse Bro logs. A separate instance of xm_csv must be configured for each log type.

Example 203. Using xm_csv to Parse Bro Logs

This example has separate xm_csv module instances for the DNS and DHCP log types. Additional CSV parsers could be added for the remaining Bro log types.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

  <Extension csv_parser_dns>
    Module      xm_csv
    Fields      ts, uid id.orig_h, id.orig_p, id.resp_h, id.resp_p, proto, \
                trans_id, rtt query, qclass, qclass_name, qtype, qtype_name, \
                rcode, rcode_name, AA, TC, RD, RA, Z, answers, TTLs, rejected
    Delimiter   \t
</Extension>

<Extension csv_parser_dhcp>
    Module      xm_csv
    Fields      ts, uid, id.orig_h, id.orig_p, id.resp_h, id.resp_p, mac, \
                assigned_ip, lease_time, trans_id
    Delimiter   \t
</Extension>

# xm_fileop provides the `file_basename()` function
<Extension _fileop>
    Module      xm_fileop
</Extension>

<Input bro_in>
    Module      im_file
    File        '/usr/local/bro/logs/current/*.log'
    <Exec>
        if file_basename(file_name()) == 'dhcp.log'
        {
            csv_parser_dhcp->parse_csv();
        }
        else if file_basename(file_name()) == 'dns.log'
        {
            csv_parser_dns->parse_csv();
        }
        else
        {
            log_warning('Bro log type not supported, check configuration');
        }
    </Exec>
</Input>