Audit logging is important for maintaining security and investigating incidents, and is required by many cybersecurity standards. Most operating systems have a native auditing system available for this purpose.
About security event auditing
Auditing systems in modern operating systems collect detailed information about security-related events. The audit or security logs generated by an auditing system facilitate identification of attempted attacks, security policy improvement, security incident investigation, and review by auditors. Logging of security events is required by ISO/IEC 27001,[1] the NIST Cybersecurity Framework (CSF),[2] various Common Criteria protection profiles,[3] and many other cybersecurity standards and guidelines.[4]
Note that an auditing system does not provide improved security directly. It is the responsibility of an operating system’s security policy to specify whether an action should be allowed or denied. Meanwhile, the auditing system is responsible for keeping a detailed record showing how the security policy was applied. In most cases, an audit record will include a success or failure status along with any relevant information such as a username. An audit policy specifies which attempted action and status combinations should be logged, allowing administrators to reduce auditing scope to the security events that are considered important for the organization. For example, an audit policy may include only successful user login events but include all attempts to access a restricted resource (successful or not).
Once an auditing system is configured and enabled, the resulting audit logs should be stored in a secure location and retained for a period of time, so they are available for any audits or investigations that may be required. In some cases it may be necessary to forward audit logs to another system with higher security, perhaps even at a remote location.
Native auditing systems
Most modern operating systems have some form of audit system available, but the functionality, configuration, and audit log formats vary. Generally, an auditing system has a kernel component and various user space utilities. The kernel portion watches for system calls and generates those events that are specified by the audit policy. User space utilities may be available for functionality such as enabling or disabling auditing, configuring audit policy, collecting events from the kernel, filtering events, or creating reports for administrators.
Implementing an organization-wide auditing policy requires working with a variety of auditing systems, depending on which operating systems are deployed in the organization’s network.
- Linux
-
The Linux Audit system is available with most Linux-based operating systems. The Audit system can log many different types of events, including file and network accesses, system calls, failed logins, and commands run by users. It is composed of a kernel component that generates events, an Audit daemon (auditd) for collecting events, and several other tools for configuring auditing, relaying events, and reading the log files.[5] The system is configured with control rules, which specify the system’s behavior; file system rules, which audit access to a file or directory; and system call rules, which log system calls made by specified programs.[6]
By default, auditd collects events from the kernel and writes them in a key-value pair format to
/var/log/audit/audit.log.[7] Theausearchandaureportcommands can be used to search audit records and generate audit reports, respectively. For more information about the Linux Audit system, see the Red Hat Security Guide, as well as the audit(8) manual page and other pages listed there. - Microsoft Windows
-
Windows security auditing provides two groups of policy settings for configuring native auditing on Windows: "basic" and "advanced". The two groups are considered incompatible in practice; if advanced settings are used, it is recommended that a Group Policy setting be enabled to force the basic settings to be ignored.[8] The current local audit policy can be displayed and manipulated with the auditpol command. Audit records are written to the Security channel of the Windows Event Log.
The basic audit policy settings, available since Windows 2000,[9] include nine settings for auditing. Each setting controls a group of related auditable events. Settings are available for a range of events including logon activity, account management, object access, policy changes, and privilege use.[10] These settings are configured using the Local Security Policy snap-in (
secpol.msc) under Local Policies\Audit Policy or via Group Policy under Computer Configuration\Windows Settings\Security Settings\Audit Policy.[11] For more information, see Basic security audit policies on Microsoft Docs.The advanced settings, introduced in Windows Vista and Windows Server 2008,[9] offer more detailed policy configuration, with more than fifty settings in ten categories. These settings overlap with the basic settings and provide more granularity for specifying auditable events. As with the basic settings, each advanced setting controls a group of events. These settings are found under Computer Configuration\Windows Settings\Security Settings\Advanced Audit Policy Configuration\System Audit Policies in Group Policy.[12] For more information, see Advanced security audit policies on Microsoft Docs.
The Sysmon utility, included in Microsoft’s Sysinternals toolset, is a Windows system service and device driver that monitors and logs system activity to the Windows Event Log. Sysmon logs events and event information beyond what is possible with standard security auditing. Events include process creations, network connections, driver loading, raw filesystem reading, changes in file creation timestamps, and early boot process activity. Additional logged details include process command lines and hashes; process and session identifiers for event correlation; and for network connections, parent processes, IP addresses, and port numbers. An XML configuration system allows the administrator to specify the events to be logged with detailed rules for filtering events. Starting with Windows Vista, Sysmon events are written to the Microsoft-Windows-Sysmon/Operational channel.[13] For more information, see Sysmon on Microsoft Docs.
- Apple macOS, FreeBSD, and Oracle Solaris
-
These operating systems each implement Sun’s Basic Security Module (BSM) auditing system.[14] The BSM auditing system can be used to audit a variety of events, including authentication, logins, program execution, inter-process communication (IPC), file activity, and network access.[15] It has a kernel component for generating events, a user space daemon (auditd) for collecting the events, and several other utilities for working with the audit logs (referred to as "audit trails"). Auditable events are grouped into event classes, such as the
fcclass for file creations. The audit policy is configured by listing these classes (with optional prefixes to select success/failure events) in the system-wide policy and in any per-user policies that may be required.[15]The audit daemon writes audit trails to files in
/var/audit/using the BSM binary format. A stream of audit records, useful for live monitoring, is also available through the/dev/auditpipedevice.[16] In both cases, theauditreducecommand can be used to filter events andprauditwill parse the audit trail and dump it to plain text. For more information about BSM auditing, see the NIST Guide to Securing Apple OS X 10.10 Systems for IT Professionals (section 6.4.2), the Security Event Auditing chapter in the FreeBSD Handbook, and Solaris Auditing in the System Administration Guide. - IBM AIX
-
The AIX auditing subsystem logs a wide array of "security-relevant occurrences": process and file activity, network connections, user account management, audit system administration, filesystem volume changes, mail delivery configuration, job scheduling, and so on.[17] The system has a kernel audit logger and several commands for controlling the audit system and processing audit logs. Audit policy is based on a per-process or per-object scope. With per-process auditing, auditable events are selected and grouped into classes, which are then assigned to users. With per-object auditing, events are assigned to file objects.[18]
Output can be written to binary audit files for long-term storage (BIN mode), to a circular buffer that can be read from
/dev/audit(STREAM mode), or both.[19] For either mode, processing of audit data is done with a chain of commands that read, filter, compress, and convert the audit records as required.[18][20] For more information about AIX auditing, see the Auditing overview topic in IBM’s AIX 7.2 documentation and AIX Auditing Best Practices on IBM Support.
Auditable system events may also be generated outside of a platform’s native auditing system. For example, PAM authentication events on Linux are logged to Syslog, and audit events for a particular application may be written to a separate Windows Event Log channel.
Using NXLog to collect audit events
NXLog Enterprise Edition is a flexible, high-performance logging solution that offers a wide range of functionality for collecting audit data on multiple platforms. All log data is collected and processed using a structured log format, where each event is comprised of a set of fields. This reduces or eliminates the need for event parsing and allows event data to be collected in a unified way across different audit log sources.[21]
NXLog includes native auditing support for several operating systems, a unique feature not found in competing products. Native auditing improves performance and simplifies administration by eliminating the need for an audit daemon—events are collected directly from the kernel as they are generated. Without native auditing, events are written to disk by the audit daemon, read from disk by the logging agent, and then parsed.
- Linux Audit system
-
NXLog’s im_linuxaudit module collects Audit events using the Linux kernel’s audit interfaces as a local agent—auditd is not required. This module also performs Audit system configuration, with Audit rules specified in the NXLog configuration or in a separate file.
For cases where direct Audit system configuration and collection is not feasible, audit events can instead be collected via Syslog or by reading from the local Audit log file written by auditd. Alternatively, for agentless audit log collection, NXLog can be configured to accept events forwarded by the Audit dispatcher (audispd).
See the Linux Audit System chapter in the NXLog User Guide for detailed information and examples.
- Windows Event Log
-
Audit events generated by the Windows security auditing system, and audit events from Sysmon, can be read from the Windows Event Log. With the im_msvistalog module, NXLog supports collection either as a local agent or remotely using MSRPC.[22] The im_wseventing module is also available for collecting events remotely from hosts with Windows Event Forwarding (WEF) configured.
For more information, see Collecting Event Log Data and Sysmon in the NXLog User Guide.
- Basic Security Module (BSM) auditing
-
For collecting events from the BSM auditing system on FreeBSD and macOS, the im_bsm module supports reading directly from the
/dev/auditpipedevice. The audit daemon is not required.Alternatively, or for Solaris where the
auditpipedevice is not available, the xm_bsm module parses events after they have been written to file by the audit daemon. - AIX auditing
-
For auditing on AIX, NXLog’s im_aixaudit module reads directly from the
/dev/auditdevice and parses the data. No system audit commands are needed for processing audit records; NXLog’s processing features can be used to implement any filtering, compression, or conversion that may be required, whether at the local agent or a remote agent (if logs are forwarded).For cases where AIX audit events are already written to disk, the xm_aixaudit module can be used to parse events in binary format.
In addition to the auditing systems above, NXLog offers many other features for meeting an organization’s auditing requirements. Any auditable system events written to regular system logs can be collected separately. Functionality is available for collecting logs generated by Exchange administrator and mailbox auditing, SQL Server Auditing, and a variety of other applications, services, and enterprise hardware. File integrity monitoring and Windows Registry monitoring can be implemented as an additional layer of security. Once audit data has been collected, NXLog can implement retention policies and/or forward log data to an event management product or service for monitoring and analysis.