Event Log: Leveraging Events and Endpoint Logs for Security

An event log is a file that contains information about usage and operations of operating systems, applications or devices. Security professionals or automated security systems like SIEMs can access this data to manage security, performance, and troubleshoot IT issues.

In the modern enterprise, with a large and growing number of endpoint devices, applications and services, it is no longer possible to manage security and IT operations with network monitoring alone. Event logs, and in particular endpoint logs, are of critical importance.

Introduction to Event Logs and Security Logs

Events that occur in end-user devices or IT systems are commonly recorded in log files. Operating systems record events using log files. Each operating system uses its own log files, and applications and hardware devices also generate logs. Security teams can use security logs to track users on the corporate network, identify suspicious activity and detect vulnerabilities.

Most security and IT organizations find that systems generate more log information than they can process. Event and log management tools help analyze logs, monitor important events recorded in logs, and leverage them to identify and investigate security incidents.

Key Concepts of Log Management

Using Endpoint Logs for Security

With the growing use of endpoint devices, many of which are laptops, phones or other mobile devices, endpoint logs are becoming more important for security. Attackers who gain access to an endpoint device can use it to penetrate your network. Therefore, it’s essential to collect data from endpoint logs and identify malicious or unauthorized activity.

Using Windows Event Logs for Security

The Windows operating system logs activity on software or hardware components. Administrators can access this information to detect and troubleshoot issues. Six default categories are used to classify events:

• Application log – events logged by applications. Developers determine the events logged by their application. The application can log information from several sources. It is important to note the source alongside the event ID.
• System log – events logged by the operating system. For example, issues experienced by drivers during the startup process.
• Security log – events related to security, including login attempts or file deletion. Administrators determine which events to enter into their security log, according to their audit policy.
• Directory service log – records active directory operations like authentication and modification of privileges. Only available on domain controllers.
• DNS server log – records DNS activity. Only available on DNS servers.
• File replication service log – records domain controller replication, only available on domain controllers.

The Windows system called Event Viewer can be used to view event logs across all the above categories. Event Viewer displays information about an event, including the date and time, username, computer, source, and type.

Security Log Event Types

While all types of events could be relevant in the investigation of a security incident, security logs are of special significance. Windows generates a security log entry upon login attempts, and logs additional information if the login attempt succeeds. The types of events logged are:

Common Windows Log Events Used in Security Investigations

Here are a few common event codes on Windows 7/Vista/8/10 and Windows Server 2008/2012R2/2016/2019 (previous versions of Windows have different codes), commonly used in security investigations:

Using Linux Event Logs for Security

The Linux operating system stores a timeline of events related to the server, kernel, and running applications. The main log categories are:

• Application logs
• Event logs
• Service logs
• System logs

There are several ways to view logs in Linux:

• Access the directory cd/var/log. Specific log types are stored in subfolders under the log folder, for example, var/log/syslog.
• Use the dmseg command to browse through all system logs
• Use the tail command, which displays the last lines written to a certain log file, where problems are usually found. For example tail -f /var/log/syslog prints the next line written to the file, letting you follow changes to the syslog file as they happen.

Following are commonly used Linux log files:

• /var/log/syslog or /var/log/messages – general system activity logs. Used to detect problems that may occur during startup or to isolate application service errors. RedHat-based systems store information in the messages folder while Debian-based systems store them in the syslog folder.
• /var/log/auth.log or /var/log/secure – all authentication and authorization logs. Used to investigate failed login attempts. RedHat-based systems store these in the auth.log folder while Debian-based systems store them in the secure folder.
• /var/log/kern.log – kernel activity logs, including custom kernels.
• /var/log/faillog – failed login attempts.
• /var/log/maillog or var/log/mail.log – logs related to mail servers. Used to track issues like emails tagged as spam, and suspicious use of postfix or smtpd.

What Data Should You Focus on in Security Investigation of Linux Event Logs?

Perform a risk assessment for Linux systems in your organization, and determine what level of logging they need, how logs should be reviewed and which log events should generate security alerts. In most cases you will need to log the following information about a Linux system for security purposes:

• User and terminal IDs
• Login attempts and log off by users
• Any attempts to access systems, data, applications, files or networks, whether on the local machine, or over a LAN or WAN
• Changes to Linux configuration
• Running executable processes on the machine
• Use of system utilities
• Security-related events, activation or deactivation of security tools

Using iOS Logging for Security

iOS does not log events, however it does log application crash reports. iOS 10.0 and later offers an API that can be used to log application events. You can use crash reports and the logging API to find and investigate errors generated by your applications, either during development or in production.

iOS devices come with their own security features, implemented in both hardware and software. The logging API provides access to data generated by these security features. They include:

• Data encryption – protects both personal and business data from unauthorized use.
• App security – verifies the security of iOS apps.
• Network security – provides developers with protocols for secure authentication and encrypted data transmission.
• Apple pay – iOS devices can be used to pay securely. Identifiable information is not collected.
• Internet services – iOS provides a variety of secure services, like iMessage and iCloud Backup.
• User password management – authentication methods such as password autofill keychain. Apps cannot access this information without user permission.
• Device controls – management tools that enable actions like remotely wiping stolen devices.
• Privacy controls – users can decide which apps access what information.

Using Android Logging for Security

Android offers a platform that provides access to all system and applications logging, including logs from the kernel driver, C, C++, and Java classes. The logging platform provides applications for viewing and filtering log messages.

Android Log Types

• Application log – an android application uses the android.util.Log class to create log messages. Applications can set log levels or message “severity”, and descriptive tags, to enable log filtering and alerting.
• Event log – messages are created using the android.util.EventLog class, which uses binary-formatted log messages. Log entries are made up of binary tag codes, binary parameters and a log message string. Message codes are stored in /system/etc/event-log-tags.
• System log – classes in the Android framework use the system log to separate their messages from application logs. Android classes perform logging using the android.util.Slog class.

Logging Sensitive Data

• Pay attention to how your organization handles user data. In Android, logs are shared to applications with READ_LOGS permission, meaning you can unintentionally leak user data to other apps.
• Keep user data to a minimum. You can do this by avoiding the storage or transmission of Personally Identifiable Information (PII). External components should not access user data if they have no reason to do so.
• If applications need access to data, provide direct access via the client. In this way, data does not have to be transmitted to other servers.

Additional Logs You Should Consider Monitoring

Beyond the common log sources mentioned above, there are many more enterprise systems and security tools that generate logs. All of them might have security implications. However, it’s crucial to prioritize logs for monitoring by analysts, since many organizations have limited security manpower.

Following is a list of most of the common log and information sources you may encounter in your organization. Select the most important sources your security team will regularly monitor.

Security Information and Event Management (SIEM) Logging

SIEM logging is the process of aggregating and monitoring logs for security purposes. SIEM systems are used by security teams to collect event data from IT systems and security tools across an organization, and use it to identify suspicious behavior that might signify a security incident.

Common Security-related Log Events Tracked by a SIEM Include:

• Alert from antivirus or endpoint protection of a malware infection
• Alert from an email system about spam or malicious content in an email
• Firewall alert about blocked network traffic
• Connection to a system from unknown host or IP
• Failed logins, especially if repeated or targeted at critical systems
• Change in user privileges, especially privilege escalation
• Use of new or unknown ports, or protocols that are not secure or violate the security policy

Detecting Security Incidents Using Correlation Rules

Traditionally, SIEMs generated alerts from logs by using correlation rules. A correlation rule specifies a series of events and specific logs values or ranges of values that may indicate a security threat (for example, three or more failed login attempts). Another way to extract security risks from logs is a vulnerability analysis where automated scanners can scan networks for software vulnerabilities that can be targeted by attackers, and some of these scans rely on logs.

Detecting Security Incidents Using Behavioral Analytics

Next-generation SIEM technology uses user and event behavior analytics (UEBA) to establish a behavioral baseline for users and other entities on the network, such as servers, endpoints or applications. The behavioral analytics engine can monitor behavior and identify if it deviates from the baseline, or in other words, if something “looks different”, even if it couldn’t be defined by a strict correlation rule. If deviations are sufficiently large and seem to indicate a security risk, the UEBA system raises an alert. This can help detect insider threats, fraud, and advanced persistent threats (APT), and other sophisticated attack techniques which can easily evade correlation rule-based detection. For an example of a next-gen SIEM with UEBA built in, see Exabeam Advanced Analytics.