Priority logs for SIEM ingestion: Practitioner guidance

Introduction

In this publication, the authoring agencies provide cyber security practitioners, with detailed recommendations, on the logs that should be prioritised for ingestion by a Security Information and Event Management (SIEM) platform. The recommendations in this publication should be treated as generic advice; each organisation should tailor the collection, centralisation, and analysis of logs to its specific environment and risk profile. Practitioners should also adopt an approach of gradually building up the number and types of data sources ingested by the SIEM, rather than adding them all at once. The authoring agencies recommend referring to vendor specific guidance where available for information tailored to each operating system.

This publication is therefore generally intended for the team/s responsible for establishing and maintaining their organisation’s SIEM. However, a hunt or blue team can also use its guidance on priority event IDs as a starting point to search for adversary activity or build a baseline of business-as-usual activity on the organisation’s network

Note on terminology: All SIEM platforms have a log ingestion function. Some Security Orchestration, Automation, and Response (SOAR) platforms also perform this function, or have an in-built SIEM. If your organisation uses a SOAR platform with an in-built SIEM, the following recommendations will be relevant to its ingestion of logs.

This series of documents

This publication is one of three in a suite of guidance on SIEM/SOAR platforms:

• Implementing SIEM and SOAR platforms: Executive guidance
  This document is primarily intended for executives. It defines SIEM/SOAR platforms, outlines their benefits and challenges, and provides broad recommendations for implementation that are relevant to executives.
• Implementing SIEM and SOAR platforms: Practitioner guidance
  This document is intended for cyber security practitioners. In greater technical detail, it defines SIEM/SOAR platforms, outlines their benefits and challenges, and provides best practice principles for implementation.
• Priority logs for SIEM/SOAR ingestion: Practitioner guidance
  This document is again intended for cyber security practitioners and provides detailed, technical guidance on the logs that should be prioritised for SIEM ingestion. It covers log sources including Endpoint Detection and Response tools, Windows/Linux operating systems, and Cloud and Network devices.

This guidance should be read alongside Best practices for event logging and threat detection, which provides high-level recommendations on developing a logging strategy.

Risk considerations

As above, logging decisions should be based on the organisation's specific environment and risk profile. While the below recommendations provide a starting point, it is critical that organisations model their threats and risks and select data sources most relevant to their risk profile. For each data source, your organisation should assess:

• its purpose or use case the authoring agencies discourage logging for the sake of logging
• its prioritisation – higher priority data sources should be ingested into new SIEM deployments first and their health should be regularly checked. This document provides a suggested order of prioritisation by broad category of data source.
• the volume of logs it may generate. For example, the volume of firewall or Domain Name System (DNS) logs may overshadow the importance of the information received.
• its analytical value. For example, high volume data sources may be queried for anomalies in timing. They may also be queried for correlations against other data sources (for instance, analysing high volume firewall logs against threat intelligence-identified malicious IP addresses).

Architectural considerations

A key premise of this publication is that the architecture of logging involves a two-stage process:

1. log creation, collection and transfer to a centralisation point
2. ingestion of those logs by the SIEM, either directly from the source or from the centralisation point.

Organisations are likely to have legal or regulatory reasons for logging a variety of sources and sending these logs to a centralised location. However, the authoring agencies strongly discourage using a SIEM as the central repository for all logs. A SIEM should only be used for centralisation of specific security logs according to the organisation’s risk profile.

Prioritised logging list

This document presents tables of logging events in a loose prioritisation by category of data source. The logging tables are not intended to be complete nor is the order applicable to every organisation. The authoring agencies recommend that organisations treat the following prioritisation as a starting point for a typical enterprise network environment. Organisations may need to account for reliability, the visibility each log or log type provides, the potential performance impacts of ingest, and the organisational cost of maintaining as well as analysing this data, them. Organisations may also need to adjust the prioritisation based on their unique threats, capabilities and needs.

For Active Directory (AD) event IDs, group policy changes have been included as an additional reference at the end of this document.

Priority logs for SIEM ingestion footnote legend

The following authoring agency documents are referenced within this document and presented as footnotes throughout the document:

1. CCST - Cloud Computing Security for Tenants | Cyber.gov.au
2. CCSCSP - Cloud Computing Security for Cloud Service Providers | Cyber.gov.au
3. DMADC – Detecting and mitigating Active Directory compromises | Cyber.gov.au
4. WELF - Windows Event Logging and Forwarding | Cyber.gov.au
5. HMWW - Hardening Microsoft Windows 10 and Windows 11 Workstations | Cyber.gov.au

Detailed logging guidance

The following categories of data source are loosely in order of prioritisation.

1. Endpoint detection and response (EDR) logs

2. Network device logs

3. Microsoft Domain Controller

Please see additional references at the end of this document for the group policy changes relevant to the following event IDs.

4. Active Directory (AD) and Domain Service security logs

Please see additional references at the end of this document for the group policy changes relevant to the following event IDs.

5. Microsoft Windows endpoints logs

Please see additional references at the end of this document for the group policy changes relevant to the following event IDs.

6. Virtualisation system logs

7. Operational technology logging

Operational Technology (OT) logging integration into a SIEM can be challenging due to the specialised nature of OT systems, which are often vendor-specific and segmented from the environments where the SIEM is typically located. The authoring agencies also note that OT devices often come with limited logging. Where possible, it is recommended that OT devices enable logging and then send and store logs in a centralised location. While implementing a dedicated SIEM specific to the OT environment may be feasible, it would also require staff to develop familiarity with the two systems and event types.

Industrial Control System (ICS) monitoring offers a solution that can safely ingest, interpret, and enrich OT data before either forwarding it to a SIEM or storing it in a central repository. Additionally, these products can monitor OT networks, assets, parse the OT-native protocols, and generate additional logs with necessary details and contextual metadata for events that are going to the SIEM.

In cases where security supersedes other requirements, organisations may implement unidirectional gateways or data diodes to securely transmit log data from the OT environment to the IT SIEM without exposing the OT network to external threats.

Due to the safety-critical risks, high speed and deterministic communications of the messaging, organisations should take a conservative approach to logging directly from OT assets and test any logging solution thoroughly before deployment to avoid impacting operations.

8. Cloud platform logging ^[103][104]

Cloud services may not be enabled by default. Every application may have its own logging format, or no logging at all. The recommendations below are only a sample of what may be logged; organisations should seek out advice from their cloud service provider and cloud application provider for security logging that meets the organisation’s security requirements and risk profile.

See Best practices for event logging and threat detection for more on this.

Logging priorities for cloud computing

The authoring agencies recommend organisations adjust event logging practices in accordance with the cloud service that is administered, whether infrastructure-as-a-service (IaaS), platform-as-a-service (PaaS), or software-as-a-service (SaaS) are implemented. For example, IaaS includes a significant amount of logging responsibility on the tenant, whereas SaaS places a significant amount of the logging responsibility on the provider. Therefore, organisations should coordinate closely with their cloud service provider to understand the shared-responsibility model that is in place, as it will likely influence their logging priorities. Logging priorities may also be influenced by different cloud computing service models and deployment models (public, private, hybrid, community). Where privacy and data sovereignty laws apply, logging priorities may also be influenced by the location of the cloud service provider’s infrastructure.

See the National Security Agency’s Manage Cloud Logs for Effective Threat Hunting^[105] guidance for additional information.

Organisations should prioritise the following log sources in their use of cloud computing services:

• critical systems and data holdings likely to be targeted
• internet-facing services (including remote access) and, where applicable, their underlying server operating systems
• use of the tenant’s user accounts that access and administer cloud services
• logs for administrative configuration changes
• logs for the creation, deletion and modification of all security principals, including setting and changing permissions
• authentication success and/or failures to third party services (for example Security Assertion Markup Language (SAML)/ Open Authorization (OAuth))
• logs generated by the cloud services, including logs for cloud Application Programming Interface (API/s), all network-related events, compliance events and billing events.

Amazon Web Services logs

Critical Azure service and app logs

Google Cloud Platform (GCP) logs

Google Workspace (GWS) logs

9. Container logs

10. Database logs

11. Mobile device management

12. Windows DNS server analytic event logs

13. Linux endpoint auditing logs

14. Apple MacOS endpoint logs

Reference and resource annex

Active directory group policy changes

The following table lists the Group Policy changes required to generate the specific event IDs within log sources.

Windows Endpoint Group Policy Changes

The following table lists the Group Policy changes required to generate the specific event IDs within log sources.

Domain Controller Group Policy Changes

The following table lists the Group Policy changes required to generate the specific event IDs within log sources.

Footnotes

1. For more, see DMADC Table 2, Table 6, Table 10, Table 14.
2. For more, see DMADC Table 1, Table 10.
3. For more, see WELF.
4. For more, see DMADC, Table 4.
5. For more, see WELF.
6. For more, see WELF.
7. For more, see WELF.
8. For more, see DMADC Table 15.
9. For more, see DMADC Table 4.
10. For more, see DMADC Table 2, Table 15.
11. For more, see DMADC Table 3.
12. For more, see DMADC Table 6.
13. For more, see DMADC Table 6, Table 18.
14. For more, see DMADC Table 6, Table 18.
15. For more, see DMADC Table 7.
16. For more, see DMADC Table 7.
17. For more, see DMADC Table 6.
18. For more, see DMADC Table 6.
19. For more, see DMADC Table 6, Table 19.
20. For more, see DMADC Table 6, Table 19.
21. For more, see HMWW.
22. For more, see DMADC Table 5, Table 8.
23. For more, see HMWW.
24. For more, see HMWW.
25. For more, see DMADC Table 8, Table12. See also Detecting using Canaries.
26. For more, see DMADC Table 1, Table 2.
27. For more, see DMADC Table 9.
28. For more, see DMADC Table 12.
29. For more, see DMADC Table 12, Table 20.
30. For more, see DMADC Table 12, Table 20.
31. For more, see DMADC Table 12, Table 20.
32. For more, see DMADC.
33. For more, see DMADC Table 12, Table 20.
34. For more, see DMADC Table 12, Table 20.
35. For more, see DMADC Table 5.
36. For more, see DMADC Table 3.
37. For more, see HMWW.
38. For more, see DMADC Table 2, Table 3.
39. For more, see WELF.
40. For more, see WELF.
41. For more, see DMADC Table 3, Table 4, Table 5, Table 11.
42. For more, see DMADC.
43. For more, see DMADC Table 11.
44. For more, see DMADC Table 3.
45. For more, see HMWW.
46. For more, see DMADC Table 8, Table 16.
47. For more, see HMWW.
48. For more, see DMADC Table 16.
49. For more, see DMADC Table 6.
50. For more, see DMADC Table 16.
51. For more, see WELF.
52. For more, see WELF.
53. For more, see DMADC Table 16.
54. For more, see WELF.
55. For more, see DMADC Table 16, Table 18.
56. For more, see DMADC Table 16.
57. For more, see DMADC Table 8, Table 12. See also Detecting using Canaries.
58. For more, see DMADC Table 16.
59. For more, see HMWW.
60. For more, see DMADC Table 16.
61. For more, see WELF.
62. For more, see DMADC Table 6, Table 10, Table 14.
63. For more, see DMADC Table 1, Table 10.
64. For more, see DMADC Table 3.
65. For more, see WELF.
66. For more, see DMADC Table 1, Table 15.
67. For more, see WELF.
68. For more, see HMWW.
69. For more, see DMADC Table 1, Table 15.
70. For more, see DMADC Table 6.
71. For more, see DMADC Table 6.
72. For more, see DMADC Table 6.
73. For more, see DMADC Table 7.
74. For more, see DMADC Table 7.
75. For more, see DMADC Table 6.
76. For more, see DMADC Table 6.
77. For more, see DMADC Table 6.
78. For more, see DMADC Table 6.
79. For more, see WELF.
80. For more, see WELF.
81. For more, see WELF.
82. For more, see DMADC Table 5, Table 7, Table 8, Table 13, Table 14, Table 15, Table 16.
83. For more, see DMADC Table 5, Table 7, Table 8, Table 13, Table 14, Table 15, Table 16.
84. For more, see WELF.
85. For more, see DMADC Table 6, Table 7, Table 8, Table 12, Table 13, Table 14, Table 15, Table 16.
86. PRC state-sponsored actors compromise and maintain persistent access to U.S. critical infrastructure | Cyber.gov.au
87. For more, see WELF.
88. For more, see DMADC Table 3, Table 4, Table 5, Table 11.
89. For more, see WELF.
90. For more, see DMADC Table 3, Table 4, Table 5, Table 11.
91. For more, see DMADC Table 8.
92. For more, see DMADC Table 8, Table 16.
93. For more, see DMADC Table 8, Table 16.
94. For more, see DMADC Table 16.
95. For more, see WELF.
96. For more, see WELF.
97. For more, see DMADC Table 8, Table 16.
98. For more, see DMADC Table 8, Table 16.
99. For more, see WELF.
100. For more, see WELF.
101. For more, see PRC State-Sponsored Actors Compromise and Maintain Persistent Access to U.S. Critical Infrastructure
102. For more, see Joint Guidance: Identifying and Mitigating Living Off the Land Techniques
103. For more, see CCSCSP.
104. For more, see CCST.
105. https://media.defense.gov/2024/Mar/07/2003407864/-1/-1/0/CSI_CloudTop10-Logs-for-Effective-Threat-Hunting.PDF
106. For more, see DMADC Table 13.
107. For more, see DMADC Table 13.
108. For more, see DMADC Table 13.
109. For more, see DMADC Table 13.
110. For more, see DMADC Table 13.
111. For more, see DMADC Table 13.
112. For more, see DMADC Table 13.
113. For more, see DMADC Table 13.
114. For more, see DMADC Table 13.