Detection Engineering Starts With the Logs You Decided to Keep

Detection content is written against telemetry. If the telemetry does not exist, the rule cannot be written, tested or tuned. The practical ordering is therefore log coverage first, detection second.

The events that carry the most weight

Authentication outcomes, including failures and successes from remote access and identity providers, are the backbone of intrusion detection. Account changes, such as new administrative roles, new credentials and modified federation settings, describe the persistence stage.
Cloud control plane logs record who changed a security group, a bucket policy or a role. Kubernetes audit logs record object creation, exec into a pod and changes to admission configuration.
Both endpoint process creation and network flow data fill gaps that identity logs cannot see, particularly after a workload is already running.

What makes a log usable

A log is only usable if it identifies the actor, the action, the target and the result, and if those fields can be joined to the same identity across systems. Timestamps must be consistent and timezone-aware, or events will be ordered incorrectly during an investigation.
Retention matters more than people expect. Many incidents are discovered weeks after the activity, and a thirty-day window removes the beginning of the story.

Tuning without blinding yourself

Alert volume is the reason detection programmes stall. The fix is not to disable rules but to add context: enrich alerts with asset criticality and identity privilege, then route the high-context ones to responders and aggregate the rest.

References

• NIST SP 800-92, Guide to Computer Security Log Management: https://csrc.nist.gov/pubs/sp/800/92/final
• Microsoft Learn, Security and governance for DevOps: https://learn.microsoft.com/en-us/security/benchmark/azure/mcsb-v2-logging-threat-detection
• MITRE ATT&CK knowledge base: https://attack.mitre.org/