Logging for the incident you have not had yet

Most application logs are written for the developer who is debugging right now. Incident logs are written for a stranger reading them six weeks later under pressure, possibly a customer's auditor, possibly your own future self at 2am. Those are different jobs, and the second one is the one almost nobody designs for until they need it.

Here is the shape of the thing that works, at a size a small team can actually build and maintain.

1. Start from the questions, not from the log lines

An incident is not a search problem, it is an answering problem. There are five questions, and every incident is some arrangement of them:

1. Who did it? Not the display name, the stable identifier.
2. What did they touch? Which specific records or resources, not which endpoint.
3. When, in a timeline you can line up across services?
4. From where? IP, device, session, API key.
5. What changed as a result, and is it still that way?

If your logging cannot answer those five for a given account over a given window, the volume of what you are storing is irrelevant. Work backwards from the questions and you get a much smaller, much more useful set of events than "log everything and buy a bigger plan".

2. The minimum event set

For a small SaaS product, this is the list I would want in place before anything else:

• Authentication. Success and failure, with the reason for failure. Include MFA challenges issued, passed, and bypassed. Include session creation and, importantly, session termination.
• Authorization denials. Successes are noise at first. Denials are the signal, because a compromised account probing for scope generates them in patterns a human never does.
• Privilege and role changes. Any change to who can do what, including group membership, including changes made by automation.
• Bulk read and export. The single most under-logged event in small products. One record read is a user working. Ten thousand records read is either a report or an exfiltration, and you want to be able to tell which.
• Configuration and secret changes. Rotations, new API keys, changed webhook targets, changed SSO settings, new OAuth grants.
• Integration activity. Which token acted, on whose behalf, against which tenant. Third party integrations are where small products quietly get very large blast radiuses.
• Admin actions taken on behalf of a customer. Support impersonation is a normal feature and a genuine risk. Log it as its own event type, not as the customer's own activity.

That list is deliberately boring. Boring is what you want. Compare it against your own product tonight and you will probably find two or three of these are missing entirely.

3. Fields that make logs joinable

A log that cannot be joined to another log is an anecdote. Four fields do most of the work:

• A stable actor identifier that never changes, alongside the human readable one. Emails get changed, users get renamed, and your six week old logs become unsearchable.
• A request or trace identifier propagated across every service and job that handles the request, including async workers. This is the difference between a timeline and a pile.
• A tenant identifier on every event in a multi tenant system. You will need to answer "what happened in this one customer's account" and you will need to answer it without reading anyone else's data.
• UTC timestamps in a single format, ideally RFC 3339. Mixed timezones and mixed formats are how a reconstruction quietly goes wrong rather than loudly failing.

Add the outcome of the action, not just the attempt. "Delete requested" and "delete completed" are different facts.

4. What to keep out

This is the part that gets skipped, and it is the part that turns a log store into a liability.

Logs inherit the sensitivity of whatever you put in them. Full request and response bodies will eventually carry personal data, health data, payment data, or a token. Once that is in a system with wide read access and long retention, you have built a second copy of your most sensitive data with none of the controls you put around the first copy.

Practical rules:

• Redact at the point of emission, not in the pipeline. A redaction step downstream means the raw value existed somewhere, and somewhere is usually a disk.
• Log identifiers, not values. card_id not the card. user_id not the email. document_id not the contents.
• Allowlist what gets logged from a payload rather than denylisting what does not. Denylists fail silently the moment someone adds a field.
• Treat secrets as a build time problem. Scan for token shaped strings in log output in CI, the same way you scan for them in commits.
• Say in writing which categories of personal data are permitted in logs. If nobody has written it down, the answer in practice is "all of them".

5. Retention is a decision, not a default

Pick the number on purpose. Two horizons, two different reasons:

• Hot, searchable. Long enough to investigate something you notice. Thirty days is short for this in practice, because breaches are routinely discovered long after they start.
• Cold, retrievable. Long enough to satisfy the obligation you actually have. That comes from your contracts, your regulator, and your customers' expectations, not from a blog post.

Write down both numbers and the reason for each. An auditor asking about log retention is not really asking for the number. They are asking whether anyone decided it.

6. Integrity, or why the attacker cleans up after themselves

If the same credentials that run your application can delete your logs, then the logs describe only the incidents your attacker did not care about.

Cheap version of getting this right, which is within reach for a team of five:

• Ship logs out of the environment that produced them, to a separate account or project with its own access boundary.
• Make the destination append only, with object lock or the equivalent on the archive.
• Make deletion a separate, logged, alerted privilege held by a small number of named people.
• Alert on the audit log going quiet. Silence is an event.

7. Test it before you need it

Here is a two hour exercise worth more than another dashboard.

Pick a plausible scenario. An employee laptop is compromised and their session token is stolen on a Friday night. On Monday, open only your log tooling, no code reading, no database queries, and try to answer the five questions from the first section. Time yourself and write down each point where you had to guess.

The gaps you find will not be exotic. They will be things like bulk exports not being logged as their own event, or the support impersonation path writing events under the customer's identity, or worker jobs dropping the trace identifier. Every one of those is a small ticket now and a bad week later.

Do this once a quarter, change the scenario, and let it drive your backlog. That single habit is most of the difference between a logging setup that is expensive and one that is useful. It also happens to produce, almost as a by product, the evidence that the SOC 2 common criteria on monitoring and evaluating security events expect you to have.

8. Two references worth the time

• NIST SP 800-92, Guide to Computer Security Log Management. Old, still the clearest articulation of the problem.
• The OWASP Logging Cheat Sheet, for what to log and what never to.

---

I work with small SaaS teams at IRM Consulting & Advisory, and the logging conversation above is one I have had more times than any other. The companion conversation, about what should never end up in a log in the first place, usually turns into data security and privacy reviews instead. Flagging the affiliation so you can weigh the advice accordingly.

I am curious about one thing in particular. For those of you who have actually run the exercise in the last section, what was the gap you did not expect to find?