Structured logging that actually helps at 3am

01Structured, not prose

A log line that concatenates values into a sentence is readable by humans and useless to a query engine. The same information as structured fields can be filtered, aggregated and correlated - which is what you actually do during an incident.

Log an event name plus fields, not a formatted message. The difference shows up the moment you need to ask how many times something happened for a particular customer in the last ten minutes, which is a question you cannot answer against free text.

• Event name plus typed fields, not an interpolated sentence.
• Consistent field names across services - user_id everywhere, not userId here and uid there.
• Include units in field names where ambiguous, such as duration_ms.
• Never log secrets, tokens or full request bodies containing personal data.

02Correlation is the single most valuable field

One identifier that follows a request across every service, queue hop and background job is worth more than any other logging investment. Without it you are correlating by timestamp and hoping, which fails precisely when the system is busy and the logs are dense.

Generate it at the edge if the client did not supply one, propagate it through headers and message metadata, and include it on every line. Then a single query reconstructs the whole path of a failing request rather than fragments of it.

03Log decisions, not just steps

Logs saying a function was entered and exited tell you the code ran. Logs saying which branch was taken and why tell you what the system decided - and during an incident the question is almost always why something behaved unexpectedly rather than whether it executed.

Log the inputs to a decision and its outcome: the rule that matched, the threshold that was crossed, the fallback that engaged. That is what turns a log from a trace into an explanation.

04Levels, used consistently

Level discipline decays quickly without a shared convention, and once everything is at info or everything is at error, the level carries no information. Agree what each level means and hold to it.

05What not to log

Secrets, tokens, full authorisation headers, card numbers, and personal data beyond what you can justify retaining. Logs are copied, shipped to third parties and retained longer than anyone intends, so treat them as a data store with the same obligations as any other.

Also avoid logging in tight loops. High-volume debug lines in a hot path cost real CPU and can be a meaningful share of latency under load - and they bury the lines that matter when someone is searching.

• Redact at the logging layer, not by remembering at every call site.
• Treat log retention as a data-protection decision.
• Sample high-volume events rather than logging every one.
• Never log inside a tight loop without rate limiting.

06Test the logs like you test the code

The practical exercise is to take a past incident and try to diagnose it from logs alone, without reading the source. Whatever you find yourself missing is the gap to close, and it is usually a decision that was never recorded rather than a step that was not traced.

Doing this after each significant incident steers logging toward the parts of the system that actually fail, which is a far better allocation than uniform verbosity everywhere.