A customer reports that a document never arrived. The application team sees a successful upload. A background process reports no recent errors. The delivery provider has no matching record.
Everyone has data. Nobody yet has an explanation.
For product teams, observability should help reconstruct what happened to a particular task across the relevant boundaries. A collection of healthy-looking component dashboards may not answer that question.
Start with the investigation you need to perform
For our hypothetical document workflow, write down the questions an operator would ask. Was the file accepted? Was it queued for processing? Did processing finish? Was delivery requested, and was receipt confirmed?
Give the important stages an explicit meaning and a way to connect them. The goal is to follow the task, not to record every detail indiscriminately. Avoid placing sensitive document contents or unnecessary personal data into diagnostic logs.
Think about who needs which view. Support may need a clear current state and a reference they can pass to engineering. Engineers may need the sequence of internal events and dependency responses. Those views can serve different purposes without exposing the same level of detail to everyone.
Distinguish a symptom from an explanation
An alert that processing time increased identifies a symptom. It does not prove which dependency caused it. Google's monitoring guidance makes a useful distinction between observing what is broken and investigating why.
Keep that distinction in incident communication. “Deliveries are delayed” may be established before “the provider is at fault” is established. Report the known state, the affected scope, and the next investigation rather than turning an early hypothesis into a fact.
The same discipline should shape dashboards. A successful request to a queue does not necessarily mean the document reached its recipient. Name the measurement according to the stage it actually observes.
Rehearse the explanation before launch
Create a controlled failure in a test environment and ask someone who did not build the feature to investigate. Can they locate the task and identify the last confirmed stage? Can they distinguish a delay from a rejection? Do they know which action is appropriate next?
If the answer requires searching unrelated systems using information the customer does not have, improve the investigation path. That may mean adding a shared reference, a clearer state transition, or a small internal view.
Then connect the explanation to recovery. Identifying a stuck task is useful; knowing whether it can be retried, corrected, or escalated makes the information operational.
The practical standard is straightforward: when a customer says the product did not work, the team should have a reasonable way to find out what happened. Design that capability alongside the customer journey, while the relevant decisions are still fresh.