That is the shape of the log problem too. Nobody decides to put a customer's bank details in a log line. A developer logs the request body to debug a failing endpoint and never removes it. A URI carries a whole customer record because the framework encoded it that way. A payment webhook is logged in full because the integration was flaky and someone needed to see what arrived. Each decision was reasonable. The log is now a copy of your most sensitive data, kept longer than the database keeps it, read by more people than the database allows, and shipped to a third party.
Why the scanner is off
Datadog's Sensitive Data Scanner exists for exactly this. It scans log content, APM span attributes, RUM and event attributes for patterns such as card numbers, bank identifiers, email addresses and credentials, and it can redact, partially redact, hash or mask the match before it is indexed. It tags what it finds, so a detection can page someone. It can run in Datadog's backend, or inside your own environment through Observability Pipelines, so the sensitive value never leaves your estate.
In every estate where we have checked, it was disabled. The reason is never a decision. It is a product that has to be switched on, scoped and tuned, and the team that set up the account was setting up logging, not data protection. The scanner sits in a menu nobody had a reason to open. Then the estate grows, the log sources multiply, and the first anyone hears of it is when a data protection officer asks a question that takes three weeks to answer.
The trap inside the fix
Switch the scanner on in redaction mode and the problem appears to go away. The bank details are replaced with a token at ingest, the log platform never stores them, and the dashboard turns green. What has not happened is any change to the application that wrote the bank details into the log in the first place.
That matters for two reasons. The leak still exists everywhere the redaction does not run: in local developer logs, in the container's stdout, in whatever the previous log pipeline shipped to before Datadog. And the people who could fix the source never find out, because the silent fix removed the evidence that there was a problem. A regex that quietly repairs a leak is a control that hides a finding.
The order matters. Detection first, so you learn which services leak and what. Then the fix at source. Then redaction, kept on permanently, as the belt that catches the next developer's debug statement.
Retention makes it worse
A database row is deleted when the customer leaves. The log line that copied it is kept for whatever the index retention was set to on day one, which is usually the default, and the archive beyond that is kept because storage is cheap and deleting feels risky. The effect is that your log platform holds personal data for longer than your retention policy allows, and holds it in a form that was never designed for a subject access request.
This is also where cost and compliance pull in opposite directions. Audit logs need to be kept for a long time and searched occasionally. Indexing them for fifteen months is expensive, so they go to an archive, and then the first time an auditor asks a question about March, the answer is a rehydration job and a two-day wait. Datadog's Flex Logs tier exists for this middle case: long retention, searchable, without paying for live indexing. It is the difference between retrievable and searchable, and that difference is what an audit clock measures.
What to switch on, in order
- Classify the sources. For each log source, write one line: what personal or payment data could appear in it, and why. If the answer is "none", say so and move on.
- Run the scanner in detection. Turn Sensitive Data Scanner on for logs with the rules that match your data, in detection only, for a week. The tags tell you which services are leaking and how much. This is the finding.
- Fix at source. Remove the debug logging, stop encoding records into URIs, log a webhook's identifier rather than its body. Each fix is small. The list is the work.
- Switch to redaction and keep it on. Hash where you need to correlate, redact where you do not. If data residency matters, run the processor in Observability Pipelines so the value is removed before it leaves your environment.
- Set retention per source, not per account. Application logs short. Audit logs to Flex Logs for as long as the obligation says, searchable. Nothing kept because deleting felt risky.
- Write the sentence for the DPO. "These sources can contain personal data; the scanner redacts these fields at ingest; retention is set to this; here is the evidence it ran." If you cannot write it, you are not finished.
Where we fit
Critical Cloud runs this as part of Managed Observability. We classify the sources, run the scanner through detection and into redaction, set the retention tiers so audit logs stay searchable without indexing everything, and keep the evidence that the controls were on. One payments customer came to us for the cost of their logs and left with a retention design their compliance team could sign. We operate the controls. You keep the decisions about what your logs may contain.
We will not tell you that you are compliant. We operate and evidence the runtime controls that support your obligations, and the log platform is one of the places those controls most often turn out to be off.
Frequently asked questions
What does Datadog Sensitive Data Scanner actually do?
It scans log content, APM span attributes, RUM and event attributes for patterns such as card numbers, bank details, email addresses and credentials, and then redacts, partially redacts, hashes or masks the match before it is indexed. It tags what it finds so you can alert on it. It can run in the Datadog backend or inside your own environment through Observability Pipelines, so sensitive data is removed before it leaves your estate. It is a product you switch on and configure; an untouched account does not have it running.
If the scanner redacts the data, is the problem solved?
No. Redaction at ingest stops the data being stored and read in the log platform, which matters. It does nothing about the application that wrote the bank details into the log line in the first place, and it can hide that fact from the people who should fix it. Run the scanner in detection first so you know which services leak, fix them at source, and keep redaction on as the belt that catches the next one.
Does Critical Cloud handle log data protection as part of Managed Observability?
Yes. Classifying log sources, running Sensitive Data Scanner in detection and then redaction, setting retention per source so audit logs stay searchable without indexing everything, and giving you the evidence that the controls were on are standard parts of Managed Observability. We operate the controls and you keep the decisions about what your logs may contain.
Sources
- Datadog Docs, Sensitive Data Scanner: scope, actions, Observability Pipelines processor and sampling.
- Datadog Docs, Flex Logs: retention tiers and intended use.
- Malwarebytes, AI chat app leak exposes 300 million messages tied to 25 million users, 9 February 2026.
- Wiz Research, Hacking Moltbook: AI social network reveals 1.5M API keys, 31 January 2026, for the private messages that carried third-party credentials.
- The "disabled in every estate where we have checked" pattern is from our own assessments. We publish the pattern, not the counts.