Ten thousand exceptions.
Four issues, each with an owner.
Datadog Error Tracking groups the exceptions from your traces, logs and browser sessions into issues, tells you when one came back, and names the commit that probably caused it. We implement it on the telemetry you already send, and we keep every issue owned.
Quick facts
Related
- APM, the trace each backend issue is attached to
- Real User Monitoring, the browser and mobile errors
- Log Management, the indexed logs that become issues
The exception, grouped with its siblings, next to the commit that introduced it
The error rate on the service is steady. It has been steady for a year, which is why nobody reads it. Inside that flat line are a few causes repeating thousands of times, and one new cause that started with Tuesday's deploy. Error Tracking separates them.
Grouped by cause, not by count. Similar stack traces become one issue with a first-seen time, an occurrence graph and the number of users it reached, so the new cause is visible under the old noise.
The commit that probably did it. With source control connected, each issue carries the suspect commit and the code owner, so the issue goes to a person rather than a channel.
Regressions, flagged as such. An issue marked resolved that appears again is reopened as a regression, not filed as new. The fix that did not hold is the first thing in the list.
The variables at the moment it threw. Exception Replay captures the local values on a backend exception, so reproducing it is reading rather than guessing.
Included until it is not, an issue list nobody owns, and a resolved state that means muted
How it bills. Error Tracking is included for errors that arrive inside APM traces and RUM sessions. Errors from logs need Log Management with those logs indexed, and Datadog also bills them on the Error Tracking meter, which counts the monthly volume of errors. Standalone errors are billed on the same meter: backend errors sent without APM, and frontend errors sent outside a RUM session. A daily rate limit caps how many billable errors are kept. Datadog publishes the rates. The lever is which source each service uses and how many errors it is allowed to send.
The configuration traps. The product lights up the moment APM or RUM is on, which is also the moment it becomes a list nobody has looked at. Without source control connected there is no suspect commit and no owner, so every issue is everyone's. Without a team assignment, the monitor for new issues pages a shared channel. An issue marked resolved to clear the view stays resolved until it regresses, at which point it is new to everyone. A service with no exception handling at the boundary reports one issue for every distinct URL that failed.
Field noteThe most common gap we see is ownership. The issue list is long, every issue is unassigned, and the people reading it treat it as a weather report rather than a queue.
Field noteThe pattern we keep finding is a single issue with thousands of occurrences that is the error-handling path working as designed. It sits at the top of every view, and the new issue beneath it with a fraction of the count is the one that matters.
Field noteError Tracking for logs is left on the default catch-all rule. Every error-level log that carries an error kind or a stack trace becomes an issue, including the ones that are informational in all but name, and each one counts on the error meter. The signal is buried under its own volume.
Field noteRegressions are not watched. A fix ships, the issue is resolved, and the regression two releases later is reopened into the same review queue as the new issues. Nobody opens its history, and it is fixed again as if it were a fresh bug.
Connect the repository. Assign the owner. Then let the list get shorter.
We start with a Datadog HealthScan. It is read-only and fully automated, and it reports issues per service, how many are unassigned, which source each service uses, and which monitors exist, before anything changes. Then the sequence is about ownership before volume.
- Decide the source per service. Traces for services with APM, RUM for the browser and mobile apps, logs only where neither exists and the logs are indexed anyway.
- Connect source control. GitHub or the equivalent, so each issue carries the suspect commit and a code owner from the day it is created.
- Assign the teams. Every service mapped to the team that owns it, so a new issue is routed to a person and the monitor pages the right rota.
- Tune the noise. Expected errors, handled at the boundary, are ignored or excluded at the source so the list is the unexpected ones.
- Build the monitors. New issue per service, high-volume issue, and regression, routed to the owning team, with thresholds set from a fortnight of real traffic.
- Hand over, in writing. Which services use which source, who owns each, which issues were ignored and why, and the issue count on the day we finished.
Access is admin on the Datadog organisation plus the source control integration, read-only. We do not need write access to any repository, and we do not ask for it.
A list that is read is a list that stays short
New services ship with no team. A service moves from logs to traces and its issues split in two. A regression is filed as new because nobody linked it. Managed Observability keeps Error Tracking owned, tuned and worth reading.
Every month, we review the sources. Which services send errors from which source, standalone volume against the meter, and whether a service moved from logs to traces since last month.
Ownership follows the estate. A new service appears in the review and gets a team before its first issue. A team that was split or renamed is remapped.
Monitors are kept honest. A new-issue monitor that fires on handled errors is retuned. A regression that was treated as new is linked back to its history.
The incident is ours to take. Under Critical Support a new-issue or high-volume monitor pages our engineers, 24x7, and they read the trace, the replay and the commit rather than forward the alert.
We operate the stack. You own the product.
Error Tracking is a queue, and a queue is only useful when somebody owns each item in it. We set up the ownership, tune out the noise, and keep both current as the services change. The source integration, the team mapping and the monitors live in your Datadog organisation, and you can take them in-house whenever you choose.
Datadog is our proof, not our headline. Datadog certified one managed provider globally. We are it. The Powered by Datadog accreditation means Datadog's own engineers reviewed how we instrument, alert, respond and report. We are also a Datadog Advanced Partner, ISO 27001 certified and Cyber Essentials Plus certified, with offices in Cardiff, London and Dublin.
Datadog Error Tracking, answered plainly
How is Datadog Error Tracking billed?
Errors that arrive inside APM traces and RUM sessions are included with those products. Errors from logs require Log Management with the logs indexed, and Datadog also bills them on the Error Tracking meter, which counts the monthly volume of errors. Standalone errors are billed on the same meter: backend errors sent without APM, and frontend errors sent outside a RUM session. A daily rate limit caps how many billable errors are kept. We do not publish prices. Which source each service uses, and how much it sends, is what we tune.
Do we need Datadog APM to use Error Tracking?
No. Error Tracking works from RUM for browser and mobile apps, from indexed logs, and from a standalone backend SDK that sends errors without traces. APM gives the richest issue, with the trace and Exception Replay attached, so where APM is already in place it is the source we choose.
How does Datadog Error Tracking decide what is one issue?
By grouping errors with the same type, message pattern and stack trace shape into a single issue, so a thousand occurrences of one cause are one row. Each issue keeps its first-seen time, occurrence graph and affected users. An issue marked resolved that appears again is reopened as a regression rather than created new.
Can Datadog Error Tracking tell us who should fix it?
With source control connected, each issue carries the commit that probably introduced it and the code owner from the repository. With Teams in place, the service's owning team is on the issue and the monitor pages their rota. Without either, every issue is unassigned, which is the state we most often find it in.
What does Critical Cloud do with Error Tracking every month?
Managed Observability reviews the sources and the standalone volume, assigns new services to a team, retunes monitors that fire on handled errors, links regressions back to their history, and reports what changed. Under Critical Support our engineers take the new-issue incident, 24x7.
- Datadog products: every product, and how we implement and run each one
- Datadog APM: the traces the backend issues come from
- Real User Monitoring: the browser and mobile errors, with the session behind them
- Datadog HealthScan: read-only, fully automated, and the first thing we run
Fewer issues. Each with a name on it.
Tell us which services throw the most. We will show you what Error Tracking groups them into today, who owns each, and what it takes to make the list short.