The database is not a black box.
It just looks like one from APM.
Datadog Database Monitoring shows every normalised query on Postgres, MySQL, SQL Server, Oracle, MongoDB and more, with its explain plan and the service that sent it. We implement it, keep the host and query meters honest, and run it every month.
Quick facts
Related
- APM, the trace that shows the query was the slow span
- Infrastructure Monitoring, the host meter the Agent host lands on
- Data Streams Monitoring, the queue in front of the database
The query, the plan it ran under, and the service that sent it
The checkout service is slow. APM says it spent the time in the database. The database says it is idle and fine. Database Monitoring is the layer in between: it shows which normalised query took the time, how often it ran, what the planner decided to do with it, and which service asked.
Every query, normalised and ranked. Literal values are stripped, so a thousand variations of one SELECT appear as one row with its execution count, total time and rows returned, across every host that ran it.
Explain plans without a DBA on call. The plan the database chose for a sampled execution, with the sequential scan or the missing index visible, and the change in plan over time when a table grew past what the planner expected.
Who is holding the lock. Active sessions, blocking and waiting queries, and the host metrics beside them, so a stuck migration is found before the queue of requests behind it is.
The service that ran it. With APM in place, each query links to the calling service and the trace it ran inside. The argument about ownership gets shorter. Bits Database Optimization, in Preview, proposes fixes from the same data for an engineer to review.
Two meters, a replica for every reader, and an explain function that was only created once
How it bills. Database Monitoring bills on two meters. Database hosts are counted hourly and billed on the high-water mark of the lower 99 per cent of usage, so a replica set that doubles for a sustained period sets the bill. A second meter counts the normalised queries being tracked at any one time, above an allotment that comes with each database host. The Agent host that collects from managed databases is an infrastructure host as well. Datadog publishes the rates. We publish the configuration that drives them.
The configuration traps. Managed databases do not run an Agent, so one Agent on a separate host collects from them over the network. That host becomes the single point of collection and, when it is undersized, the first thing to drop samples. The explain plan function has to exist in every database on the instance, not just the one the setup guide was tested against. The Agent is configured for the primary, and the replicas, where the reads happen, are left out of its configuration. Each is a quiet failure. The dashboards fill with something and nobody notices what is missing.
Field noteThe most common gap we see is the replica. The Agent is configured with the primary's endpoint, each replica needs an entry of its own and never got one, the application reads from the replicas, and the slowest queries in the estate never appear in the table.
Field noteThe pattern we keep finding on managed Postgres is explain plans that work in one database and silently return nothing in the others. The function was created once, during setup, and the instance hosts several databases.
Field noteRead replicas are added for the month-end reporting run and kept for the days it takes. Each one is a Database Monitoring host for every hour it exists, days are far more than the top one per cent of hours the billing method ignores, and the bill reflects an estate that only existed at month-end.
Field noteDatabase Monitoring is on, APM is on, and the two never meet, because database propagation was never switched on in the tracer. The query is visible, the service that sent it is a guess, and the ownership argument runs for a week.
The user, the extension and the Agent host, before any dashboard
We start with a Datadog HealthScan. It is read-only and fully automated. Before we change anything, it reports which hosts send Database Monitoring data, which engines they run, and the sample rates. It also shows where explain plans are missing and what the host count has done over the quarter. Then the sequence is the same on every engine.
- Inventory the databases. Every engine, instance, replica and managed cluster, with the service that depends on it and whether it matters at 2am.
- Create the datadog user and the extensions. The read-only user, pg_stat_statements or the engine's equivalent, and the explain function in every database, scripted so the next instance gets the same.
- Place the Agent host. One or more Agent hosts for the managed instances, sized for the number they poll, inside the database network.
- Switch on the calling-service link. Database propagation in the tracer, so each query appears with the service and the trace that ran it.
- Set the sample rates and query limits. Query samples and explain plan collection tuned for the engine's load, and the tracked-query count agreed before the meter decides it.
- Build the monitors and hand over. Long-running queries, blocking sessions, replication lag and connection saturation, with thresholds from real data. The handover note records the host and query counts on the day we finished.
Access is admin on the Datadog organisation, a read-only database user we create with you, and a host for the Agent inside the database network. We do not need write access to any database, and we do not ask for it.
The schema changes. The replicas scale. The plan that was fine in spring is a sequential scan by autumn.
A table grows past the point where the planner's choice made sense. A new instance appears with no datadog user. A migration moves the heaviest database to a managed service the Agent host cannot reach. Managed Observability keeps Database Monitoring matching the databases it watches.
Every month, we review the meters. Database host count against the high-water mark, tracked queries per host, replica hours. A replica set that grew gets a reason or a schedule.
Coverage follows the estate. A new instance, a new database on an old instance, or a move to a managed service is found in the review and brought under the same user, extension and Agent host.
Plans are kept honest. The queries with the worst plan regressions are listed each month with the index or the rewrite we recommend, for the team that owns the schema to decide.
The incident is ours to take. Under Critical Support a blocking-query or replication monitor pages our engineers, 24x7, and they find the session, the lock and the service behind it rather than forward the alert.
We operate the stack. You own the product.
A Database Monitoring setup is a set of grants, extensions and sample rates that has to be repeated on every instance the estate adds. We script it, document it and keep it current. The datadog user, the Agent host and the monitors live in your accounts, and you can take them in-house whenever you choose.
Datadog is our proof, not our headline. Datadog certified one managed provider globally. We are it. The Powered by Datadog accreditation means Datadog's own engineers reviewed how we instrument, alert, respond and report. We are also a Datadog Advanced Partner, ISO 27001 certified and Cyber Essentials Plus certified, with offices in Cardiff, London and Dublin.
Datadog Database Monitoring, answered plainly
How is Datadog Database Monitoring billed?
Two meters. Database hosts are counted every hour and billed on the high-water mark of the lower 99 per cent of usage, so a sustained increase in replicas sets the bill and a short one does not. Normalised queries tracked at any one time are counted on a second meter, above an allotment that comes with each database host. The separate host that runs the Agent for managed databases is billed as an infrastructure host. We do not publish prices. The host count, the replica schedule and the query limits are what we tune.
Which databases does Datadog Database Monitoring support?
Postgres, MySQL, MariaDB, SQL Server, Oracle, MongoDB, Amazon DocumentDB and ClickHouse, self-hosted or managed. Managed services such as RDS, Aurora, Cloud SQL, Azure Database and MongoDB Atlas are covered, with the Agent running on a separate host that connects over the network. Datadog's documentation carries the current list and the supported versions for each engine.
Does Datadog Database Monitoring need an Agent on the database server?
For self-hosted databases, yes, the Agent runs on the host and collects locally. For managed databases there is no server to install on, so one Agent on a separate host inside the same network polls each instance over a database connection. That host needs to be sized for the number of instances it polls, and it is the first place we look when samples go missing.
Can Datadog Database Monitoring show which service ran a slow query?
Yes, when APM is in place and database propagation is switched on in the tracer. The query then appears with the calling service and the trace it ran inside, and the trace shows the query. Without the propagation setting the two products run side by side and never meet, which is the most common reason the link is missing.
What does Critical Cloud do with Database Monitoring every month?
Managed Observability reviews the host count and tracked-query count against the meters, brings new instances and replicas under the same user and Agent host, lists the queries whose plans regressed with a recommended fix, and reports what changed. Under Critical Support our engineers take the blocking-query or replication incident, 24x7.
- Datadog products: every product, and how we implement and run each one
- Datadog APM: the trace that points at the database in the first place
- Datadog HealthScan: read-only, fully automated, and the first thing we run
- Datadog setup and configuration guides: Agent, integrations and tagging, step by step
The slow query, named. The service that sent it, known.
Tell us which databases the product cannot live without. We will show you what Database Monitoring sees today, what it is missing, and what it takes to close the gap.