Know the checkout is broken
before a customer does.
Datadog Synthetic Monitoring runs API and browser tests against your flows, on a schedule, from the places your users are. We implement it on the flows that matter, set the frequency and locations so the run meters hold, and run it every month so the tests still pass for the right reasons.
Quick facts
Related
- Real User Monitoring, what real users saw on the same pages
- Continuous Testing, the same tests run in the pipeline before deploy
- Service Level Objectives, the uptime the tests are evidence for
A customer who tries the checkout every five minutes and tells you when it fails
At 3am nobody is buying anything, so nobody notices the payment provider's certificate expired. At 7am everybody notices. Synthetic Monitoring is the customer who was there at 3am, and the alert is what they would have said.
The API checked, not just pinged. HTTP, SSL, DNS, TCP, WebSocket and gRPC tests with assertions on the response, and multistep tests that sign in, fetch a token and use it. A 200 from a login page that returns an error in the body is still a failure.
The flow, clicked through. Browser tests record the steps a user takes and replay them on a schedule, with screenshots at each one. When the test fails, the screenshot shows the page the user would have been stuck on.
From where the users are. Managed locations around the world, and private locations you run inside your own network for the admin tool, the staging site and the internal API.
The failure joined to the trace. A failing API test opens the APM trace of the request it made. The alert says the checkout failed, and the trace says it was the inventory service, before anyone has opened a laptop.
Every location, every minute, on every page, and a browser test that broke when the button moved
How it bills. Synthetics is billed per test run, on separate meters by type. API tests are one meter, and each step of a multistep API test counts as a run. Browser tests are another. A browser run covers up to 25 recorded steps, so a longer test counts as more than one run. Mobile app tests on real devices are a third. Every run counts: each location, each interval and each retry. A private location costs nothing to set up, and tests run from it are billed the same as tests from a managed location. Continuous Testing runs in the CI pipeline draw on the same meters, with parallel slots as a separate add-on. Datadog publishes the rates. We publish nothing here, because the price is not the problem. Frequency times locations is.
The configuration traps. A test is created with the shortest interval and every managed location ticked, because the defaults were there and the page was important that week. It stays that way. Browser tests are recorded against the markup of the day, and a redesign that moves a button turns a monitoring tool into an alert generator. Retries are set to zero, so a single slow response from one location pages someone. Or retries are set high enough that a real outage takes ten minutes to page anyone. None of these is an error. All of them show up either on the invoice or in the on-call rota's opinion of the tool.
Field noteThe most common gap we see is tests on pages and no tests on flows. The home page is checked from eight locations every minute. The checkout, which is where the money is, has no test at all because it needed a test account and nobody had one to hand.
Field noteBrowser tests are recorded once and never re-recorded. The frontend ships weekly. Self-healing locators absorb the small changes but not the redesigns, so within a quarter tests start failing on elements that no longer exist, the failures are muted, and the muted tests include the one that would have caught the outage.
Field noteThe pattern we keep finding is a private location deployed for one project and never upgraded. The worker image is two years old, and so are the browsers that shipped in it. A test that passes from a managed location fails from the private one, for reasons that have nothing to do with the application.
Field noteThe synthetic traffic is never filtered out. Datadog marks it with its own user agent and headers, but nobody told the web analytics or the fraud rules to look for them, so it is in the conversion figures and in the fraud model. A test account gets locked for suspicious activity, the test fails, and the alert is about the lock rather than the product.
Flows first. Frequency and locations from the promise, not the default.
If Synthetics is not on yet, LaunchPad runs the rollout as a managed project, and the Digital Experience Accelerator delivers the critical flow monitor pack with RUM and the dashboards in four weeks. If it is already on, a Datadog HealthScan reports run volumes, failing tests and location spread before we change anything. Either way, the sequence is the same.
- List the promises. Which flows earn money, which are in a contract, and which an executive would hear about first. Each one gets a test. Pages that are none of these do not.
- Set up the test accounts and the test mode. A dedicated account per flow, a header that marks synthetic traffic, and an agreement with the application owner about what a test may do. A test that places real orders is not a test.
- Build API tests before browser tests. API tests are cheaper to run and cheaper to maintain. A browser test is added where the failure can only be seen in a browser.
- Choose frequency and locations from the promise. A flow with a contractual uptime target runs often from the regions in the contract. An internal tool runs less often from one private location. Retries are set so one slow response does not page and a real outage does.
- Deploy and version the private locations. A worker per network segment, in the orchestrator the estate already uses, with a health check, a monitor and an upgrade schedule.
- Hand over, in writing. Which flows are tested and why, which accounts the tests use, and the frequency and location of each. Who gets paged, and what the run counts were on the day we finished.
Access is admin on the Datadog organisation, test accounts on the applications, and a place to run the private location workers. We do not need production shell access to implement Synthetics, and we do not ask for it.
A test is a promise about the product. The product changes every week.
A button moves. A flow is retired. A new market opens and nobody added the location. A worker falls three versions behind. Managed Observability is the monthly service that keeps the tests matching the product they test.
Every month, we review the meters. Runs per test type against frequency, locations and retries. A test that runs every minute from eight locations gets a reason or a smaller footprint.
Coverage follows the product. New flows get tests before launch. Retired flows lose theirs. A new region gets a location when the first customer there signs, not after the first complaint.
Broken tests are fixed, not muted. When a browser test fails because the page changed, we re-record it the same week. A muted test is a test that is not there, and we treat it that way.
The incident is ours to take. Under Critical Support our engineers answer the alert the failing test raised, 24x7, and fix the cause rather than report it. Datadog has published that our work reduces mean time to resolve incidents by 40 to 60 per cent.
We operate the stack. You own the product.
An implementation partner records the tests and hands them over. A managed partner is still there when the checkout button moves in March and the test breaks with it. We are the second kind. The tests, the test accounts, the private locations and the decisions stay in your Datadog organisation, documented, and you can take them in-house whenever you choose.
Datadog is our proof, not our headline. Datadog certified one managed provider globally. We are it. The Powered by Datadog accreditation means Datadog's own engineers reviewed how we instrument, alert, respond and report. We are also a Datadog Advanced Partner, ISO 27001 certified and Cyber Essentials Plus certified, with offices in Cardiff, London and Dublin.
Datadog Synthetic Monitoring, answered plainly
How is Datadog Synthetic Monitoring billed?
Synthetic Monitoring is billed per test run, on separate meters by test type. API tests are counted on one meter, and each step of a multistep API test counts as a run. Browser tests are counted on another. A browser run covers up to 25 recorded steps, so a longer test counts as more than one run. Mobile app tests have a third meter, counted per run on real devices. Every run counts: each location, each interval and each retry is a run. Private locations are workers you host, at no extra charge, and tests run from them are billed the same as tests from managed locations. Continuous Testing runs from the CI pipeline draw on the same run meters, with parallel slots as a separate add-on. We do not publish prices. Frequency, locations and retries are what we tune.
Which flows should we put a Synthetic test on?
The ones that cost money or trust when they break and that nobody would notice for an hour otherwise. Sign-in, search, add to basket, checkout, the API your biggest integration calls, and the login page of the admin tool the support team lives in. A test on every page is noise and a bill. A test on every promise you have made to a customer is monitoring.
Will Synthetic tests put load on production?
A browser test every few minutes from a handful of locations is a few extra users. The traps are elsewhere. A test that completes a real checkout creates a real order unless the flow has a test mode. A test that signs in with a shared account trips the rate limiter or the fraud rules. We use dedicated test accounts, flag the traffic with a header so it can be excluded from analytics, and agree with the application owner what a test is allowed to do.
Can Synthetics test services that are not on the public internet?
Yes, with a private location. It is a worker you run in your own network, as a container or a Windows service, that fetches test configuration from Datadog over outbound HTTPS and runs the tests from inside. Internal APIs, staging environments and admin tools behind a VPN can all be tested. The worker is yours to run, so it needs a version policy and a health check like any other service.
What does Critical Cloud do with Synthetic Monitoring every month?
Managed Observability runs it every month. We review run counts per test type against frequency and locations. We fix tests that broke because the page changed rather than because the product did. We add tests for new flows, retire tests for flows that no longer exist, check the private location workers are current, and report what changed. Under Critical Support our engineers also take the incident the failing test raised, 24x7.
- Datadog products: every product, and how we implement and run each one
- Mobile App Testing: the same idea on real phones
- Continuous Testing: the tests run in CI, blocking the deploy
- Digital Experience Accelerator: Synthetics, RUM and the dashboards in four weeks
Tests on the promises. A bill that holds.
Tell us which flows you would want to know about at 3am. We will tell you what Synthetics is testing today, what it should be testing, and what it costs to get there.