Know which stage
ate the afternoon.
Datadog CI Visibility traces every pipeline run the way APM traces a request: each stage and job timed, each failure tied to a commit, every run kept for a year. We connect it to GitHub Actions, GitLab, Jenkins and the rest, scope it to the committer meter, and run it every month so the delivery dashboard still describes the pipeline that exists.
Quick facts
Related
- Test Optimization, the tests inside the pipeline, traced the same way
- DORA Metrics and AI Impact, what the pipeline data adds up to
- Software Catalog, the services the pipelines belong to
The pipeline as a trace, not a green tick
The build takes forty minutes and everyone agrees it used to be faster. Nobody can say when it changed, or which stage did it, because the CI provider shows one run at a time and the old runs have scrolled off. CI Visibility keeps every run, as a trace, so the question has an answer.
Every stage and job, timed. A pipeline run is a trace with a span per stage and per job, and on some providers per step. The flame graph shows where the forty minutes went, including the time spent waiting for a runner.
Failures with a cause and a commit. Each run carries the branch, commit and author. A failing stage opens the change that introduced it, and log analysis proposes a reason from the job output before anyone has read it.
Trends over a year, not a week. Duration per stage, failure rate per job, queue time per runner pool, across every pipeline. The stage that got slower in March is a line on a chart that bends in March.
The pipeline beside production. The deploy job and the incident that followed it are on the same timeline, in the same tool, tagged with the same service. The retrospective starts with the facts already in order.
A committer meter counting the bot, a pipeline nobody linked to its repository, and a dashboard built on stages that were renamed
How it bills. CI Visibility is billed per committer per month. A committer is an active git author, identified by email, who makes at least three commits in the month. A pipeline that is not linked to a repository, or that arrives without git metadata, bills the user who triggered it instead. Each committer brings 400,000 pipeline spans a month, pooled across all CI Visibility committers, where a span is a pipeline, a stage or a job, and spans beyond the pool are sold separately. Pipeline spans are kept for 15 months, and that retention cannot be changed. Storing job logs is billed separately from CI Visibility, and is switched on per organisation or per repository. Datadog publishes the rates. We publish nothing here, because the price is not the problem. Who counts as a committer is.
The configuration traps. The integration is switched on for every repository in the organisation, including the ones where a release script or a shared service account commits a lock file update every hour. Verified bots are excluded. A service account is a committer. A pipeline triggered by a scheduler and never linked to a repository bills the service account that triggered it, and the service account triggers everything. Custom tags added for one investigation get added to every run, and the dashboards are built on stage names that the platform team renames the following quarter. None of these is an error. They show up as a committer count nobody recognises and a delivery dashboard with a gap where the renamed stage used to be.
Field noteThe most common gap we see is the committer count including people who do not exist. Release automation running under a service account and a shared deploy user each commit often enough to be counted, and neither is a verified bot. The invoice says the team is larger than the org chart, and the first month is spent explaining why.
Field noteThe pipeline is connected but the repository is not. Runs arrive without a commit, an author or a branch, so the trace shows a slow stage and nothing about what changed. The question the tool was bought to answer is the one it cannot.
Field noteThe pattern we keep finding is a self-hosted runner pool with no Agent on it. The job trace says the build took twenty minutes. Nothing says the runner was swapping the whole time, because the host that ran it is not in the picture.
Field noteThe delivery dashboard was built in the accelerator on the stage names of the day. The platform team restructures the workflow into reusable jobs with new names. The dashboard goes flat, nobody opens it, and the next time anyone asks about build times the answer is a feeling again.
Repositories first. Committers counted before the first invoice. Dashboards on tags, not names.
If CI Visibility is not on yet, the Software Delivery Accelerator connects the pipelines, builds the delivery dashboards and reports the slowest and flakiest stages in four weeks, fixed scope. If it is already on, a Datadog HealthScan reports the committer count, the span pool and which pipelines are linked before we change anything. Either way, the sequence is the same.
- Inventory the pipelines and the providers. Which pipelines ship production, which build libraries, which run on a schedule and which are abandoned. Which provider each runs on and which features that provider supports.
- Decide who is a committer. Bots and service accounts are listed and excluded with Datadog's committer exclusion filters, or scoped out by repository. The expected committer count is written down before the integration is enabled.
- Connect the providers and link the repositories. The GitHub App, the Jenkins plugin or the webhook, per provider, and every pipeline linked to its repository so runs carry commit and author.
- Tag the runs. Service, team and environment on every pipeline, through datadog-ci or provider configuration. The dashboards then group by what the pipeline is for, not what it is called.
- Put an Agent on the runners and build the monitors. Self-hosted runner pools get the Agent so job time and host time line up. Monitors on pipeline failure, stage duration regression and queue time, routed to the team that owns the pipeline.
- Hand over, in writing. Which pipelines are connected and how, the committer exclusions, the tagging convention, and what each dashboard answers. Plus the committer and span counts on the day we finished.
Access is admin on the Datadog organisation plus admin on the CI provider for the integration, and a pull request into the pipeline definitions for tags. We do not need production shell access to implement CI Visibility, and we do not ask for it.
Pipelines are refactored more often than the applications they build
A new repository appears with a new bot. A workflow is split into reusable jobs. A runner pool is replaced. Managed Observability is the monthly service that keeps CI Visibility describing the pipelines that exist and the committers who are people.
Every month, we review the meters. The committer count against the org chart and the exclusion list, the span pool against the pipelines connected, and the logs storage meter if it is on. Anything that moved gets a reason or a fix.
Coverage follows the repositories. New pipelines are linked to their repositories and tagged in the month they appear. Dead pipelines are disconnected so they stop contributing spans and noise.
The dashboards are kept honest. When a stage is renamed or a workflow restructured, the dashboards are rebuilt the same month, on tags where possible so the next rename does not matter.
The slow stage gets a recommendation, then an owner. Each month we report the stages and jobs costing the most delivery time. Fixing them is engineering work on your pipeline, scoped through Catalyst when it is more than an afternoon.
We operate the stack. You own the product.
An implementation partner connects the pipelines and hands them over. A managed partner is still there when the workflow is restructured in March and the dashboards go flat. We are the second kind. The integrations, the tagging convention, the dashboards and the decisions stay in your Datadog organisation and your pipeline definitions, documented, and you can take them in-house whenever you choose.
Datadog is our proof, not our headline. Datadog certified one managed provider globally. We are it. The Powered by Datadog accreditation means Datadog's own engineers reviewed how we instrument, alert, respond and report. We are also a Datadog Advanced Partner, ISO 27001 certified and Cyber Essentials Plus certified, with offices in Cardiff, London and Dublin.
Datadog CI Visibility, answered plainly
How is Datadog CI Visibility billed?
CI Visibility is billed per committer per month. A committer is an active git author, identified by email address, who makes at least three commits in the month. If a pipeline is not linked to a git repository, or git metadata is missing, the user who triggered it is counted instead. Each committer brings 400,000 pipeline spans a month, pooled across all CI Visibility committers, where a span is a pipeline, a stage or a job. Spans beyond the pool are sold separately. Pipeline spans are kept for 15 months. Job log storage is billed separately, and is switched on per organisation or per repository. We do not publish prices. Which pipelines are connected, and whether they are linked to a repository, is what we tune.
Which CI providers does Datadog CI Visibility support?
GitHub Actions, GitLab, Jenkins, Buildkite, CircleCI, Azure Pipelines, AWS CodePipeline, Codefresh and TeamCity, plus a custom route for anything else through the datadog-ci command line tool. The mechanism differs by provider. GitHub Actions uses a GitHub App with read access to Actions. Jenkins uses a plugin. Others use webhooks or native integrations. Feature support also differs: step-level spans, running pipelines and automatic retry detection are available on some providers and not others, and we check the table for yours before promising anything.
Does this replace the CI provider's own dashboard?
It replaces the part of it that answers questions across pipelines and over time. The provider shows you one run. Datadog shows you every run of every pipeline for a year, which stage got slower in March, which job fails most on Mondays, and how long jobs sit queued before they start. It also puts the pipeline next to the production it deploys to, so a failed deploy and the incident that followed are on the same timeline.
Can we see which commit broke the pipeline?
Yes, when the pipeline is linked to the repository and the git metadata is on the run. Each pipeline execution carries the commit, the author and the branch, so a failing stage can be traced to the change that introduced it. Log analysis, which is on by default, reads the job logs and proposes a root cause for the failure. Storing the logs themselves for correlation is a separate opt-in.
What does Critical Cloud do with CI Visibility every month?
Managed Observability runs it every month. We review the committer count and the span pool against the pipelines connected. We link any pipeline that has drifted from its repository, add new pipelines and retire dead ones. We keep the delivery dashboards matching the stages that exist, and report the slowest and flakiest stages with a recommendation. The fix to the pipeline itself is engineering work, scoped through Catalyst when it is more than an afternoon.
- Datadog products: every product, and how we implement and run each one
- Test Optimization: the test suite inside the pipeline, on its own committer meter
- Continuous Testing: Synthetic tests as a gate in the same pipeline
- Software Delivery Accelerator: pipelines connected, dashboards built, hotspots found in four weeks
Build times as a number. Committers as people.
Tell us which pipelines ship production. We will tell you what CI Visibility would count, what it would show, and what it costs to get there.