Every cloud cost disaster has a detection problem at its root. The spend started increasing; the team did not know until it showed up in a bill. Sometimes the root cause is a misconfiguration. Sometimes it is a runaway process, a traffic spike that triggered auto-scaling, or an unintended data transfer. In almost every case, the underlying cause was visible in the infrastructure telemetry before it became visible in the billing data. The gap is that cost and infrastructure observability are treated as separate concerns.

Understand the most common causes of cost spikes

Before instrumenting for detection, it helps to know what you are looking for. The cost spikes we see most frequently in environments we manage fall into a handful of patterns:

  • Auto-scaling without a ceiling. A traffic event triggers scale-out, the ceiling is not configured correctly or is too high, and the environment runs at 10x its normal instance count for longer than necessary. The spike ends when someone notices, which may be hours or days later.
  • Runaway processes or jobs. A batch job, a data processing pipeline, or an analytical query that runs longer than expected or enters an infinite retry loop. Often involves managed services with per-unit billing (Lambda invocations, API calls, database query costs).
  • Unintended data transfer. A configuration change moves data flows between regions or out of a provider entirely, triggering egress charges that were not budgeted. Often invisible until the billing data arrives.
  • Forgotten resources. Development instances, load testing environments, or temporary infrastructure left running. Individually small; collectively significant over a month.
  • LLM token consumption. For organisations using AI APIs, a change to prompt design or an unexpected traffic volume can significantly increase token consumption and the associated cost.

Use native anomaly detection as a first layer

Both AWS Cost Anomaly Detection and Azure Cost Anomaly Alerts provide automatic detection of unusual spend patterns and can be configured to send alerts when spend deviates beyond a threshold. These are free to enable and require no engineering work beyond setup. Switch them on for every account and subscription, route alerts to a channel that gets reviewed daily, and set the sensitivity high enough that you are alerted at the beginning of a spike rather than after it has accumulated.

The limitation of native anomaly detection is that it lags: billing data is typically 24-48 hours behind actual resource usage. You will find out about a spike faster than at month end, but not in real time.

Instrument cost as a real-time observability signal

The better approach is to treat cost as a metric alongside your infrastructure and application telemetry, so that an engineer monitoring a production incident can see cost implications in the same dashboard as latency and error rates. With Datadog Cloud Cost Management, cost data is brought into Datadog as a time-series metric with the same tagging structure as the rest of your telemetry. This means you can alert on cost rate changes in near real time, correlate cost spikes against deployment events, and investigate the resource-level breakdown without leaving your observability platform.

The practical difference: with native anomaly detection you find out about a spike after it has accumulated. With real-time cost instrumentation you find out while it is happening and can act immediately. For the kinds of spikes described above, the difference in detection time directly translates to the difference in the size of the bill.

Set resource-level alerts, not just account-level budgets

Account-level budgets are useful but too coarse to be actionable in isolation. A budget alert that triggers when total monthly spend reaches 80% of budget tells you that something is wrong but not where to look. Resource-level alerts - alerting when a specific service's hourly spend rate exceeds a threshold - tell you exactly what to investigate.

Configure separate budget and rate alerts for: compute (by service and by tag), data transfer, managed service costs for the highest-volume services in your environment, and any AI API costs where token consumption is unbounded. Review these thresholds quarterly and adjust them as your environment changes.

Review instance and resource state on a scheduled basis

Forgotten resources do not show up in anomaly detection because their cost is not anomalous: it is the same every day, because they are always on. Catching these requires a regular inventory review: a weekly or monthly scan of all running instances, databases, and storage volumes that have not had recent activity. AWS Config and Azure Resource Graph can generate these inventories. Route the output to your engineering team for review and action.

Tag lifecycle on resources helps enormously here. Resources tagged as "temporary" with a target end date can be automatically flagged when they are still running past that date. This is one of the higher-value applications of a tagging standard in practice.

Correlate cost with performance and reliability events

Some cost spikes are legitimate: a traffic event that triggered scale-out and cost more money also delivered more revenue or handled a genuine demand increase. Others are waste. The key to distinguishing them is correlating cost changes against the performance and reliability signals that should explain them. A cost spike during a traffic event is expected. The same cost spike on a Sunday night with no corresponding traffic change is not.

This correlation is only possible when cost data and infrastructure data live in the same platform. Combining observability with cost management is one of the primary reasons we standardise on Datadog across our managed environments: it makes cost investigation a natural part of operational investigation rather than a separate billing exercise.