Print it, or paste it into a ticket. Anything unticked is not a failure. It is the backlog, in the order that matters.

1. You can see it

  • Every model call is traced end to end: the request that caused it, the prompt, the model and version, the tokens, the latency and the response.Evidence: a trace for any customer-facing request, retrievable by request ID, retained for as long as the decision could be questioned.
  • Every tool call an agent makes is in the same trace: which tool, with what arguments, what came back.Evidence: a trace showing a complete agent run, including the calls it decided not to make.
  • Cost is attributed per service, per feature and per customer, not per month.Evidence: a dashboard that answers "what did this feature cost yesterday" without a spreadsheet.
  • External dependencies are monitored as dependencies: the model provider, the vector store, the tool APIs.Evidence: an alert fired the last time the provider degraded, and you saw it before the customers did.
  • Prompts and model versions are versioned and the version is in the trace.Evidence: you can say which prompt produced a given answer on a given date.

2. Someone owns it at 3am

  • A named person is on call for the service, with a rota that is not "the developer".Evidence: the rota, and the last page they answered.
  • Every alert that can page has a runbook linked from the alert itself.Evidence: open the monitor; the runbook link is there and the runbook is current.
  • Alerts are routed to a person, not only a channel.Evidence: a detection that fired out of hours and the record of who acknowledged it.
  • The service has an owner tag and the tag is on every component, including the vector store and the tool endpoints.Evidence: a query by owner returns the whole service and nothing else.

3. It can be stopped, and the stop stops

  • There is a kill switch that a person on call can operate without a deploy.Evidence: the runbook step, and the name of who may pull it.
  • Pulling it halts in-flight requests and any delegated sub-agents, not only new requests.Evidence: you have pulled it in a test and watched the in-flight work stop.
  • There is a degraded mode: what the customer sees when the AI is off.Evidence: a screenshot of the degraded experience, approved by whoever owns the customer.
  • Rate and spend limits exist per service and are enforced by the platform, not the prompt.Evidence: the limit configuration, and the alert that fires at a set fraction of it.

4. Its reach is bounded

  • What the service can read is written down, and it is less than everything.Evidence: the identity it runs as, and the permissions attached to it, reviewed in the last quarter.
  • What it can write or change is written down, and each write path has a human approval or a reversible action.Evidence: the list, and the approval queue or the undo.
  • Which external systems it can call is an allow-list, including which MCP servers or tools, and the list is versioned.Evidence: the allow-list in source control with its last change.
  • Instructions are not controls. Anything the model must never do is enforced outside the prompt.Evidence: the enforcement point, and a test that shows the prompt cannot override it.
  • Customer data is tenant-isolated at retrieval, so one customer's context can never be another's.Evidence: a test that attempts the cross-tenant read and fails.

5. Data is handled as data

  • Prompts, responses and logs are classified: what personal or sensitive data can appear in them and why.Evidence: the one-line classification per source.
  • Sensitive data is detected and redacted in logs and traces before indexing.Evidence: the scanner configuration and a sample of what it caught last week.
  • Retention is set per source, and prompts containing customer data are not kept longer than the purpose.Evidence: the retention settings, and the deletion that ran.
  • The model provider's data terms are read and recorded: what they retain, for how long, whether they train on it.Evidence: the terms, the date read, and who signed off.

6. There is evidence, written by something other than the agent

  • Decisions the service makes that affect a person or a record are logged with the inputs and the output, by the platform, not the model.Evidence: the decision log, queryable by customer and by date.
  • Human approvals are recorded with who, when and what they saw.Evidence: an approval entry from the last week.
  • Incidents involving the service have a post-incident record written by a person.Evidence: the last one.

7. You can tell whether the output was any good

  • Output quality is measured on real traffic, continuously, not only on the test set before launch.Evidence: the quality metric on a dashboard, with its trend.
  • A quality regression alerts someone, like an error rate would.Evidence: the monitor and its threshold.
  • There is a feedback path from the people who use the output back to the people who run it.Evidence: the last three items that came through it and what changed.

8. The supply is known

  • Every model, library, tool server and dependency the service uses is listed, with a version and an owner.Evidence: the inventory, updated by the pipeline rather than by hand.
  • Dependency and code scanning runs in the pipeline and findings reach a named person.Evidence: the last critical finding and who closed it.

The 3am test. Pick the item on this list you are least sure about. Imagine the service misbehaved at 03:00 last night in exactly that way. Write down who was woken, what they could see, and what they were allowed to do. If any of the three is blank, that is the item to do first.

Where we fit

Critical Cloud runs this review against AI services as the start of AI Runtime Operations, on the stack you already use, and then puts the missing items in place: tracing of models, prompts, tools and cost on Datadog, detection routed to a named engineer, a stop that has been tested, and the decision record. The operating model is set out in AI operations governance, agents own the analysis and humans own the outcome, and the failure modes we watch for are in AI incident response. You keep the model and the product. We operate the stack and keep the evidence.

Frequently asked questions

What counts as an AI service being used in the business?

Any point where a person or a process depends on its output without checking it first. A chatbot a customer can reach. An agent that files tickets, sends emails or changes records. A model whose score decides what a human sees next. The trial ended the day someone stopped reading every answer.

Do we need all of this for an internal tool?

You need the sections on visibility, ownership, stopping and reach for anything that can write to a system or read customer data, internal or not. Half of the AI-built apps found leaking data this year were internal tools. The evidence and quality sections matter as soon as a decision made with the tool's help could be questioned later.

Can Critical Cloud run this checklist against our AI service?

Yes. AI Runtime Operations starts with exactly this review on the stack you run, then puts the missing items in place: tracing of models, prompts, tools and cost on Datadog, detection routed to a named engineer, a stop that is tested, and the decision record. You keep the model and the product. We operate the stack it runs on and keep the evidence.

Sources

  • The checklist expands the readiness points James Smith took to the Datadog Secure panel at Datadog Live London, 4 November 2026: the same fundamentals as any service, an owner, and observability before go-live. See Running AI in production: what I took to the Datadog Secure panel.
  • The "internal tools" point is from the RedAccess research of May 2026, as reported by Security Boulevard and verified by Axios: a large share of the exposed AI-built apps were internal dashboards and trackers.
  • The data-handling items follow Datadog's Sensitive Data Scanner scope for logs, APM and RUM.