The Production AI Checklist: 6 Gates Before You Go Live

The demo answered the friendly questions, drafted a convincing email, and made the meeting feel ten minutes shorter. Everyone leaned forward. Someone said, “Can we launch this next week?”

Maybe. But production-ready AI needs more than a good performance in a controlled room. It needs a defined job, task-specific evaluation evidence, data boundaries, useful human control, a visible failure path, and monitoring with a real owner.

That is the production AI checklist in one sentence. If any part still depends on “the model will probably figure it out,” you have a promising prototype—not an application ready to meet customers, employees, deadlines, and Monday-morning weirdness.

Production begins where the happy-path demo ends

A demo proves that an idea can work. Production asks whether people can depend on the complete system when the input is vague, the source is stale, a vendor is slow, a permission is wrong, or the answer lands in a situation with consequences.

That distinction matters because an AI application is not just a model response. It is the model plus prompts, business data, retrieval, integrations, permissions, interfaces, review steps, logs, people, and the decisions they make with the output.

NIST’s AI Risk Management Framework Core treats AI risk work as a continuous cycle of governing, mapping, measuring, and managing. It calls for a defined business context, documented human oversight, testing before deployment, monitoring in production, privacy review, and recovery planning. You do not need to turn a small-business launch into a federal binder parade. You do need evidence proportionate to the job and the cost of being wrong.

The six-gate Ready for Monday checklist turns that idea into a practical go-live conversation. Each gate needs an artifact the team can show—not a reassuring adjective.

Gate 1: Name the job and draw its fence

Start with one sentence:

This application helps this person complete this job using these approved sources or actions, and it must stop or escalate when this boundary is crossed.

“Help the service coordinator draft a reply using the approved knowledge base” is a job. “Put AI in customer service” is a weather forecast.

The fence matters as much as the job. Write down what the application is not allowed to do. It may draft but not send. It may summarize an account but not change it. It may suggest an appointment window but not book one without confirmation. It may answer from approved service documentation but refuse questions that require account-specific or regulated information.

Then identify the people affected by a mistake. An internal brainstorming assistant and a customer-facing tool that can change an order should not face the same launch bar. Consequence should drive the depth of testing, approval, logging, and recovery—not how charming the demo felt.

Gate evidence: a one-page job statement with users, sources, permitted actions, forbidden actions, and a clear escalation boundary.

Gate 2: Replace “looks good” with evaluation evidence

An evaluation set is a collection of representative cases with a defined way to judge the result. It should include the ordinary work, the uncomfortable edge cases, and the failures you especially do not want to discover through a customer.

OpenAI’s current evaluation guidance recommends task-specific tests, early and repeated evaluation, production-shaped datasets, continuous evaluation, and human feedback to calibrate automated scoring. The sharpest warning is also the simplest: “seems to work” is not an evaluation strategy.

Build cases from the actual job:

• common requests that represent normal use;
• messy phrasing, missing details, and conflicting instructions;
• stale, incomplete, or unavailable source material;
• attempts to reach data or actions outside the user’s permission;
• inputs that should produce a refusal or a handoff; and
• previously observed failures that must not return.

Define what passes before looking at the new output. Depending on the job, that might include factual accuracy against an approved source, required fields, correct tool choice, correct arguments, tone boundaries, citations, refusal behavior, latency, or human acceptance.

Do not hide every concern inside one average score. A system that performs beautifully on harmless requests and fails the permission test is not “mostly ready.” Some requirements are gates. One serious failure should stop the release until it is understood.

Gate evidence: a versioned evaluation set, pass criteria, current results, known failures, and a release decision tied to those results.

Gate 3: Trace the data before the data traces you

Draw the path of information through the application. Include user input, retrieved documents, model providers, tool calls, logs, feedback, analytics, and any human-review queue.

For each stop, answer:

• What data arrives here?
• Why is it needed?
• Who or what can access it?
• Where is it stored, and for how long?
• Can a user with one role reach information belonging to another?
• What is sent to an outside provider?
• What gets redacted or excluded before logging?

This is not just a privacy exercise. It is how you uncover a support transcript entering an analytics tool, a shared service account bypassing user permissions, or a debug log quietly retaining content nobody intended to keep.

If an outside model or platform will handle business information, run the AI vendor privacy checklist before launch. Provider settings and contract language are part of the architecture. A checkbox in the admin panel is not a data-flow diagram.

Gate evidence: an approved data-flow map, access rules, retention decisions, vendor boundaries, and test results showing those boundaries hold.

Gate 4: Give human review an actual job

“Human in the loop” sounds responsible and can mean almost nothing.

A useful review point tells the person what decision they are making, shows the evidence needed to make it, and gives them enough authority to correct or stop the action. A useless review point asks someone to click Approve on an opaque result while the queue grows faster than anyone can inspect it.

Match review to consequence:

AI behavior	Practical human control
Suggests low-consequence internal wording	Edit or accept in the normal workflow
Drafts an external customer message	Review the draft, source context, recipient, and attachments before sending
Proposes a record change	Show the current value, proposed value, reason, and affected record before approval
Cannot meet a required quality or source rule	Block the action and route the case to a person

The reviewer also needs a realistic workload. If the design sends every ordinary case to a person, the “automation” has built a new inbox. If it sends only rare but indecipherable cases, reviewers will lose context and click through. Test the review experience with the people who will use it, including how they disagree, correct the output, and report a missing rule.

Gate evidence: a review map showing which actions require approval, what the reviewer sees, how they change or reject the result, and who handles escalations.

Gate 5: Make failure boring, visible, and recoverable

Every dependency eventually has a bad day. The model times out. Retrieval finds nothing. An API changes. A tool completes half the work. A user retries and creates a duplicate. The question is whether the application fails like a professional or improvises like a raccoon in an unlocked pantry.

Write down the expected response for each critical failure:

• stop and show a clear message;
• fall back to a simpler non-AI path;
• save a draft without performing the action;
• route the work to a person with the relevant context;
• retry only when the operation is known to be safe and idempotent; or
• temporarily disable the capability while preserving the rest of the product.

Google Cloud’s guidance for deploying and operating generative AI applications emphasizes versioning prompts and connected components, end-to-end evaluation, production-like delivery tests, and lineage across the complete application. Those details are what make rollback and diagnosis possible. “The AI was weird” is not an incident report.

Test the ugly paths on purpose. Disconnect a dependency. Return malformed data. Revoke a permission. Force a timeout. Submit the same request twice. Confirm that the person sees the truth, the system does not take an unintended action, and the team can trace what happened.

Gate evidence: a failure-mode table, bounded retry rules, a tested fallback or shutdown path, rollback instructions, and an incident owner.

Gate 6: Decide what Tuesday will tell you

Passing the launch test does not freeze the application in amber. Real people will bring different language, different cases, and different habits. Providers will update models. Your documents, prompts, tools, and business rules will change.

The UK National Cyber Security Centre’s secure-operation guidance recommends monitoring system behavior and inputs, logging in line with privacy requirements, and treating changes to data, models, or prompts as changes that require testing. Google Cloud likewise recommends end-to-end logging, alerts for performance decay, and continuous evaluation of production output.

Choose signals that reveal whether the business job is still working:

• task completion and abandonment;
• pass rates on a sampled set of production cases;
• human corrections, rejections, and escalations;
• refusals and out-of-scope requests;
• tool errors, timeouts, duplicate prevention, and fallback use;
• latency and cost by workflow; and
• newly discovered cases added to the evaluation set.

Then name the owner. Someone needs authority to review the signals, pause a feature, approve a model or prompt change, update the evaluation set, and decide whether a failure is isolated or systemic. “Engineering monitors it” is not ownership unless a person, cadence, threshold, and response are attached.

Gate evidence: a production dashboard or review report, alert thresholds, a sampling plan, a change-and-retest process, and a named operating owner.

Use red, yellow, and green—then respect the red

Score each gate with a color:

• Green: the evidence exists, has been tested, and has an owner.
• Yellow: the evidence is partial, but the remaining gap has a narrow limit and a dated plan.
• Red: the team cannot show the evidence or the failure could cross an unacceptable boundary.

A low-consequence internal release may move forward with a controlled yellow. A red on access control, irreversible action, sensitive data, or a required safety behavior should stop the launch. The purpose of the checklist is not to produce six green icons for a slide. It is to make a better decision while changes are still cheaper than an incident.

This also keeps the article separate from broader planning. If the question is whether your people, workflow, information, ownership, and guardrails are ready for a project at all, start with the AI readiness checklist. If the prototype already exists but the surrounding operational bridge is missing, use the guide to why AI pilots fail between the demo and daily operations.

The Ready for Monday checklist asks the final, narrower question: can this application do its defined job under real conditions without hiding uncertainty, authority, or failure from the people responsible for it?

Build the evidence into the product

The best production controls do not live in a launch spreadsheet forever. They become part of the application: permission checks, evaluation runs, approval screens, source visibility, bounded retries, logs, alerts, feedback capture, version history, and recovery tools.

That is where custom AI work earns its keep. The model matters, but the durable value comes from designing the complete system around a real job and making it supportable after the applause from the demo has faded.

If you have an AI prototype that needs to become dependable software, explore custom AI software development. We can turn the six gates into product requirements, test evidence, operating controls, and a go-live decision your team can explain without crossing its fingers.