Project Moonshot: control an AI release

Project Moonshot can support a repeatable release control for large language model applications. IMDA describes the toolkit as combining benchmarking and adversarial testing, with integration into continuous integration and deployment pipelines. The control value comes from defined thresholds, reproducible test inputs and an authorised decision, not from running the tool once.

Define the release unit and risk boundary

Record the model, version, system prompt, retrieval source, tools, filters, user group and intended decision. Identify harms such as hallucination, undesirable content, data disclosure, adversarial prompts and unsafe actions. State which changes require a new full test rather than a limited regression run.

The IMDA artificial intelligence page explains that Moonshot can test LLM applications and integrate with delivery pipelines. The Project Moonshot factsheet places the toolkit within Singapore’s practical, risk-based AI governance approach.

Create reproducible tests and thresholds

Version the benchmark set, adversarial prompts, expected behaviour, scoring method and test environment. Include ordinary, edge and hostile cases that reflect the actual use. Set pass, conditional-pass and stop thresholds before the result is known, then assign an owner for every exception.

Protect confidential prompts and personal data. Use synthetic or properly authorised test material where possible. Record tool configuration and software versions so another reviewer can reproduce a material failure rather than relying on a dashboard screenshot.

Connect results to deployment and monitoring

Block production when a mandatory threshold fails. Require documented approval for a temporary exception, with compensating controls, expiry date and retest. Preserve the results, issue remediation and final release identifier in the deployment record.

After release, monitor user feedback, incidents and data or model drift. Feed representative failures into the versioned regression set. A production incident should update the next release check and may require rollback, user notice or suspension under the company’s response plan.

Test the control itself

Run a rehearsal with a deliberately failing release. Confirm that the pipeline blocks deployment, the alert reaches the correct owner and an unauthorised user cannot override the result. Then test the documented exception route and rollback. Record timing and gaps so the release control is proven before it is relied on for a material application.

Turn the decision into a controlled file

Use a dated decision record that identifies the applicant, business need, current official source, chosen route, responsible owner, costs, milestones and evidence. Separate facts already verified from forecasts and assumptions. A catalogue listing, supplier statement, consultant recommendation or portal acknowledgement does not establish approval. The company remains responsible for its declarations and commercial decision.

Give each quotation, contract, invoice, payment and deliverable a stable reference. Preserve earlier versions and explain material changes instead of silently replacing them. Where the authority, tax adviser, auditor or another professional must decide a point, record the open question and obtain the required view. Do not convert uncertainty into a confident eligibility statement.

Review before commitment and again before closeout

Run four reviews: suitability, authority, implementation and closeout. At suitability, test the entity, activity and business outcome. Before commitment, freeze the proposed scope, vendor and financial assumptions. During implementation, compare actual work with approved terms. At closeout, reconcile deliverables, accepted results, invoices, full payment, operating evidence and obligations that continue.

Name the person who may approve each stage and the records that person must see. Procurement should not treat a draft application as permission to contract. A project team should not treat supplier completion as company acceptance. Finance should not treat a submitted claim as cash received. Clear status words prevent accidental commitments and weak reporting.

Handle exceptions without rewriting history

Maintain an exception log for changed facts, delayed milestones, supplier variations, cost movements, missing records and failed tests. State the effect on scope, eligibility, timing and expected value. If an authority decision or formal variation is required, use the published route before relying on the change. Silence and helpdesk receipt are not approval.

If a material defect appears, pause the affected acceptance, filing or claim item. Keep original evidence, assign corrective action and set a review date. Do not backdate records, move costs between categories or adjust a baseline after seeing the result. A candid exception with an owned resolution is more credible than a clean-looking file that cannot explain what happened.

Plan handovers before the original project team disperses. The handover should identify the current version, source locations, system access owner, next review date, open exceptions and any condition that continues after payment or filing. Restrict personal, financial and commercially sensitive records to authorised users, while giving each responsible owner enough information to perform the assigned control. Record the external event that should trigger an earlier review, such as a changed programme page, revised authority letter, supplier failure or material business change.

Verify the final outcome independently

Ask someone who did not prepare the final pack to compare the decision, current official source, approved terms, deliverables, payments and operating evidence. Recalculate totals and sample source records. The reviewer should be able to reproduce the conclusion without relying on the preparer’s memory. Record unresolved points with named owners and dates.

Use the AI Verify and Moonshot test-plan guide for the wider decision and keep this article as the narrower operating guide. After submission, preserve the receipt and reconcile questions, adjustments and the final registered or paid outcome. Continue any reporting, retention, asset-use or other condition that survives closeout.

Practical review checklist

• Release unit recorded
• Risk cases selected
• Thresholds fixed
• Pipeline block tested
• Post-release feedback connected

The control is operational when a named release cannot enter production without reproducible safety tests, predetermined thresholds, owned exceptions and a retained approval record.