Raffles Consulting Services
Use AI Verify and Project Moonshot to build a risk-led testing plan for traditional AI and generative AI applications.
AI Verify and Project Moonshot give Singapore organisations practical testing resources, but running a toolkit is not the same as proving an application is safe or compliant. A useful plan starts with the deployed use case, affected users and unacceptable outcomes. The team can then select tests, evaluators and human review that match the actual risk.
Classify the system and its decisions
Record whether the system predicts, ranks, recommends, generates content or takes action. Identify data sources, model providers, integrations, users and people affected. Mark where a human reviews output and where automation can change a customer, employee or operational outcome. This system map determines whether traditional model testing, generative-AI testing or both are relevant.
The IMDA artificial intelligence resource describes AI Verify for governance principles and technical tests, and Project Moonshot for large language model evaluation. Use their current documentation to match tools to the system. Do not cite a toolkit name as evidence when the selected tests do not address the application’s real failure modes.
Design tests around harms and operating conditions
List credible failures such as inaccurate answers, undesirable content, data disclosure, biased treatment, prompt attacks and unsafe automation. For each, define test data, evaluator, threshold, reviewer and response. Include Singapore-specific language, facts and operating context where they matter. Keep protected or customer data out of test environments unless governance and security controls permit its use.
Project Moonshot combines benchmarking, red teaming and baseline testing. Benchmark results support comparison, while adversarial testing explores failure paths. Neither should be reduced to one pass mark. Record tool versions, prompts, datasets, model settings and evaluator limitations so a result can be reproduced after the model or application changes.
Connect test results to release decisions
Define which failures block release, which require mitigation and which can be accepted by an authorised owner. Retest after changes and monitor production evidence for drift, misuse and emerging attacks. An application can pass a laboratory benchmark yet fail because retrieval data, user permissions or human handoffs differ in production.
Link the plan to the agentic AI use-case screen and the AI vendor data review. The business support cornerstone can help separate a testing need from a grant decision. Keep legal and sector-specific assessments separate from tool output.
Keep one controlled project record
Use a dated record that connects the business decision, approved scope, source evidence, responsible owner, milestones, costs and acceptance. Distinguish proposed, approved, completed, submitted and paid items. A portal acknowledgement, consultant report or vendor invoice is evidence of one event, not proof that the complete outcome has been achieved.
Preserve earlier versions and explain material changes. Where an authority, auditor or professional adviser must decide a point, record that dependency instead of replacing it with an internal assumption. Management remains responsible for the declarations, commercial choice and continuing obligations after any support or filing is approved.
Review the decision in stages
Use four reviews: fit, authority or application, implementation and closeout. At fit, confirm the entity, activity and real business need. Before commitment, freeze the proposed scope and evidence. During implementation, compare actual work with approved terms. At closeout, reconcile deliverables, cost, payment, operating results and any obligations that survive completion.
Name the person who may approve each review and the evidence that person must see. Procurement should not treat a draft application as authority to contract. A project team should not treat vendor completion as company acceptance. Finance should not treat claim submission as receipt. These distinctions protect operating quality and the integrity of every declaration.
Manage exceptions without rewriting history
Maintain an exception log for delayed milestones, changed suppliers, cost movements, technical limits, missing evidence and failed tests. State the effect on scope, eligibility, timing and business value. Where authority approval is needed, use the official route before relying on the change. Silence, a draft upload or a helpdesk acknowledgement is not approval.
If a material defect appears, pause the affected acceptance, filing or claim item. Preserve the original evidence and assign corrective action. Do not backdate documents, move costs between categories or alter a baseline after seeing the result. A transparent exception with an owned resolution is more credible than a clean file that cannot explain the actual project.
Finish with independent readback
Ask a person who did not prepare the final pack to compare the decision, current official source, approved terms, deliverables, invoices and operating evidence. Recalculate totals and test source records. Record unresolved items separately with owners and dates. The reviewer should be able to reproduce the conclusion without relying on the preparer’s memory.
After submission, preserve the receipt and reconcile questions, adjustments and final outcome to the project record. Continue record retention, reporting, asset-use or other conditions that survive closeout. Completion means management can explain what was decided, what was delivered, what evidence supports it and what still needs attention.
Schedule a later review at a date suited to the decision. Compare the expected outcome with actual operating, financial and compliance evidence. Record benefits that did not appear, new risks and actions still open. This follow-up keeps the project useful after the immediate application, filing or implementation work has ended.
Keep the current official page and the date it was checked beside the record. Programme terms, filing instructions and forms can change. Before a later commitment or submission, confirm the live rule again and update the analysis without deleting the evidence used for the earlier decision.
Practical review checklist
- System and users mapped
- Failure modes prioritised
- Tests and evaluators selected
- Release thresholds approved
- Production monitoring assigned
AI Verify and Project Moonshot are most useful when test evidence changes a release decision. Start from the system’s real harms, preserve reproducible results and keep an accountable person responsible for the final deployment choice.