Resaro and Partnership on AI push for practical AI quality assurance at ATxSummit
Resaro and Partnership on AI publish a report on operationalising AI assurance during a closed-door working session at ATxSummit.
Resaro and Partnership on AI convened a closed-door working session at ATxSingapore to examine how organisations can assess whether AI systems are ready for deployment, as organisations move from experimentation to operational use.
Table Of Content
The session, titled Practical Pathways to AI Assurance and Quality, brought together enterprise AI adopters, public sector leaders, and assurance practitioners to discuss how AI systems should be defined, measured, verified, and compared in real-world contexts. The event also marked the release of Pathways for Operationalising AI Assurance, the fourth report in Resaro and Partnership on AI’s series on strengthening the AI assurance ecosystem.
AI assurance moves towards operational testing
The roundtable focused on a practical gap in AI adoption. Organisations deploying AI often rely on vendor-supplied benchmark scores and product demonstrations when assessing solutions. According to the report, these signals do not reliably predict how systems will perform under real operating conditions.
Around 45 senior practitioners joined the working session, which was structured around unresolved industry challenges. Participants developed quality indicator sets for real use cases and engaged with an overview of end-to-end testing, evaluation, verification, and validation practices.
The session also included a fireside conversation with Minister of State for Digital Development and Information Jasmin Lau, Partnership on AI CEO Rebecca Finlay, and Resaro Co-CEO April Chin. Lau encouraged early AI adopters to share knowledge across sectors, with diverse perspectives helping organisations rethink familiar deployment problems.
The event was held on 19 May as part of ATxSummit and in support of Singapore’s Smart Nation initiative.
Report identifies gaps in AI quality assessment
Pathways for Operationalising AI Assurance report argues that the current AI market lacks an independent, standardised quality framework comparable to the metrics used in more mature sectors. The report cites examples such as crash-test ratings, engine power, fuel consumption, and boot space in the automotive sector as indicators that help buyers compare products beyond vendor claims.
The report identifies six reasons why this gap remains difficult to close. These include the lack of a shared quality language across technical, operational, and governance teams, the difficulty of translating abstract principles into measurable criteria, and the divergence between benchmark performance and deployment performance.
It also points to the cost and limited scalability of bespoke evaluation, the context-dependent nature of what counts as “good enough”, and the tendency to treat trustworthiness and performance as competing rather than complementary properties.
ASQI offers a use-case-based framework
The report also presents the AI Solutions Quality Index methodology as a practical approach to AI quality assessment. ASQI structures evaluation around orthogonal, use-case-specific quality indicators and uses a five-level scale instead of a binary pass-or-fail model.
The framework is designed to be automatable and repeatable, with results that can be understood by non-technical decision-makers. It is also intended to support ongoing assessment rather than one-time certification.
Case studies in the report cover deployments in public administration, public safety, and legal services, including work connected with Singapore’s IMDA Global AI Assurance Pilot. Across these cases, the framework produced structured, evidence-based assessments that vendor benchmarks could not provide, including clearer deployment decisions and specific areas requiring improvement before systems were ready for use.
The broader Strengthening the AI Assurance Ecosystem series examines the practical requirements for independent AI assurance. Its 2026 reports cover ecosystem architecture, standards for assurance providers, demand and incentives for independent assurance, and the operational challenge of defining and verifying AI quality. Further reports will address post-deployment accountability.
