Which three of the following are key elements of a general assurance engagement?

Download now

Jump to Page

Search inside document

You're Reading a Free Preview
Pages 6 to 12 are not shown in this preview.

Buy the Full Version

Reward Your Curiosity

Everything you want to read.

Anytime. Anywhere. Any device.

No Commitment. Cancel anytime.

Share this document

Share or Embed Document

Sharing Options

• Share on Facebook, opens a new window
• Share on Twitter, opens a new window
• Share on LinkedIn, opens a new window
• Share with Email, opens mail client
• Copy Link

Quick navigation

• Home

• Books

• Audiobooks

• Documents

  , active

This challenge of assessing the trustworthiness of systems, processes and organisations, to build justified trust is not unique to AI. Assurance as a service draws originally from the accounting profession, but has since been adapted to cover many areas such as cyber security, product safety, quality and risk management. In these areas, ecosystems of assurance products and services enable people to understand whether systems are trustworthy.

The requirements for a robust assurance process are most clearly laid out in the accounting profession, although we see very similar characteristics in a range of mature assurance ecosystems, from cybersecurity to product safety.

In the accounting profession the 5 elements of assurance are specified as: A 3-party relationship; agreed and appropriate subject matter; suitable criteria; sufficient and appropriate evidence; conclusion/opinion.

The five elements of assurance engagements provide conditions that ensure trustworthiness can be [a] adequately and consistently assessed and [b] communicated to relevant parties, to enable them to build justified trust in a defined subject matter.

While we have drawn on the formal definitions developed by the accounting profession in this roadmap. Across the range of assurance ecosystems, similar roles, responsibilities and institutions for standard setting, assessment and verification provide transferable assurance infrastructures.

Within these common elements, we have also seen variation in the use of different assurance models across mature ecosystems. Some rely on direct performance testing, while others rely on reviewing processes, or ensuring that accountable people have thought about the right issues at the right time. In each case, the need to assure different subject matters has led to variation in the development and use of specific assurance models, to achieve the same ends.

The five elements are summarised in the following. Below each element, we summarise the key questions that need to be addressed when applying these elements in context:

1. Three Parties: Identifying who needs to demonstrate trustworthiness, who needs to gain trust, and who is performing assurance activities

• The responsible party [first party] wants to demonstrate and communicate the trustworthiness of the subject matter. This requires the right motivations and incentives to provide true and faithful information about the subject matter.
• The assurance user [second party] needs to be able to trust the first party. This requires the assurance engagement to provide information that matches the criteria that matter to them, even if it requires subjective assessment
• The assurance provider [third party] needs to be able to review information provided by, or collected from, the first party about the subject matter. This requires the expertise to make a judgement about the trustworthiness of the subject matter.

• In an assurance engagement, there are often a number of different assurance users. The direct assurance user creates the demand for assurance, for example the executive requires assurance from the developer that an AI system will function as claimed. The eventual users of this system, affected individuals, regulators are all indirect assurance users in this example as they will also gain confidence/trust in the system through this engagement.

1. Subject matter: Identifying the observable subject matter that needs to be trusted or proxies where this subject matter is unobservable or difficult to observe.

Subject matter in assurance

There are many different systems, processes, products, organisations and people that someone might want to build confidence/trust in and therefore will require assurance about.

Someone might want to gain confidence in the trustworthiness of a product such as a medical device, a company’s IT system, a food manufacturing process, whether an organisation has trustworthy internal controls or whether a person is appropriately skilled or qualified to perform a particular task.

Assurance engagements enable people to build confidence in the trustworthiness of systems, processes, products, organisations and people by evaluating and communicating reliable information about their trustworthiness. The subject matter of an assurance engagement are the specific aspects of the system, processes, product, organisation or person that the user requires information about to build justified trust.

For example, when assuring an AI system, there are a number of distinct subject matters that need to be assured to build confidence and trust in the whole system.

Subject matters that can be assessed to build confidence in AI systems include their accuracy, robustness, bias/fairness, societal and human rights impacts, privacy, security, management and intended use. Each of these subject matters have different characteristics and therefore require different criteria and assurance techniques - such as impact assessments, different types of audits and conformity assessments - to assess their trustworthiness.

• What specific aspects of the system does the assurance user need to build trust in? For example, they need to trust that a system will perform in a certain way consistently, they need to understand possible societal impacts, and they need to trust that appropriate governance processes have been followed.
• Can we observe these aspects directly e.g. the accuracy of the system in performing a specific task? If we cannot observe these aspects e.g. potential societal harms, what proxies can we rely on to produce evidence about the trustworthiness of this aspect of the system? For example, a standardised and documented impact assessment which assigns accountabilities and mitigation processes can provide a proxy for trustworthy information about the potential societal harms an AI system might cause.

1. Suitable criteria: Having mutually understood ways of assessing the trustworthiness of this subject matter

• What measures or standards are appropriate for assessing the subject matter that needs to be assured? For example, you cannot use a performance standard to assure an acceptable threshold for societal risk because the potential harms you are trying to mitigate are inherently uncertain and unobservable. In this case, assuring a system to a specific risk threshold would provide false certainty, reducing the trustworthiness of the overall system.
• Are there commonly accepted standards for this subject matter? Or to what extent can commonly accepted standards be agreed for this subject matter? Not all subject matter can be standardised fully, for example, it is unlikely that we will agree on acceptable thresholds for bias in automated decision making.
• How can the assurance provider assess the trustworthiness of the subject matter, and to what extent can this be objectively measured? Can we reliably test for conformance against these criteria?
• Are the criteria adopted by the assurance provider [3rd party] understandable by the assurance user [2nd party] and do they provide a useful basis for establishing their trust in the subject matter?

1. Evidence: Having commonly understood methods for evaluating the trustworthiness of this subject matter.

• What sort of testing, review, or investigation is necessary to assess the subject matter against the criteria?

• What evidence does this measurement provide about the trustworthiness of the subject matter?

• Is the evidence provided Available to, Intelligible to, and accessible by the assurance users?

1. Conclusion: Providing a consistent and understandable declaration of assurance.

• What does the evidence collected tell us about the trustworthiness of the subject matter?

• How much certainty/confidence can we have in the conclusions we draw about the trustworthiness of the subject matter based upon the evidence collected?

• How can the level of trustworthiness be communicated in a way that is understood by the assurance user?

Mature ecosystems also make use of more automated tools and services to handle certain parts of the assurance ecosystem more efficiently. For example, automated, continuous cybersecurity monitoring and the automation of transactional work in accounting. However, automation doesn't eliminate the need for expert services to address the more complex and ambiguous areas, where expert judgement is required to build justified trust.

Across different mature assurance ecosystems we can see how different assurance models respond to different types of risks in different environments. We summarise findings from our comparative landscape analysis in the table below.

What are the three 3 most commonly sought assurance services?

What are the three parties involved in an assurance engagement?

What are the three statements of assurance?

What are the types of assurance engagement?