AIEB_Quiz5_Operations Management

According to the AI model development process described in the framework, what characterizes the relationship between the three main stages?

• They must be completed in strict sequential order without iteration

• Each stage is independent and can be completed by different organizations

• The process can be a continuous process of learning with iteration

• Only the first two stages require human oversight

Answer explanation

The AI model development and deployment process is not always unidirectional – it can, and

usually is, a continuous process of learning.

What is the primary consequence of building AI models using biased, inaccurate, or non-representative data?

• Increased computational costs and slower processing times

• Reduced model accuracy but no impact on fairness

• Increased risks of unintended discriminatory from the model

• Better performance in controlled testing environments

Answer explanation

Model AI Governance Framework 3.22 - If a model is built using biased, inaccurate or non-representative data, the risks of unintended discriminatory decisions from the model will increase.

The framework identifies three types of data lineage. Which type combines both backward and forward lineage approaches?

• Comprehensive data lineage

• End-to-end data lineage

• Bi-directional data lineage

• Complete data lineage

Answer explanation

Refer to Model AI Governance Framework 3.23 (iii) End-to-end data lineage combines the two and looks at the entire solution from both the data’s source to its end-use and from its end-use to its source.

What does "selection bias" refer to in the context of AI datasets?

• The bias introduced when humans manually select which data points to include

• When data used are not fully representative of the actual environment

• The tendency to select algorithms that favor certain demographic groups

• The bias that occurs when data collection devices malfunction

Answer explanation

Model AI Governance Framework 3.23 (c) - Selection bias: This bias occurs when the data used to produce the model are not fully representative of the actual data or environment that the model may receive or function in.

How can organizations mitigate the risk of inherent bias in datasets according to the framework?

• Use only data from a single, highly reliable source to ensure consistency

• Remove all data attributes that might be considered sensitive

• Have a heterogeneous dataset by collecting data from a variety of reliable sources

• Focus exclusively on quantitative data and avoid qualitative inputs

Answer explanation

Model AI Governance Framework 3.23 (c) ...organisations can mitigate the risk of inherent bias by having a heterogeneous dataset (i.e. collecting data from a variety of reliable sources). Another way is to ensure the dataset is as complete as possible, both from the perspective of data attributes and data items. Premature removal of data attributes can make it difficult to identify and address inherent bias.

What is the key difference between "selection bias" and "measurement bias" as described in the framework?

• Selection bias affects model accuracy while measurement bias affects model speed

• Selection bias can be corrected after modelling while measurement bias cannot

• Selection bias is more serious than measurement bias for AI applications

• Selection bias skews data representation while measurement bias is systematically skewed data collection

Answer explanation

Model AI Governance Framework 3.23 (c) - Selection bias: This bias occurs when the data used to produce the model are not fully representative of the actual data or environment...

Measurement bias: This bias occurs when the data collection device causes the data to be systematically skewed in a particular direction.

According to the framework, when should organizations consider using different datasets for training, testing, and validation?

• Only when working with large datasets where splitting doesn't significantly reduce data quality

• Always, regardless of dataset size, as it's considered mandatory practice

• Only for high-risk AI applications requiring regulatory approval

• When the organization has access to multiple independent data sources

Answer explanation

Model AI Governance Framework 3.23 (d) - ...the trained model can be validated using the validation dataset. It is considered good practice to split a large dataset into subsets for these purposes, if it does not lead to a significant reduction in the quality of data in terms of accuracy and representation.

What does "explainability" aim to achieve in AI systems according to the framework?

• Explainability aims to make AI systems' decisions and processes understandable to humans.

• Explainability aims to increase the speed of AI computations.

• Explainability aims to reduce the cost of AI hardware.

• Explainability aims to maximize data storage in AI systems.

Answer explanation

Model AI Governance Framework 3.26 - Explainability is achieved by explaining how deployed AI models’

algorithms function and/or how the decision-making process incorporates model predictions. The purpose of being able to explain predictions made by AI is to build understanding and trust.

When might it not be practical to provide information about AI algorithms according to the framework?

• When the algorithms are too complex for human understanding

• When disclosure might allow bad actors to avoid detection or expose confidential business information

• When the computational cost of explanation is too high

• When the AI system operates in real-time environments

Answer explanation

Model AI Governance Framework 3.29 - There may be scenarios where it might not be practical or reasonable to provide information in relation to an

algorithm. For example, disclosing algorithms deployed for anti-money laundering detection, information security, and fraud prevention may allow bad actors to avoid detection; likewise, providing detailed information about proprietary algorithms or

the decisions made by the algorithms may expose confidential business information.

What distinguishes "repeatability" from "explainability" in AI systems?

• Repeatability is more important than explainability for building trust

• Repeatability focuses on consistent performance while explainability focuses on understanding algorithm function

• Repeatability applies only to simple algorithms while explainability applies to complex ones

• Repeatability is a technical measure while explainability is a business measure

Answer explanation

Model AI Governance Framework 3.30 - Where explainability cannot practicably be achieved given the state of technology, organisations can consider documenting the repeatability of results produced by the AI model. Repeatability refers to the ability to consistently perform an action or make a decision, given the same scenario. ...some degree of assurance of consistency in performance could provide AI users with a larger degree of confidence.