Evolve Quality Day 3 Reviewer

Select which is not a type of calibration.

• Anonymous Calibration

• Metric Focus Calibration

• Training Calibration

• Client Calibration

What are the responsibilities of a Quality Manager or Quality Supervisor during Calibration? Select all that may apply?

• Prepare calibration roster and forward it to the delivery.

• Send the invite per schedule.

• Identify the sample/s to be discussed.

• All of the above.

What is Calibration?

• It is our method of ensuring alignment of Client expectations, whether those are from compliance standpoint, critical behavior, soft skills, etc.

• Calibration allows us to measure understanding or our auditors.

• Calibration is one of the processes that we get feedback on from our Delivery partners.

• Calibration helps identify gaps.

Why is Calibration important?

• Calibration impacts the end user’s experience.

• Poorly calibrated Evaluators could adversely impact the knowledge of the front-line staff.

• Calibration impacts staff scorecards, staff and program performance and the quality of information provided to management.

• Calibration is one of the cornerstones of the QA organization. 

Select which statements are correct about Calibration sample size. Select all that apply.

• 3-Shall be used across all programs in order to understand accuracy performance trend and to develop focus area for evaluators or specific attributes. This is the default minimum sample unless the program falls into another category as defined herein.

• 4-Shall be used in programs where major changes to support client’s products, services, systems, and procedures occurred within the past month and/or the current month.

• 2-Shall be used in programs where the past 6 monthly calibration effectiveness analysis results indicate that all evaluators are within the acceptable variance set for the program.

• 0-Shall only be used in programs where no major changes to support client’s products, services, systems and procedures occurred within the past month and/or the current month.

Select which statement is TRUE.

• Quantitative Calibration is designed to measure individual evaluator accuracy and attribute level accuracy.

• The Operations Manager shall be responsible for determining when to perform Training Calibration (new process, new business, onboarding, etc.) for the program supported, in addition to the regular Quantitative Calibration sessions.

• At the minimum, quantitative calibration must be performed weekly for each program.

• The Quantitative Calibration shall be designed to measure group evaluator accuracy and attribute level accuracy.

Select which statement is TRUE.

• Ops TL’s/managers/training team should be part of the audience/participant group during external and internal calibration.

• At minimum we would like to see least 4 calibration sessions/week.

• Calibration sessions should cover only one channel supported, whether that is voice, chat, email, messaging.

• Ideally, we target <= 15% with 85%-line item accuracy as our target, however it may vary from process to process

Select which statements are correct about Gauge and R&R.

• Repeatability tells that our QE’s are able to rate the same call the same way as the first time.

• Reproducibility tells that our QE’s are able to rate the call the same way as the gauge.

• Sources of variation may be the gauge, the evaluators or the form used in the calibration.

• When you conduct a Gauge R&R study and calculate the acceptable variance is 40%.

Select which statement is TRUE.

• Individual evaluator accuracy measures the accuracy of the evaluator’s scoring, as compared to the gauge’s scores. It is calculated as: # of attributes scored correctly / total attributes assessed

• Attribute level accuracy measures the accuracy of the evaluator group’s score as compared to the gauge’s score, by attribute. It is calculated as: # of evaluators who scored the attribute correctly / total number of evaluators.

• Every calibration will be a Quantitative Calibration session.

• Call Listening session involves the use of an audit form for calibration.

In Gauge R&R study, what is the guide for acceptability.

• Less than 10% - the measurement system is acceptable.

• Between 10% and 30% - the measurement system is acceptable depending on the application, the cost of the measuring device, cost of repair, or other factors.

• Greater than 30% - the measurement system is unacceptable and should be improved