Evaluating Impact Post Assessment

Employees in a sales team attend a training program on developing effective communication skills. After the program, a talent development professional asks managers of the sales team members to complete an assessment. The assessment asks questions that help to evaluate if the employees were able to apply what was learned in the program in day-to-day work. Which of Kirkpatrick's Four Levels of Evaluation is the talent development professional primarily focusing on?

• Level 1

• Level 2

• Level 3

• Level 4

Answer explanation

Level 3 measures the extent to which employees are able to apply program content when on the job. Level 3 could be assessed through observation, manager assessment, and self-assessment.

A talent development professional works in an organization that addresses complicated social issues through social innovation and change initiatives. The organization mostly operates in complex and uncertain environments. The talent development professional has been assigned the task of evaluating the effectiveness of a recently delivered learning solution. Which evaluation approach should the talent development professional use that’ll best help to deal with the constant flux in the organization’s environment?

• Developmental evaluation

• Return on expectations

• Total quality management (TQM)

• Human Performance Technology (HPT) Evaluation Model

Answer explanation

Development evaluation is useful in complex or uncertain environments such as innovation, radical program redesign, or crises.

A talent development professional needs to evaluate the success of a learning curriculum. The talent development professional decides to use an evaluation method that is based on the curriculum’s goals and chooses the Ralph Tyler’s Goal Attainment Method. In order to use the approach effectively, which of these actions should the talent development professional take?

• Use five key steps: focusing and planning a case study, creating an impact model, implementing a survey for best and worst cases, documenting success cases, communicating findings.

• Use four levels of evaluation: reaction, learning, behavior, and results.

• Measure effectiveness from four perspectives: the customer perspective, the innovation and learning perspective, the internal business perspective, and the financial perspective.

• Ask four questions: What objectives should learners meet?; What learning activities will help learners to meet objectives?; How should the curriculum be organized?; and How should learner achievement be assessed?

Answer explanation

Ralph Tyler’s Goal Attainment Method poses four questions: What goals should learners achieve?; What learning activities will help learners to achieve goals?; How should the curriculum be organized?; and How should learner achievement be evaluated?

What is a difference between summative and formative evaluation?

• Unlike formative evaluation, summative evaluation takes place after a talent development solution has been implemented.

• Unlike formative evaluation, summative evaluation’s primary purpose is to improve the draft learning program.

• Unlike summative evaluation, formative evaluation uses participant reaction forms and stakeholder satisfaction surveys.

• Unlike summative evaluation, formative evaluation focuses on the impact of talent development solutions.

Answer explanation

As opposed to formative evaluation, summative evaluation takes place after a talent development solution has been implemented.

An organization sees an increase in sales after the implementation of a training program that aimed to improve knowledge-sharing practices between the sales and marketing teams. A talent development professional is tasked with determining whether the increase in sales could be attributed to the training program. To do this, the talent development professional isolates the effects of the training through a trend-line analysis. Which method of evaluation is the talent development professional most likely to be using?

• The Brinkerhoff Case Method

• The Phillips ROI Model

• Kirkpatrick's Return on Expectations

• The Balanced Scorecard Approach

Answer explanation

The Phillips ROI Model involves a step to isolate the effects of training, which helps to determine whether the training caused the expected results.

A talent development professional is using the Brinkerhoff Case Method to examine the impact of recently implemented organizational solutions. The talent development professional has completed the planning of a successful case study. What is the next step the talent development professional should take?

• Design and implement a survey to search for best and worst cases.

• Interview and document success cases.

• Communicate findings and recommendations

• Create an “impact model” that defines what success should look like.

Answer explanation

After planning a successful case study, the talent development professional should create an “impact model” that defines what success should look like.

What percentage is generally considered adequate for a test’s reliability coefficient?

• At or above 75 percent

• Above 35 percent

• Above 50 percent

• At or above 45 percent

Answer explanation

At or above 75 percent is generally considered adequate for a test’s reliability coefficient.

A talent development professional is conducting a survey to gather employees’ feedback on a newly formulated work process. The talent development professional must decide if the survey should be shared with participants via email or in person. In this scenario, what action is the talent development professional performing?

• Identifying which rating scale to use

• Determining how the survey results will be communicated

• Assessing the format that will be used to track the survey results

• Checking how the tool will be distributed

Answer explanation

When developing evaluation instruments, it is important for talent development professionals to check how the tool will be distributed. By checking if the survey should be shared with participants via email or in person, the talent development professional is assessing how the tool will be distributed.

A talent development professional developed an assessment for managers in a company to test decision-making skills. The talent development professional noticed with the exception of one employee, all other employees scored very high in the assessment. Which factor is likely to be true in this scenario?

• The assessment tested content that employees already knew well.

• There was not sufficient context provided to answer the questions.

• The assessment questions were mostly of a high difficulty level.

• The assessment included questions that were ambiguously written.

Answer explanation

If a majority of test takers score high in a test, it is likely that the assessment tested content that test takers already knew well, the test content was well taught, or the answers were cued in some way.

A talent development professional is developing a survey to gather data on employees’ preferences for lunch food items that the company provides during talent development events. Which action should the talent development professional take to ensure that the survey is effective?

• Ask multiple, complex questions at one time.

• Identify ways to obtain a high response rate.

• Make the survey as lengthy and detailed as possible.

• Use complex words and sentence structures.

Answer explanation

One of the ways to ensure effectiveness when developing surveys is to identify ways to obtain a high response rate.