Magistrate Monitoring and Evaluation

Evaluation Guidance: Evaluation Tools

Questionnaires Interviews Focus Groups
Observations self assesment Self-assessment Verbal Feedback
self reflection Self-reflection

Verbal Feedback

General advice

Verbal feedback, whether formal or informal, should be used to supplement, support or as part of other evaluation activity.

Informal feedback primarily provides information for the facilitator.
Formal feedback can assist you to evaluate for validation, whether objectives met or for learning.

Informal verbal feedback

General

Informal verbal feedback is most often used to evaluate just after a training event. Delegates may provide verbal feedback during, at the close, or after a training intervention. Sometimes it may be in the form of a throw away comment or something more specific. It is important to place any informal feedback in context - it may or may not be representative of the majority view. Informal feedback may be used to support or validate other methods of evaluation, particularly where issues are described in a narrative format.

Good for

Informing the tutor of any immediate reactions to the training.
Raising issues that the facilitator may not have considered.
Allowing the facilitator to focus subsequent evaluations around specific issues.

Approach

Receiving and using feedback is an important communication skill that all trainers should develop. The fact that facilitators listen to feedback and are seen to act on it reinforces the value of the process in the learner's eyes.

Hints and tips

Be prepared to question feedback to determine the precise nature of the issue, for example, someone may say that a particular exercise was poor, the facilitator will need to ask questions to identify what led to that view - was it out of data? Was it not relevant to the objectives? Was it too focused on legal issues?
Try to encourage others to share their views in order to validate the feedback.

Analysis

The information that you gather is almost always qualitative in nature and will require careful analysis. A suggested approach for this analysis is:

Review findings/views to identify suggestions for improvement. Group similar points from verbal feedback together, by looking for common themes and phrases.
Assemble and report the results of this grouping exercise before making a separate analysis of the information. It is often a good idea to tabulate these results or use another graphical method such as a distribution chart to help you to see the overall picture.
Write up a report highlighting key points and issues raised. Always include 'minority' information with this so that the whole picture can be seen.
The information is perception based but by gathering a number of perceptions an overall picture can be put together.
Always share the outcome of your analysis with those providing the feedback.

Formal verbal feedback

General Advice

Forewarn delegates at the beginning of the training event, or at an appropriate time within it, that you will ask them to provide verbal feedback. This helps them to appreciate that verbal feedback is important.
It also conveys the strong idea that you value such contributions and that it might help with the design of future training.

Good for

Evaluating at validation, learning and meeting of objectives levels.
Carrying out formative assessment of training programmes.
Supporting and validating other forms of evaluation.

Approach

To encourage delegates to provide constructive feedback you should ensure that their verbal comments are:

Specific: pointing out examples of what worked well or not during the event (i.e. lunch arrangements, presentation of slides, syndicate exercises).
Immediate: encouraging delegates to provide comments on the event itself or shortly afterwards, not at a future date.
Based on issues: any criticism should be directed at aspects of the event itself (i.e. administration/organisation) not at you as the facilitator/tutor!
Helpful: concentrate on the positive and offer ways to take on board comments made (i.e. session too long - is this borne out by others?).
Not only related to criticisms: ensure that praise is recorded and noted.
Forward looking: feedback should not dwell on the negative aspects of an event but instead look to the future, developing joint solutions to problems.
It may help to prepare a brief for the session detailing the areas you want to explore. However, there is a fine line between a formal verbal feedback session and a focus group approach - compare this guidance with the guidance on focus groups to see which approach is appropriate.

Hints and tips

Ask appropriate open and probing questions. In this way you ensure that verbal feedback from delegates is adding some real value to the evaluation process. Some examples of such questions are:

What would make this process better?
Would you be prepared to..?
What makes you say that?
What specifically do you mean by..?
Can you say a little more about..?

Analysis

The information that you gather is almost always qualitative in nature and will require careful analysis. A suggested approach for this analysis is:

Review findings/views to identify suggestions for improvement. Group similar points together, by looking for common themes and phrases.
Assemble and report the results of this grouping exercise before making a separate analysis of the information. It is often a good idea to tabulate these results or use another graphical method such as a distribution chart to help you see the overall picture.
Write up a report highlighting key points and issues raised. Always include 'minority' information with this so that the whole picture can be seen.

Always be prepared to communicate any actions that you have taken, with the learners that have given feedback, in order to demonstrate the value of their inputs.