Magistrate Monitoring and EvaluationMagistrate Monitoring and Evaluation

 

Evaluation Guidance: Evaluation Tools

Questionnaires Questionnaires Interviews Interviews         Focus Group Focus Groups
observation Observations   self assesment Self-assessment feedback Verbal Feedback
self reflection Self-reflection

Self-assessment Questionnaires

General advice

Self assessment questionnaires can be used to support evaluating learning and objectives. As the approach is similar irrespective of the reason, the following applies to each of these reasons. Self assessment questionnaires (SAQs) are often used in learning materials and sessions to review and reinforce learning points. They are a useful tool that allows the learner and facilitator to establish what has been learned so far, before moving on to the next set of learning points. Their use for evaluation is less common and usually restricted to the validation and learning levels. SAQs may be called a number of different things - learning checks, quizzes, section reviews etc. Essentially all of these are the same thing. Their use as an evaluation method is usually formative, i.e. to check within the training that points have been understood and if not, to allow the facilitator to remedy the situation.

Good for

  • Interim checks on learning and can be used for validation. This may be particularly useful with new training.
  • Involving the learners in the training with a degree of interactivity.
  • Allowing learners time to reflect on what they have learned or not learned in an immediate way rather than wait for an end of course questionnaire.

Back to top

Approach

  • Learners vary in their preferences for different formats.
  • Using a variety of methods will help learners do their best.
  • Multiple-choice, short answer and true - false questions are appropriate for assessing learners' understandings of details and specific knowledge, and will be suited to most learning checks.
  • Examples of different question types are given later in this section.

Hints and tips

  • Prepare clear instructions so that learners are clear on what is to be done.
  • Place several questions all learners can answer at the beginning of the SAQ.
  • Ensure that the time allocated to undertake the SAQ is appropriate. As a rule of thumb, allow about one-half minute per item for true-false questions, one minute per item for multiple-choice questions and two minutes per short answer requiring a few sentences.
  • Consider making SAQs cumulative. Cumulative SAQs require learners to review materials already covered, thus reinforcing what they have learnt. An example for this approach could be the Induction programme for new magistrates. However, be aware that questions which are dependent upon a previously correct answer must be used carefully so as not to demotivate. If you use such questions make sure that participants have access to the correct answer before being required to answer the second question.
  • Give some thought to the layout of the SAQ. Use margins and line spacing that make the learning check easy to read. Group similar types of items, such as all true-false questions, together. Keep in mind that the amount of space left for short answer questions often signifies to learners the length of the answer expected of them.

Back to top

Analysis

  • As the results of the SAQ are to be used immediately it is often desirable for learners to self mark their SAQs or have them marked by a colleague.
  • You will need to decide what threshold of marks in your marking scheme represents the results you want. For example, do all learners need to answer all questions correctly or is their some room for error?
  • The rapidity of the analysis for use as formative evaluation does not preclude the deeper analysis post event of the SAQ results. This can be useful for the facilitator to look at emerging trends.

Different question types

  • Multiple choice questions can be used to measure knowledge concepts. Since multiple choice questions can be answered quickly, trainers/facilitators can assess learners' understanding of a topic in a short quiz.
  • Short answer questions can call for one or two sentences or a paragraph. Short answer questions are easier to write, though they can take longer to score, than multi-choice versions.
  • True/false questions are less reliable due to the possibility of random guessing producing the correct answer. However these types of questions are appropriate for occasional use. Inserting an "explain" column after a true-false question in which learners write a sentence justifying their response provides a suitable option.
Click here Word Icon [26KB] for an example of a SAQ using true/false and short answer questions in Microsoft Word. More question types are covered in the Questionnaire section of this guidance

Back to top

Home | Guidance | Training Plans | Frameworks | Feedback
Crown copyright 2007