Magistrate Monitoring and Evaluation

Self-Assessment Guidance

Completion of MATC Area self-assessments

Content

Introduction
Stage 1: Identification of Training Needs
Stage 2: Design of Training Solutions
Stage 3: Delivery of Training Solutions
Stage 4: Application of Training in the Court Environment
Stage 5: Evaluation of Training solutions

1. Introduction

General guidance for completing self-assessment reviews is issued to all Areas when they are notified that they need to submit their self-assessment to the JSB Monitoring & Evaluation (M&E) team for annual review. The notes below give additional support and guidance on completing MATC Area self-assessments for each of the stages in the Framework of Standards for Magistrate Training and Development (the Standards).

2. Stage 1: Identification of Training Needs

Have you provided narrative and illustrative documents to show:

How locally identified training needs are arrived at and included in the training process? (Standard 1:1a & b)
How the appraisal, mentoring and post-sitting review processes help to identify training needs? (Standard 1:1b)
How different agencies feed into the identification of training needs? (Standard 1:1f)
What improvements have taken place since the JSB M&E team's previous review of your self-assessment? For example, use of post-sitting reviews in identifying training needs.

Examples of evidence

Copies of minutes from Bench Training Development Committee (BTDC) and Magistrates' Area Training Committee (MATC) meetings that explicitly make it clear who identified and decided on the training.
BTDC formal training and development needs submissions to the MATC.
Appraiser, Mentor, Youth or Family Panel minutes of meetings or from records of completed post-sitting reviews and follow-up action.
Justices' Issues Group and/or Area Judicial Forum minutes of meetings where training needs have been discussed.
Any trainers' forums or legal managers' meetings that have identified any local training needs outside of the MATC/BTDC meetings.
Any discussions with external agencies, for example Probation Service, Crown Prosecution Service or Police, that have taken place about identifying training needs.
A local training needs analysis undertaken.
Articles that appeared in legal updates, rota bulletins or magazines that take account of training needs outside of formal training.
Training proposals from the local Magistrates' Association branch to the MATC.
Event evaluation questionnaires and summary sheets that have highlighted a particular training need.

These are examples of evidence and should not be taken as all being required to support Stage 1.

Support and help

Note any support or guidance required on the self-assessment template issued by the JSB M&E team. An example might be asking the M&E team for further help with understanding what is meant by the term success criteria.

Guidance on Training Needs Analysis and Success Criteria is available on the JSB Magistrate Monitoring and Evaluation website.

3. Stage 2: Design of Training Solutions

Have you provided narrative and illustrative documents to show:

How training is designed in a systematic manner? (Standards 2:1 –2:3)
How aims and objectives for training are clear and measurable? (Standards 2:1a – 2:2)
How training design is quality assured? (Standards 2:8 – 2:9)
What improvements have taken place since the JSB M&E team's previous review of your self-assessment? For example, use of critical readers.

Examples of evidence

Training materials and session plans for locally designed courses. For example, tutor notes, handouts, case scenarios, PowerPoint slides and so on
Use of JSB produced tutor packs.
Design of local distance/open learning workbooks, CD Roms, DVDs, e-learning.
Use of case studies within courses that reflect equal treatment and diversity guidelines.
Any trainers' forums, legal managers' meetings or workshops that have discussed any training design issues.
Event evaluation questionnaires and summary sheets/reports that have highlighted the need to make changes in the design of a course.
Event evaluation questionnaires that highlight the variety of training methods used within courses.
Any local training observations carried out by MATC/BTDC members or other designated persons.
Critical readers' feedback on design issues.
Any feedback received following pilot events.
Copies of MATC/BTDC minutes that refer to the design of training.
The procedure in place for quality assuring the design of training materials – the 'sign-off' process.

These are examples of evidence and should not be taken as all being required to support Stage 2.

Support and help

Note any support or guidance required on the self-assessment template issued by the JSB M&E team. An example might be asking the M&E team for further help with writing measurable objectives.

Guidance on Aims and Objectives is available on the JSB Magistrate Monitoring and Evaluation website.

4. Stage 3: Delivery of Training Solutions

Have you provided narrative and illustrative documents to show:

How training is delivered in a consistent manner? (Standards 3:2 – 3:5)
How training is prepared for? (Standard 3:1)
What effect using a variety of training techniques had on the delivery of training? For example, accelerated learning. (Standards 3:2 – 3:5)
What improvements have taken place since the JSB M&E team's previous review of your self-assessment? For example, use of MATC/BTDC observers.

Examples of evidence

Local observation of training undertaken by MATC/BTDC members, justices' clerk or other designated persons.
Use of trainer notes or session plans.
Any peer review meetings that trainers undertake after delivering courses.
Any trainers' forums, legal managers' meetings or workshops that have discussed any training delivery issues.
Event evaluation questionnaires and summary sheets that have highlighted the need to make changes in the delivery of a course. What changes have resulted?
Event evaluation questionnaires that highlight the variety of training methods used within courses.
Support offered by trainers in the use of distance learning or e-learning to magistrates.
Copies of MATC/BTDC minutes that refer to the delivery of training.
The procedure in place for quality assuring the delivery of training.

These are examples of evidence and should not be taken as all being required to support Stage 3.

Support and help

Note any support or guidance required on the self-assessment template issued by the JSB M&E team. An example might be asking the M&E team for further help with how to undertake local observations of training.

Guidance on Observation of Training and the Delivery of Training Solutions is available on the JSB Magistrate Monitoring and Evaluation website.

5. Stage 4: Application of Training in the Court Environment

Have you provided narrative and illustrative documents to show:

How application of training in the court environment is measured? (Standard 4:1)
How, through the appraisal and mentoring processes, improvements in competence and performance have occurred? (Standard 4:1b)
How post-sitting reviews support the application of training? (Standard 4:1)
How training is linked to court performance data? (Standard 4:1)
What improvements have taken place since the JSB M&E team's previous review of your annual self-assessment? For example, use of statistical data as a basis for key training themes.

Examples of evidence

Any trainers' forums, legal managers' meetings or workshops that have discussed application of training issues.
Copies of MATC, BTDC, Mentor, Appraiser, Youth or Family Panel minutes that refer to the application of training.
Key trends identified from the use of court performance data. For example, training required from patterns emerging from appeals to the Crown Court, ineffective trial data and so on.
Training bulletins/newsletters that highlight improvements in performance that have an indirect correlation to training. For example, fines enforcement.
The analysis of post-sitting reviews following a major training initiative. For example, Criminal Justice, Simple, Speedy, Summary (CJSSS).

These are examples of evidence and should not be taken as all being required to support Stage 4.

Support and help

Note any support or guidance required on the self-assessment template issued by the JSB M&E team. An example might be asking the M&E team for further help with how to interpret court performance data for training purposes.

The JSB Magistrate Monitoring and Evaluation website gives detailed information on evaluation principles.

6. Stage 5: Evaluation of Training Solutions

Have you provided narrative and illustrative documents to show:

How various methods of evaluation are used? (Standard 5:1b)
How improvements and refinements are made to training courses as a result of evaluation? (Standards 5:1 – 5:2)
How evaluation data is used for forward planning purposes? (Standards 5:1 – 5:2)
How longer-term evaluation strategies are determined and implemented? (Standards 5:1 – 5:2)
What improvements have taken place since the JSB M&E team's previous review of your self-assessment? For example, use of focus groups.

Examples of evidence

Any trainers' forums, legal managers' meetings, Appraiser and Mentor Panel meetings, or workshops that have discussed evaluation of training issues.
Copies of MATC/BTDC minutes of meetings that refer to the evaluation of training.
MATC evaluation sub-committee minutes of meetings.
Evaluation end-of-course questionnaires and/or summary sheets/reports prepared for MATC/BTDC meetings.
The MATC Evaluation Plan.
MATC Annual Report (latest available) extracts.
Training bulletins/newsletters that highlight how training is evaluated and the results from this.
Longer-term evaluation strategies and/or reports. For example, focus groups, structured interviews, questionnaires issued to magistrates 3-6 months after a specific training course.
Local thematic reviews undertaken. For example, appraisals, Framework of Standards.
A post-sitting review analysis and any common trends emerging.
Evaluation of local mentoring or appraisal processes.
Evaluation of local open/distance learning/DVD/CD Rom initiatives.

These are examples of evidence and should not be taken as all being required to support Stage 5.

Support and help

Note any support or guidance required on the self-assessment template issued by the JSB M&E team. An example might be asking the M&E team for further help with long-term evaluation strategies.

The JSB Magistrate Monitoring and Evaluation website provides detailed information on evaluation principles.

Further information

If you would like us to provide additional support or advice on this topic, please do not hesitate to contact the JSB M&E Team.