Consensus Moderation

Consensus Moderation

An exemplar of post-assessment moderation

Moderation refers to a range of processes for checking and confirming that assessment is explicit and valid, fair and consistent. Moderation is a quality measure ensuring that assessment tasks and grading practices are rigorous and align with the JCU Learning and Teaching Assessment Policy. Consensus moderation is a process like peer review that has reaching consensus as its aim (Duncan D Nulty, 2012). This exemplar is about reaching consensus around our expectations for the standard to each grade awarded and its focus is on consensus moderation in the post-assessment phase.

Core activities in a consensus moderation process include:

Pre-assessment moderation - In these activities all teaching staff review the requirements for upcoming assessment, including the criteria by which they will be graded, and come to a shared understanding (consensus) of the requirements for the task so that they are enabled to communicate consistent messages to students about the required tasks. Post-assessment moderation - In these activities, consensus is sought on the grading of tasks following completion.

This exemplar looks at the consensus moderation process in the Bachelor of Education second year subject, ED2491: Managing Teaching and Learning 1. ED2491 is a professional practices subject delivered to four modes or cohorts (Cairns internal students, Townsville internal students and external students in an Early Childhood major and a community based Indigenous teacher education program). In 2012, the subject had a teaching team of three lecturers, including the cross campus coordinator, who were supported by three sessional markers. The School’s workload model allows one hour per student marking time per study period. Staff are encouraged to design assessment tasks that can be marked within this time frame.

The ED2491 assessment schedule involves two tasks wherein students critically reflect upon the planning and implementation of lessons in their prac school setting, with a view to outlining strategies and implications for enhanced practice. Students’ critical reflections are informed by the education literature and multiple sources of evidence (e.g. feedback from their peers, School based Teacher Educators and their own classroom students).

All ED2491 students across the four modes of delivery share the one LearnJCU subject platform enabling them access to a range of course materials and learning spaces and the cross campus coordinator to provide weekly syntheses and assessment task pointers. The subject coordinator states that ‘we want to communicate to our students that we prioritise their experience, that it is a high quality delivery, that it is equitable and that students have access to all materials and information’. Within this collaborative context then ‘moderation is a satisfying process and everyone involved has genuine input.’ Moderation processes are explicitly outlined to students, and there is real investment in communicating assessment task requirements and expectations to students, as reflected in the following SFS results (ED2491, 2011):

The staff made it clear right from the start what they expected from students: 4.62/5

The assessment requirements and criteria were clearly specified: 4.46/5

Background

Dr. Lasen explains that consensus moderation is supported through detailed task descriptions and criteria-standards descriptors. Assessment is the aspect of a subject that comes under the most scrutiny by students and, hence, ‘the level of detail in the task description, the way that the assessment task is supported in class, the degree of specificity in the rubric descriptors, the extent of constructive feedback that is provided to students and their awareness of the moderation processes that are adhered to’ are all important in terms of student satisfaction with assessment.

This is true for all subjects, especially so when assessment is high stakes. In ED2491, students find the assessment tasks to be professionally relevant yet challenging. Further, there is a whole-of-school commitment to academic literacies serving as a discriminating criterion in ED2491. Students have to meet minimum standards in the academic literacies criterion to pass the subject. Students need to pass ED2491 to proceed to their second-year extended practicum. Failure of ED2491 has implications for course progression.

One aspect of the course coordinator’s role is to monitor the group of students who require additional literacy support. Students are identified as such through their performance on a non-weighted formative assessment task and the first weighted task. Importantly, the course coordinator pays particular attention to moderating the low pass-fail boundary across modes. Detailed feedback on tasks and additional support are provided to students who have not met minimum criteria. In spite of the high stakes nature of the assessment, the subject coordinator suggests that in the previous study period (2011), there was not a single student appeal ‘which I believe is a testament to assessment and moderation processes that have been refined over a number of years across the four modes and involving all markers.’

Consensus Moderation in Practice

Assessment tasks are submitted both electronically and in hard copy. Prior to the submission of each task, all markers receive a marking pack containing the task description and key supporting materials that students have been given in order to prepare themselves for the task.

Processes:

Cross campus processes

  1. The cross campus coordinator selects four samples – one submission from each of the four cohorts. These samples are sent via email to all markers, who assess them against the task rubric.

  2. All markers in Cairns and in Townsville meet on Skype and begin dialogue, starting with broad questions like:

Of the four samples have any students failed? OR

Have any students failed on the academic literacy criteria?

  1. For every sample, each criterion is then systematically addressed with markers providing rationale for their judgments.

  2. On some occasions, a detailed marking guide is put together after the Skype session to document points agreed upon in the moderation meeting.

  3. Markers go away to mark their quota within the allocated three week marking period.

  4. The subject coordinator continues to send samples via email to the marking team to further support marker judgments.

Within campus processes

  1. Lecturers continue to work with markers on respective campuses to further moderate cases where there is any uncertainty at the boundaries (N-P; P-C; C-D; and D-HD).

  2. Lecturers finally moderate all submissions that have received a fail or low level pass.

Refining Processes

Cross campus and within campus moderation processes are undertaken every year. So too, the cross-campus coordinator reviews grades distributions across the respective cohorts, however, in the knowledge that cohorts do present differently. Assessment and moderation are treated seriously and tasks, rubrics and processes continue to be refined. Marking guides and student questions generated in one year serve to inform the assessment task descriptions and rubric descriptors the following year. The team now seeks to further support student success by developing annotated samples of student responses at different standards.

For further information on any aspect of this exemplar, please email Dr Michelle Lasen on michelle.lasen@jcu.edu.au

Links:

JCU Learning, Teaching and Assessment Policy