JCU - Quality Assurance

James Cook University conducts a program of Course Reviews on five-year cycles for all coursework postgraduate degrees. Each Faculty will conduct Postgraduate Course Reviews following guidelines and a schedule established by Academic Board. Because postgraduate course enrolments can be quite small it is expected that Faculties will group courses for Review. Where it is appropriate, coursework doctorates may be grouped with other postgraduate degrees. The groupings will be subject to the approval of the Chair of Academic Board who will consult with the Pro-Vice-Chancellor Staff Development and Student Affairs. The term 'course' is used in this document to designate individual courses, or programs and courses clustered together by the Faculty for the purposes of each Postgraduate Course Review. The term is taken to embrace the subjects and various clusters of subjects that make up the course or courses involved. It is expected that a Postgraduate Course Review Committee will comment on the whole cluster or parts thereof at its own discretion. The two key purposes of Postgraduate Course Review are to:

(i) establish regular cycles of disciplined reflection and course enhancement; and

(ii) provide information to Academic Board to enable it to assure the academic quality of courses to the University Council and to the community.

The Academic Board has this authority and responsibility under the James Cook University Act s40.(3):

The academic board is to – (a) advise the council about teaching, scholarship and research matters concerning the university; and (b) formulate proposals for the academic policies of the university; and (c) monitor the activities of the university's faculties; and (d) promote and encourage scholarship and research at the university.

Academic Board will expect the Postgraduate Course Review to anticipate the interests of key stakeholders. The key stakeholders are taken to be students, the community (including the professions), the industries in which graduates are employed, and other academics working in the same field. In other words, underlying questions concern the appropriateness of the course and its constituent parts. Is this course appropriate for students? Does it appropriately address a community need or employment opportunities? Do the students attain appropriate academic standards in a legitimate field of enquiry and social practice? The Postgraduate Course Review Committee will include expertise capable of making informed judgments on these matters. To inform its work, the Committee will examine relevant evidence, and will consult directly with students and staff associated with the course.

Where courses are also subject to statutory external accreditation, in some cases it may be desirable to hold Postgraduate Course Reviews prior to or in concert with external accreditation in order to take advantage of a rationalisation of effort in the production of documentation. However, separating the two processes may allow for greater reflective discussion of course enhancement and strategic planning and link more closely with the university's quality assurance needs, especially in relation to the Australian Universities Quality Agency audit. The Postgraduate Course Review Committee will be established by the Executive Dean following guidelines established by the Academic Board. The Review Committee will report to the Executive Dean and to the Academic Board and thence to the Council. The Executive Dean is required to respond simultaneously to the Review in a submission to the Academic Board outlining actions in response to recommendations of the Review and the draft timetable for reporting back to the Board. Academic Board will take into account the views of any relevant accreditation authority and the Review Committee in its assessment of a revised Faculty proposal for Course Approval.

Return to Contents

2. KEY QUESTIONS FOR POSTGRADUATE COURSE REVIEW

1. How and how well do the aims of the course relate to the University's role and mission?

2. How do the aims, processes, and assessment practices of the course comply with University aspirations for internationalisation?

3. Is the course appropriate to the stated aims of the academic unit?

4. Does the course attract students appropriate to its aims?

5. Are the completion, pass, and retention rates appropriate?

6. How well does the course attend to the needs of its intended (professional) clientele?

7. What are students' views of the course: are they satisfied with the academic standard, content, organisation, structure and teaching (including flexible delivery) of the course?

8. What are the students' views about library support for the course?

9. What are the students' views about IT support for the course?

10. What are the students' views about other support services, for example, student administration, School/Faculty administration, the Research Office, careers advice, language support for students from non-English speaking backgrounds, counselling?

11. Are the organisation, structure, and content of the course appropriate?

12. Are the methods of delivery and assessment articulated with the objectives?

13. Is the course and subject documentation adequate?

14. Are academic standards appropriate?

15. How do the aims, processes, and assessment practices of this course comply with University objectives dealing with the attainment of Postgraduate Generic Skills?

3. HOW WILL THE COURSE REVIEW BE CONDUCTED?

The Postgraduate Course Review process will extend over a five-year cycle. For a good part of that time, information will be set aside in preparation for the formal part of the Review which will take a few days. The early phase can be envisaged as the establishment of a series of files, which will aggregate into documentation to be provided to the Postgraduate Course Review Committee when it is appointed. To keep the task for the Committee and the compilers of the documentation manageable and realistic, some careful selection and organisation of information will be necessary. The documentation gathered will be provided to the Committee prior to its formal meeting at the University. The Committee will seek further information during its deliberations, primarily by interviewing and meeting (in focus groups) with staff and students participating in the course. The relevant School(s) will be responsible for the organisation of reasonably representative contact with staff and students.

The checklist of questions and sources of data presented below will guide the process of documentation and subsequently the program of contact with staff and students for the Postgraduate Course Review Committee.

Return to Contents

4. COMPOSITION OF THE REVIEW COMMITTEE

The composition will be:

· Chair, appointed by the Vice-Chancellor on the advice of the Executive Dean and Chair of the Academic Board. The Chair will be an external, expert consultant from the profession or field and would be expected to have experience in conducting similar courses or programs;

· Two senior staff with relevant disciplinary knowledge from other Faculties, at least one of whom is currently a member of Academic Board; both nominated by the Chair of Academic Board;

· A nominee of the President of the Postgraduate Student Association. The nominee is not to be involved with the course being reviewed. (It is noted that participation by a student will be a substantial time commitment, and that therefore this position on the Committee may not always be filled, or may be filled by an employee of the Student Association).

Support for the Review Committee will be provided by the Faculty.

5. WHAT ARE THE KEY SOURCES OF DATA FOR THE REVIEW?

The checklist of data sources presented here expresses a serviceable minimum of points of reference for the Postgraduate Course Review. A Review Committee will want to explore the meaning of the information presented to it and to seek further information and help from staff and students in drawing inferences about it. The Committee is not expected to make unnecessary work for course teachers and coordinators, but may be expected to delve more deeply into some areas than this abstracted description suggests. Of course, different emphases will be pursued with different courses. Each Postgraduate Course Review Committee should determine the most appropriate emphases for the course being reviewed. To ensure that the Review is comprehensive, a range of sources and types of information should be used. However, each Review Committee will be appraised of the need to be reasonable in any requests for further information.

Aside from the basic data collection and Review process described here, Faculties will not be expected to conduct evaluations of courses, subjects or groups of subjects specifically for the Review. Nevertheless, there will be many existing external accreditation activities and other evaluations (including self-evaluations) of subjects, groups of subjects and whole courses which will be of interest and use to a Review Committee. Attention may be drawn to these as part of the documentation.

The Faculty Executive Dean, together with the Head of School for each course under review, will be invited to arrange for the preparation of a document outlining the background and history of the course, the internal mechanisms for course enhancement, review and feedback, together with a summary of the main issues and details of any proposed changes to the course not yet reflected in the official documentation. The document should also address aspects of the Fifteen Key Questions for Course Review, including internationalisation, the development and articulation of generic skills, on-line development and flexible learning and information on the destinations of graduates. Copies of the School strategic plan, and any other existing School documentation including policies, guidelines, practices and procedures should be appended to this document. This will be distributed to the Review Committee with the course review documentation four weeks prior to the course review.

Return to Contents

6. REVIEW TIMETABLE

The Postgraduate Course Review Committee will normally meet for one or two days. The Faculty and School concerned will prepare a program of contacts with staff and students in consultation with the Committee Chair.

7. SAMPLING

7.1 Subjects

The Review process involves sampling of student work in the document accumulation phase, and of students and possibly teaching staff for the interview phase. There may be wide variation across courses about how this will be done. Clearly the aggregation of vast amounts of student work is both onerous for staff and poor communication to the Course Review Committee. Sampling requires commitment to ‘representativeness' and the following principles should be kept in mind:

1. Key subjects should be sampled across disciplines and fields where there is such diversity in the course.

2. A balance between compulsory subjects and electives should be achieved where this is relevant.

3. Levels of achievement should be sampled, especially illustrating the transition from one grade level to another.

4. An appropriate mix among kinds of assignments and examination papers should be selected. Staff should anticipate the need to keep copies of selected samples in the semester prior to review.

The idea here is to provide evidence that standards are observed and that some thought has gone into the distinctions between performance at different levels. The examples are not ‘representative' but rather indicative of assessment practice and standards. A total of approximately 50 samples of students' work would be an upper limit for all courses. Where the products of student work are not amenable to inclusion in a document portfolio, a display during the meeting of the Committee may be appropriate as a complement or substitute.

Return to Contents

8. GUIDELINES FOR FOCUS GROUPS

8.1 The Purpose of Focus Groups

The purpose of holding ‘focus groups' for separate sessions of staff and students during the course review process is to enable the Committee to obtain information that will assist them in answering some of the fifteen key questions to be addressed in their report. It is recognised that some course enrolments may be quite small or students quite distant from main campuses. The Committee may use teleconferencing with a reasonable sample of students in such cases.

Information gleaned from interviews will be both supplementary and complementary to other sources of information that will include university mission statements, handbook entries, subject descriptions, statistical data, written submissions, sample assessments and survey data.

Properly constituted focus groups will provide the Review Committee with first hand accounts of the course under review. They have the potential to contradict or authenticate other evidence presented to the review committee and allow it interrogate aspects of the course more rigorously. They can also be used to clarify or elaborate on issues raised in other evidence to the panel. In some circumstances the Review Committee will rely entirely on the focus groups to receive information on certain aspects of the fifteen key questions.

In courses where student numbers are low the Committee may be dependent on quite small numbers of informants. Special effort must be made here to test whether the views expressed are idiosyncratic.

8.2 Organisation of Focus Groups

It is recommended that one person, usually from within the School that has responsibility for the course under review, be appointed to co-ordinate the focus groups and the documentation aspects of the course review. This person would be designated the Postgraduate Course Review Coordinator. It is the responsibility of the designated ‘Postgraduate Course Review Coordinator' for each review to prepare a program of contacts with staff and students in consultation with the Review Chair.

An important role of the ‘Postgraduate Course Review Coordinator' will be in constituting the focus groups, and briefing them prior to the review. Liaison with staff and students in both Townsville and Cairns (or elsewhere where appropriate) will be necessary.

8.3 Staff (or Staff Focus Groups)

It will be more efficient if the review committee meets with staff as a group and openly discusses issues raised in the key questions. The ‘Postgraduate Course Review Coordinator' should ensure that all staff are aware of the purpose of the review, and distribute copies of relevant key questions prior to the scheduled meeting. The Review Committee may, however, decide to depart from the listed questions.

The ‘Postgraduate Course Review Coordinator' should ensure that a cross-section of staff is represented in both Townsville and Cairns or elsewhere, if appropriate. This should include a range of teaching staff across academic levels, and from a range of subjects within the course. Staff should be free to express their views but should indicate to the Committee the level of support they have for those views among their colleagues.

The ‘Postgraduate Course Review Coordinator' should prepare a profile of all staff members for distribution to the Committee and indicate to the Committee which staff members will be available for the focus groups. This staff profile should include the following aspects: academic level, qualifications, subjects taught and the address of any existing personal web page.

The Review Committee may request to interview other staff not represented in the focus group if deemed appropriate in a follow-up session.

8.4 Key Questions for Staff Focus Groups

· How do the aims, processes and assessment practices of this course comply with University aspirations for internationalisation?

· Does the course attract students appropriate to its aims?

· Are the organisation, structure and content of the course appropriate?

· Are academic standards appropriate?

· How well does the course attend to the needs of its intended (professional) clientele?

· How do the aims, processes and assessment practices of this course comply with University objectives dealing with attainment of Postgraduate Generic Skills?

8.5 Student Focus Groups

Students will be an important source of information for the Postgraduate Course Review Committee. The identification of participants should ensure that a range of voices is heard, and that individuals have some authority to speak on behalf of others. Student participants should be encouraged to solicit input from others.

The ‘Postgraduate Course Review Coordinator' should ensure that student focus groups are a representative and demographic sample from participants in the course. (While ‘convenience sampling' will sometimes be unavoidable, this should be combined with ‘judgement sampling'.)

The following categories should be addressed in constituting the groups:

· Students from a range of subjects that reflect the course as a whole

· Students with different experience

· Students from Townsville, Cairns and regional origin

· International students

· Gender balance

· Different abilities - high, average and low achievers.

A profile of the focus group participants (with the exception of the last category) should be provided to the Review Committee. This profile should be seen to be not dissimilar to the profile of the course participants provided to the Review Committee in their documentation. The Review Committee may ask to interview a particular category of student during their follow-up sessions.

Students should be free to express their views but should indicate to the Committee the level of support they have for those views among their colleagues.

8.6 Key Questions for Student Focus Groups

· What are students' views of the course: are they satisfied with the academic standard, content, organisation, structure and teaching (including flexible delivery) of the course?

· What are the students' views about library support for the course?

· What are the students' views about IT support for the course?

· What are the students' views about other support services, for example, student administration, School/Faculty administration, the Research Office, careers advice, language support for students from non-English speaking backgrounds, counselling?

· How do the aims, processes and assessment practices of this course comply with University objectives dealing with attainment of Postgraduate Generic Skills?

8.7 Briefing Session for Focus Groups

Prior to the focus groups, the ‘Postgraduate Course Review Coordinator' should hold separate briefing sessions to inform staff and students as to the purpose and aims of the course review. They should explain the procedure of the focus groups and distribute the key questions. This will maximise the time available in the focus groups by eliminating the need for focus group participants to address questions to the Review Committee. In the case of students it will also allow time to canvass views widely on the key questions.

8.8 Focus Group Procedures

It is intended that all Committee members will be present at all the focus groups. These should be structured around the key questions and be semi-formal in nature.

The focus group will be coordinated by the Chair of the Review Committee (or nominee) and should allow enough time to address all the key questions. Administrative staff will record the key statements at these focus groups for review by the Committee.

Each focus group will be typically 50 minutes duration.

Return to Contents

9. WRITTEN SUBMISSIONS

The Postgraduate Course Review Committee will be expected to take written submissions from any staff member or student who wishes to provide them. These will be solicited in time to include in the Review Committee documentation (minus 4 weeks from review) or tabled during the review. Time will be set aside to interview the authors of written submissions during the review process if necessary. The Committee will protect the anonymity of authors of submissions where this is requested. Such authors should mark their submission “Confidential”. The Committee will be responsible for the confidential disposal of such information immediately the final report is complete and accepted.

Return to Contents

10. HOW WILL THE COURSE REVIEW BE REPORTED?

The report of the Course Review should be concise enough to be included as part of Academic Board papers. The portfolio of documentation prepared for the Postgraduate Course Review Committee should be available to Academic Board, as if it were an appendix to the report. The appendix should be kept in the Faculty for future reference. The Report should attend explicitly to the questions above, but present a precis of the key findings.

It is expected that a preliminary draft version of the Review Report will be prepared before the Committee breaks. The author of the report will be the ‘Executive Officer' appointed by the Faculty to service each Postgraduate Course Review Committee. The draft report will be circulated to the Committee members to be checked for fairness, relevance, and accuracy. Once finalised, the report will be submitted to the Executive Dean to be checked for factual accuracy and in order for the Faculty Executive Dean to prepare a response to the report. The Chair of the Postgraduate Course Review Committee will have the right and responsibility to mediate claims against these criteria.

10.1 Components of the report

1. Commentary on the aims and objectives of the course, and on how individual subjects contribute to those aims and objectives.

2. A summary of relevant performance indicators for the course and how well they are achieved.

3. A synthesis and summary of the issues arising in the Review, and of the strengths and weaknesses of the course, together with an account of the degree to which the course is internally coherent and contributes to University and Faculty goals.

4. Recommendations concerning any changes to the course to be considered by the Executive Dean and Academic Board.

On finalisation of the Review Report it will be considered by Academic Board along with a response from the Executive Dean of the relevant Faculty.

10.2 Appendices to report

Appendices to the report will include the information provided to the Postgraduate Course Review Committee and identified on the Information Sources Checklist. Other information provided to the Committee (for example, evaluation studies done on subjects, majors and the like) should be included for reference. Submissions to the Review unless specifically marked “confidential” should be included. Submissions marked “confidential” must be shredded immediately on completion of the Review.

Return to Contents

APPENDIX A: Summary of Course Review Tasks and Responsibilities

Stage 1: Review Panel Constituted

Task	Who responsible	Who approves	Comments/ Follow up action
Date of Review set	Executive Dean	Chair Academic Board	Confirmation that key personnel available
Nomination of external reviewers as Chair	Executive Dean	Vice Chancellor Chair Academic Board
Invitations to external reviewer	Executive Dean		Book Travel & Accommodation
Appointment of review panel - internal	Executive Dean	Chair Academic Board	Invitations to: · 2 senior staff from other faculties, at least 1 from Academic Board · 1 nominee of President of Postgraduate Student Association; not from course
Facilities booked - TSV & Cairns (if appropriate)	Faculty Office	Executive Dean

Stage 2: Documentation Folder generated

Task	Who responsible	Who approves	Comments/ Follow up action
Preparation of background paper on the course.	Executive Dean
Briefing meeting with Executive Dean (ED) and School Postgraduate Course Review Coordinator (PCRC)	Executive Dean		Action list compiled and assigned
Sample subjects decided	ED & Postgraduate Course Review Coordinator PCRC
Sample student work scoped	ED & PCRC	ED	Maximum 50 samples of students' work plus Honours
Sample student work selected	PCRC	ED
University mission statement assembled	Faculty
Course documentation assembled	PCRC	ED	Course proposals Handbook descriptions Subject descriptions Assessment practices Other School policies, guidelines, practices and procedures
Compilation of any existing reviews	PCRC
Folder completed at minus 4 weeks	ED & PCRC	ED

Stage 3: Statistical information generated (previous 5 years data where available)

Task	Who responsible	Who approves	Comments/ Follow up action
Course codes identified and agreed	Manager, Statistics	ED
International student enrolment profile, Completion, pass and retention rates, Staff/student ratios, Staff profile, qualifications and student load	Manager, Statistics
Graduate Destination Survey	Statistics
Course Evaluation Questionnaire	Statistics
SFS 2000-	SDSA (Jodie Simpson)

Stage 4: Timetable for staff and student focus groups developed

Task	Who responsible	Who approves	Comments/ Follow up action
Students selected	PCRC		Geographic distribution important
Staff selected	PCRC
Timetable developed	PCRC	ED

Stage 5: Pre-Review actions

Task	Who responsible	Who approves	Comments/ Follow up action
Written submissions invited and assembled, including confidential submissions	ED
Agenda prepared and distributed to panel	PCRC	ED
Documentation folders distributed to panel at minus 2-3 weeks	ED	ED
Travel to Cairns & accommodation or Videolink booked if appropriate	Faculty Administration
Catering organised	Faculty Administration

Stage 6: Review

Task

Who responsible

Who approves

Comments/ Follow up action

Review Panel meets, as per agenda

Review panel

Including trip to Cairns or Videolink if appropriate

Designated staff and students available to review panel

Report drafted

Executive Officer

Chair

Stage 7: Post Review

Task	Who responsible	Who approves	Comments/ Follow up action
Draft report sent to Review Panel	ED	Chair Review Panel to mediate claims
Draft report sent to Faculty Executive Dean to be checked for factual accuracy	ED	Chair Review Panel to mediate claims
Final Review Panel Report sent to Academic Board	ED		With documentation as appendix (minus confidential submissions)
Written response to Review Panel Report by Executive Dean sent to Academic Board	ED
Disposal of confidential submissions	Chair AB
Copy of Report and Documentation held by Faculty	ED

APPENDIX B: Information Sources Checklist

CRITERIA	Document Review	Peer Review	Graduate Survey	Employer Survey	Visiting Committee Interviews	Student Survey	Bench-marking	Performance Data
1. Are the aims of the course appropriate to the University's role and mission?	Mission statement Course proposal Handbook entry
2. How do the aims, processes and assessment practices of this course comply with University aspirations for internationalisation?	Course proposal Course information for students Assessment tasks	Course proposal Course information for students Assessment tasks			Staff interviews Student interviews			International student enrolment statistics
3. Is the course appropriate to the stated aims of the academic unit?	Mission statement Course proposal
4. Does the course attract students appropriate to its aims?	Student profile				Staff interviews Student interviews
5. Are the completion, pass and retention rates appropriate?								Internal and DETYA statistics
6. How well does the course attend to the needs of its intended (professional) clientele?			GDS		Staff interviews Student interviews			GDS
7. What are students' views of the course: are they satisfied with the academic standard, content, organisation, structure and teaching (including flexible delivery) of the course?			CEQ		Student interviews	SFS

CRITERIA	Document Review	Peer Review	Graduate Survey	Employer Survey	Visiting Committee Interviews	Student Survey	Bench-marking	Performance Data
8. What are the students' views about library support for the course?					Student interviews	SFS
9. What are the students' views about IT support for the course?					Student interviews	SFS
10. What are the students' views about other support services, for example, student administration, School/Faculty administration, the Research Office, careers advice, language support for students from non-English speaking backgrounds, counselling?					Student interviews	SFS
11. Are the organisation, structure and content of the course appropriate?	Handbook entry Course information	Handbook entry Course information	CEQ		Staff interviews Student interviews
12. Are the methods of delivery and assessment articulated with the objectives?	Handbook entry Course information Assessment tasks (samples)	Handbook entry Course information Assessment tasks (samples)			Student interviews	SFS
13. Is the course and subject documentation adequate and easily accessible?	Documents above	Documents above					Comparator contacts

14. Are academic standards appropriate?

Completed, & assessed student work

Grade distribution for key subjects

Completed & assessed student work

Grade distribution for key subjects

Staff interviews

Student interviews

SFS

Comparator

contacts

15. How do the aims, processes and assessment practices of this course comply with University objectives dealing with attainment of Postgraduate Generic Skills?

Course proposal

Course information for students

Assessment tasks

GSA

Course proposal

Course information for students

Assessment tasks

GSA

Employer survey

Employer participation

Staff interviews

Student interviews

Acronyms and terms Comparator

· CEQ: Course Evaluation Questionnaire Universities selected by the Faculty for comparison.

· GDS: Graduate Destination Survey

· SFS: Student Feedback about Subjects (from late 2000: incorporating subject feedback and CEQ style items)

APPENDIX C: Checklist of Documentation to be Assembled & Submitted

This tabulated checklist will provide the template/contents list of the Documentation Folder to be assembled at least four weeks prior to the review.

Documentation	Section Responsible
University Mission Statement	Chancellery
Course Proposal	Faculty
Course Handbook Entries	Student Affairs
Background paper on course under review	Faculty Executive Dean to arrange
Course Information for Students	School
*Completed and Assessed Student Work, (maximum 50 samples)**	School
Grade Distribution for Key Subjects (last 3 years)	Resources Office (Statistics)
Statistical Profile of Course (last 5 years): Completion, Pass and Retention rates Course Evaluation Questionnaire Graduation Destination Survey	Resources Office (Statistics)
SFS results: Item summaries of key subjects (3 years) Course summaries by item (3 years)	Academic Support Division
Staff Profile – level, qualifications, subjects taught, personal web page address	School
Focus Group Participant Profiles	School
Written Submissions by staff and students (including confidential submissions)	Chair, Academic Board
Other Reviews of Course	School

*where student work is not amenable for inclusion in a folder, an exhibition should be arranged for the Review Committee

Academic Board Course Reviews

Checklist of Material to be submitted by Heads of School

Part 1: Submission Date:

1	A document outlining the background and history of the course, together with a summary of the main issues and details of any proposed changes to the course not yet reflected in the official documentation.
2	Copy of most recent external accreditation submission and related papers, if course externally accredited
3	Faculty Strategic Plan
4	School Strategic Plan
5	Copies of any other School documentation, eg policies, guidelines, practices, procedures
6	Subject Outlines for all Subjects in the course
7	Guides to the course including compulsory units, electives, majors
8	Sample minutes of staff/student consultative meetings
9	Sample minutes of external liaison meetings
10	Curriculum Vitae for all Academic Staff who teach the course with details of academic qualifications, refereed publications and grants attracted
11	A summary table of Staff workload distribution
12	Any graduate employment details collected independently of Graduate Destination Survey
13	Student work samples - representative range of assessments, (eg essays, examinations and tests), across different types and levels of subjects and from different teaching staff . This should number a maximum of about 50 samples in total, and include multiple examples to show HD/D/C/P/F.

Part 2: Submission Date:

14	List of staff who will attend scheduled meetings with Review Committee
15	List of students who will participate in scheduled meetings with Review Committee with details of course, year
16	Research Papers if applicable – up to about 10to show different grades

[1] Refers only to coursework post graduate courses

Return to Contents

Homepage

Issued by Professor Robin McTaggart,

Pro-Vice-Chancellor (Staff Development and Student Affairs)