What are CEQ, SETU and AUSSE and why are they important?

The Course Experience Questionnaire

Work by Ramsden and Entwistle in Britain in the early 1980s with a Course Perception Questionnaire established a link between students' perception of their learning environment and their quality of learning (Ramsden & Entwistle, 1981). Subsequent work in Australia during the 1980s on a Course Experience Questionnaire (CEQ), commencing with an initial 80 item inventory that was consolidated via trials to a 30 item inventory, lead to a 1990 national survey of students which confirmed the reliability and validity of the 30 item inventory (CEQ30) (Ramsden, 1991). A shortened (23 item - CEQ23) version of the CEQ (including the addition of a 'Generic Skills' scale) was developed in consultation with the then Department of Employment, Education and Training. Work that confirms the value of the CEQ23 instrument has also been done (Byrne & Flood, 2003; Wilson, Lissio & Ramsden, 1997). A version of this instrument has been included in the Graduate Careers Council of Australia (GCCA) national survey of graduates from 1993 onward. Clearly, there is a need to be clear about which version of the CEQ is being referred to.

One of the criteria for the initial development of the CEQ was that it be generally applicable to all students, hence discipline-specific questions (for example questions about lab work) were not included (Ramsden, 1991). Since its initial development and use in the GCCA national student survey, the number of CEQ-related items has increased to 49 to cater for discipline-specific course aspects, though individual institutions are only required to report results for 13 'core' items:

GT01 - The staff put a lot of time into commenting on my work.
GT03 - The teaching staff normally gave me helpful feedback on how I was doing
GT10 - The teaching staff of this course motivated me to do my best work.
GT15 - My lecturers are extremely good at explaining things.
GT16 - The teaching staff worked hard to make their subjects interesting.
GT27 - The staff made a real effort to understand difficulties I might be having.
GS06 - The course helped me develop my ability to work as a team member
GS14 - The course sharpened my analytic skills
GS23 - The course developed my problem solving skills
GS32 - The course improved my skills in written communication
GS42 - As a result of my course, I feel confident about tackling unfamiliar problems
GS43 - The course helped me to develop the ability to plan my own work
OSI49 - Overall, I was satisfied with the quality of this course

For all CEQ items, respondents are asked to express their degree of agreement or disagreement on a five-point scale. On the national standard form only the 'strongly disagree' and 'strongly agree' points are labelled, however the instruments used at some institutions label all five points. The five-point response categories are generally interpreted as 'strongly disagree','disagree', 'undecided', 'agree' and 'strongly agree'.

The theoretical construction and the practical application of the CEQ are not without their critics. Some argue that the focus of the CEQ is too narrow as measure of the entirety of the student experience. Since its original development as a proxy measure of quality of student learning, the CEQ has been used for a range of purposes, some very different than for what it was intended, i.e. for determining institutional funding and use by third parties to construct league tables (Niland, 1999). The originally validated CEQ30 was reduced to the CEQ23, of which only 12 items are retained in the current 13 item core of the GCCA CEQ instrument. Some of the optional CEQ items relate to resource-dependent aspects of the university experience, potentially advantaging well-resourced institutions. There is some evidence that aspects of the CEQ may not be well suited to 'unconventional' teaching and learning environments, such as problem-based learning (Lyon & Hendry, 2002). Nevertheless, the CEQ (in particular the GCCA version) remains a widely used measure of student quality of learning.

Another reason that the CEQ has historically been important in higher education was its relationship to the former Learning and Teaching Performance Fund (LTPF). The LTPF provided performance-based funding that was determined by a combination of performance indicators, including the core 'student satisfaction' indicators from the CEQ, 'outcome indicators' from the Graduate Destination Survey (GDS), and 'success indicators' from national student statistical information. It is likely that the CEQ will continue to play a role in performance-based funding in higher education, and hence remain an important concern for universities.

The developer of the CEQ suggests that the use of mean CEQ scores to rank organisational units is problematic, as they are normative data (the highest ranked unit may still be unsatisfactory). It is more useful to consider the proportions of students agreeing with scale items. It is also valuable to consider the changes in results over time. The validity of all inferences from respondent data depends on how representative the sample is (Ramsden, 1991). It is also noted that systemic differences have been observed in CEQ ratings based on size of institution, field of study, age, gender and other demographic characteristics, and interpretation of CEQ results needs to be done with knowledge of local conditions (Graduate Careers Australia, 2006).

In addition to the 'quantitative' response items noted above, the CEQ instrument employed by the GCCA also includes an invitation to respondents to write open-ended comments on the best aspects (BA) of their university course experience and those most needing improvement (NI). These responses provide additional information that can help in understanding what students had in mind when agreeing or disagreeing with the CEQ response items. As noted above, a large analysis of open-ended comments made by university graduates on their studies as part of the course experience questionnaire (CEQ) has recently been completed (Scott, 2006). More than 160,000 comments from students graduating from 14 Australian universities (including Deakin) over the period 2001-2004 were analysed to identify common themes that were reported by students. Key findings include:

• the total university experience counts - not just what happens in the classroom;
• students desire learning methods that engage them;
• the preferred learning methods varied by discipline;
• key areas needing improvement are assessment, student administration and support, and course structure and expectations;
• computers and information technology don't figure highly in student ratings; and
• staff make a principal difference in almost all aspects of the course.

Activity

In addition to the 13 core CEQ items listed above, are there any other aspects of the student learning environment that you think have an important influence on student learning outcomes?  What are they?

Forward to: Student Evaluation of Teaching and Units