111.3 Universal Student Ratings of Instruction
In recognition of the University's commitment to teaching, the General Faculties Council endorses a system of Universal Student Ratings of Instruction. This system, however, is only one part of the multi-faceted approach described in Section 111.2. (GFC 09 JUN 1995) (GFC 24 NOV 1997) (EXEC 29 MAR 1999)
The Universal Student Ratings of Instruction are administered electronically via a system known as the eUSRI system. (GFC 22 SEP 2014)
The Universal Student Ratings of Instruction are designed to provide a minimal university-wide base of information on student ratings to the parties listed in this Section. With this purpose in mind, the General Faculties Council adopts the following policies: (GFC 24 NOV 1997)
A. All Faculties will ensure that evaluation of all instructors and courses will take place each time a course is offered. The term 'instructors' is meant to include tenured professors, tenure-track professors, sessional instructors, clinical instructors, field supervisors and graduate teaching assistants with responsibilities for courses. The term 'course' is meant to include undergraduate and graduate courses, laboratory courses, non-degree courses, seminars, clinical supervision courses, and reading or directed study courses. With the exceptions noted in Section 111.3.B, the assessment will include the Universal Student Ratings of Instruction as set out below.
B. The Universal Student Ratings of Instruction will be modified in the following circumstances:
i. courses with between four and nine registered students will use a department or Faculty developed questionnaire, which may be administered via the eUSRI system, with non-scored questions, such as:
a) comments on the quality of this course;
b) suggestions for improving this course;
c) comments on the quality of instruction in this course;
d) suggestions for improving the instruction in this course. (EXEC 29 MAR 1999) (GFC 22 SEP 2014)
ii. courses with multiple instructors will use a modified Universal Student Ratings of Instruction questionnaire that will include one set of course-related questions for the entire course and one set of instructor-related questions for each instructor who has taught the equivalent of twenty percent or more of the course. If no instructor is responsible for at least twenty percent of the course, only course-related questions should be used on the questionnaire. (EXEC 29 MAR 1999)
iii. in courses with fewer than four registered students or courses such as alternate delivery style courses, the Chair, Director or Dean will arrange for an alternate method of obtaining student feedback. Such methods could include student course or program exit interviews with the Chair, Director or Dean; or a department or Faculty developed questionnaire, which may be administered via the eUSRI system, with non-scored questions as described in point i. above. (EXEC 29 MAR 1999) (GFC 22 SEP 2014)
C. The Universal Student Ratings of Instruction will take the form of a questionnaire. The following statement of purpose will be included at the beginning of the questionnaire:
The University of Alberta would appreciate your careful completion of this questionnaire. The results help instructors and departments or faculties to initiate constructive change in curriculum and instruction. In addition, the results are one important factor in decisions affecting the career of your instructor. The numerical summaries for the ten questions listed below are available through the Students' Union and the Graduate Students' Association.
The eUSRI system will be accessible only by CCID and students' anonymity will be protected. Students who are concerned about the anonymity of their responses should submit their typewritten comments within the period for which eUSRI is available to the Chair, Director or Dean , making sure to note the course number, section and name of the instructor. (GFC 24 NOV 1997) (GFC 22 SEP 2014)
Questions about this questionnaire should be addressed to your Chair, Director or Dean.
D. The anonymity of student responses to the Universal Student Ratings of Instruction is of fundamental importance in maintaining student confidentiality and encouraging the free expression of views. Under normal circumstances, the anonymity of students will be protected. Universal Student Ratings of Instruction offer an avenue of feedback, including feedback critical of instructors. It is understood that it is a normal feature of criticism that it may be regarded as offensive and/or unjustified, and that such characteristics would not justify a departure from the normal rules pertaining to confidentiality and anonymity. (GFC 28 FEB 2000)
However, the University has a parallel duty to protect the safety (physical or mental) of members of the University community. If a Department Chair has concerns for the safety of faculty, staff or students, arising from statements that are part of a Universal Student Rating of Instruction, the Chair will consult with the Dean of the Faculty. If the Dean believes that there is a valid concern for safety, he or she may recommend to the Provost and Vice-President (Academic) that the identity of the author of the statements be sought out and disclosed to the appropriate University officials. At any time during this process, the Chair or Dean may invoke the Protocol for Urgent Cases of Disruptive, Threatening or Violent Conduct (Section 91.3, GFC Policy Manual). (GFC 28 FEB 2000)
On receiving such a request from a Dean, the Provost and Vice-President (Academic) will follow the terms of the Protocol for Urgent Cases of Disruptive, Threatening or violent conduct in determining whether there is
i. reasonable cause to believe that the safety or security (including significant psychological harm) of persons may be threatened and
ii. that under existing University policies, the statements are grounds for disciplinary action and hence whether confidentiality of USRI should be breached and the provisions in Section 91.3.2 and/or 91.3.3 of the Protocol invoked. (GFC 28 FEB 2000)
If the identity of the author is disclosed, the Provost and Vice-President (Academic) will notify the author of the statements. The Provost and Vice-President (Academic) will also notify any individuals mentioned in the statements. (GFC 28 FEB 2000)
E. The Universal Student Ratings of Instruction questionnaire will use the rating scale
Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree (EXEC 29 MAR 1999)
to gather responses to the following questions:
1. The goals and objectives of the course were clear.
2. In-class time was used effectively.
3. I am motivated to learn more about these subject areas.
4. I increased my knowledge of the subject areas in this course.
5. Overall the quality of the course content was excellent.
6. The instructor spoke clearly.
7. The instructor was well prepared.
8. The instructor treated the students with respect.
9. The instructor provided constructive feedback throughout this course.
10. Overall, this instructor was excellent. (EXEC 29 MAR 1999)
These constitute the ten required Universal Student Ratings of Instruction questions. Instructors, departments, and faculties are encouraged to supplement the set of universal questions.
The questionnaire will include an opportunity to provide comments. (GFC 22 SEP 2014)
F. Certain policies are necessary in order to ensure that the Universal Student Ratings of Instruction Questionnaire is administered in as consistent a fashion as possible. These are:
i. Access to the electronic Universal Student Ratings of Instruction will normally be available from the day after the withdrawal deadline until the last day of classes. Note that an instructor may choose to allow class time for completion of the questionnaires. In these cases, the instructor will not be present in the room during the time allotted for completion of the questionnaire. Departments or Faculties will create policies to ensure that other individuals (e.g. other instructors, students within the class, teaching assistants) are available to be present in the room during the time allotted for completion of the questionnaire. Also in these cases, online access for completion of the questionnaires will still be available for the period described above. (GFC 22 SEP 2014)
ii. The Chair or delegate will be responsible for transmission of results and comments to the instructor under the conditions set out in Section G. (GFC 22 SEP 2014)
G. The numerical summaries for the ten Universal Student Ratings of Instruction questions will be reported to the instructor, the Chair, Director or Dean and students.
i. the number of students responding in each category;
ii. the median score to one decimal point for the question; and
iii. numerical values from Tukey's boxplot statistics will be provided to describe the distribution of scores in the Faculty/Department:
a. lower cut-off for outlier scores
b. lower hinge (25th percentile)
d. upper hinge (75th percentile)
e. it is expected that the upper cut-off will always be 5.0 and, therefore, unnecessary to report. (EXEC 29 MAR 1999)
Note: Statistics from Tukey's box-and-whisker plot analysis (John W. Tukey, Exploratory Data Analysis, Addison-Wesley Publishing Company, Inc. 1977) have been selected to describe the distribution of USRI data. These statistics are chosen to achieve two main objectives: (i) summarizing skewed data and (ii) identifying outliers from the general population if they exist.
The median (middle of a ranked set of numbers) is generally preferred rather than the mean in defining the centre of a skewed data set.
The 25th and 75th percentiles provide information about the spread of individual scores around the median. By definition, half of the scores in a distribution are below the median and 25 percent of the scores are below the 25th percentile. Since this occurs "by definition", these values should not be used to determine whether a particular score is "good" or "bad".
The lower whisker or cut-off, which is 1.5 box lengths below the 25th percentile (box length is the distance from the 25th to the 75th percentile), defines a reasonable limit beyond which any score can be considered an outlier. Outliers are scores that identify ratings of instruction falling outside the usual distribution of the scores for the population being tabulated.
Given the nature of the USRI data, the upper whisker or cut-off (1.5 box lengths above the 75th percentile) will usually be above 5.0, and so need not be reported.
i. Access to USRI Data: Parties having access to numerical summaries of the ten Universal Student Ratings of Instruction questions and student comments will be the instructor the Chair, Director or Dean of the unit offering the course; members of Tenure Committees; and members of Faculty Evaluation Committees, including the secretary to the FEC. (EXEC 07 NOV 2011) (GFC 22 SEP 2014)
For questions selected by an instructor, only the instructor will receive the results. For questions initiated or mandated by a department or Faculty, the results will be reported to the instructor and the Chair, Director or Dean.
Normally, instructors will receive the results from the student ratings of instruction within twenty working days after the course is complete and the grade sheet has been signed by the Chair, Director or Dean. (EXEC 29 MAR 1999) (EXEC 07 NOV 2011)
ii. Access to Online USRI Data: Online access to the numerical summaries for the ten Universal Student Ratings of Instruction questions scores for all courses will be provided to undergraduate and graduate students. Instructors will have online access to USRI scores for their own courses. Chairs will have online access to USRI scores for instructors in their departments and Deans will have online access to USRI scores for instructors in their Faculties. Deans and Chairs may also request access for a designated assistant. (EXEC 07 NOV 2011)
The results will not be released online for at least ten days following the provision of the results to the instructor. (EXEC 07 NOV 2011)
Access to online USRI data is provided to students only for the purpose of assisting with the selection of courses. Neither the Students' Union nor the Graduate Students' Association will undertake analysis of USRI data available to members of those organizations. (EXEC 07 NOV 2011)
I. All results given out to students, Chairs, Directors and Deans will have the following cautionary preface:
Student questionnaires form an important part of evaluating teaching effectiveness but cannot be taken alone as a complete assessment of an instructor or course. Factors other than an instructor's teaching ability may influence ratings. These factors include class size, class level, Faculty, time of class, required versus optional course, grade expectations, student GPA, gender, race, ethnicity, age of both students and instructors.
Small differences in evaluation should not be considered meaningful. Scores will be interpreted using the rating scale defined in 111.3 (E): 1=Strongly Disagree; 2=Disagree; 3=Neutral; 4=Agree; 5=Strongly Agree. By definition, a score of 4.0 means that students agree that "Overall, the instructor was excellent." (GFC 22 SEP 2014)
J. Nothing in this section will prevent instructors from seeking other means of feedback from students during the term.
K. The central administration of the University will undertake the financing and operation of the eUSRI system in support of the University's commitment to teaching. (GFC 22 SEP 2014)