Evaluating SmartEvals
by Bryanne McDonough | published Dec. 19th, 2017
At the end of every semester, students are hounded with reminders to fill out end-of-semester evaluations. Those who have not completed their evaluations through the SmartEvals system receive an email every 72 hours until evaluations close, just before finals begin.
"These ratings are used by faculty to improve their teaching and by administrators to evaluate the teaching effectiveness component of each faculty member's yearly appraisal," one email said. The others contained similar information.
Various associated deans and the department chair of the School of Film and Animation confirmed that these evaluations are taken seriously. Some students are less optimistic that their concerns are being heard, however. Reporter covered the case of a SOFA professor, Peter Kiwitt, who had been arrested for possession of child pornography. Savanah Bayse, a former student of his, said that complaints against his alleged inappropriate behavior had been made through student evaluations.
“Basically all of the women that I know that I talked about it to as well as other students were very direct and open [in their teacher evaluations] about how uncomfortable Peter made us and that we didn’t think he fostered a good teaching environment, as well as him discriminating against women of color,” Bayse said.
To address these concerns, Reporter investigated what happens after an evaluation is submitted.
Implementing Evaluations
“We really value student feedback,” Matthew Marshall, KGCOE associate dean for Undergraduate Programs, said.
Marshall, who is also a professor, oversees the different KGCOE programs and their improvement. Each of the nine RIT colleges has an associate dean that fills a similar role, which involves overseeing the evaluations.
RIT has had student evaluations for many years, but originally each college had their own system and it was done with Scantrons on pencil and paper. For the fall of 2013, the Academic Senate voted to move to an electronic system called SmartEvals in which there would be a standardized set of questions to assess teachers.
Jeremy Haefner, provost and senior vice president for Academic Affairs, wrote a letter announcing the change after three years of consideration by the Academic Senate.
“RIT prides itself on placing teaching and learning at the forefront of what we do,” the letter stated. “Collecting student feedback, as well as information from other assessment methods, is an important part of our continual improvement efforts.”
Students are asked seven standard questions about the professor for the department. Student Government also includes questions about the professor and the course in their OpenEvals system that is available for students to see.
There are also two comment sections where students can write what they believe the instructor did well and what they need to improve on.
“I find [comments] much more meaningful — and so do the faculty — than the scores,” Malcolm Spaull, department chair of SOFA, said.
Each college can add specific questions to evaluations for their courses and instructors — for example, CAST asks about textbooks and GCCIS inquires about facilities like computer labs.
The Saunders College of Business asks about course rigor, whereas timeliness, consistency and fairness of grades and the availability of out-of-class help are the questions the College of Science asks. NTID includes questions about ease of communication, teaching skill and whether the student feels as though they learned a lot.
Marianne Gustafson, an associate dean of NTID, stressed that communication was of the utmost importance in NTID courses because of the varied languages and modes of communication used by the Deaf community.
NTID also uses the SmartEvals system to conduct mid-semester evaluations.
“Students want to have a voice during the semester too,” Gustafson explained.
NTID has had traditionally higher response rates than other colleges for the end-of-semester evaluations, at 76 percent for Spring 2017, according to Gustafson. Their mid-semester evaluations have response rates in the 40s. Gustafson credits their high response rates on the emphasis each instructor places on filling them out.
The associate deans have access to the results of the evaluations, but each college has multiple schools with many classes, so they can’t review each teacher’s evaluations thoroughly. Instead, the department chairs of each school use the student evaluations as part of each faculty member’s annual evaluations.
A few methods are used to prevent a professor from retaliating against students who submit negative reviews. Responses remain anonymous and aren't released until after final grades are published. The evaluations are only released to the campus body if more than 65 percent of students fill them out.
It's possible that the negative evaluations about Kiwitt, the SOFA professor, were never seen because of that cutoff. It's also possible that they just slipped through the cracks. Some departments have many classes and many evaluations along with those. In SOFA, professors submit their self-evaluation of the student reviews; Spaull admits that sometimes faculty will only present what paints them in a good light, however he will only look at the evaluation results to substantiate what he hears from students.
“There’s no way that I read every evaluation every semester, I look to them as a database to refer to if need be,” Spaull said.
The evaluations are not treated equally across the entire university. Some of the associate deans glanced through the comments in a random manner, but expected the department heads to review them thoroughly at the time of annual evaluations.
Bad Reviews
The department chairs take bad reviews seriously. The comments and scores are used to set improvement goals.
“The goal is not to increase the score,” said Mike Yacci, associate dean of GCCIS. “The goal is to do a better job teaching and then the score will increase.”
“The goal is not to increase the score. The goal is to do a better job teaching and then the score will increase.”
Faculty are evaluated on more than just teaching. John Tu, associate dean of Saunders, gives professors opportunities to attend teaching workshops and has the department chairs look for improvement areas. Scholarship and service are also factors, although less important than teaching.
“Teaching is the most important thing at RIT,” Tu said. “We cannot tolerate a faculty member that is consistently not performing well.”
Department chairs do not put weight on one semester of reviews. If just one semester is showing bad results, the department chairs try to identify what went wrong and what could be improved. Over time, department chairs hope to see a pattern of improvement.
“We expect [faculty] to address the problems in their next appraisal,” Gustafson said.
Marshall, of KGCOE, stressed that constructive feedback in the comments was most helpful for professors when they try to improve.
“[Department heads] are looking for useful, constructive comments from students,” Marshall said.
The evaluations were originally designed for evaluation of faculty members by the school, but students wanted to see the evaluations as well. In September 2014, a PawPrints petition titled "Make Course Evaluation Results Public" garnered 214 signatures, an impressive feat in the early days of PawPrints.
OpenEvals: Student Access
In November 2015, then Student Government President Nick Giordano announced the addition of six student-oriented questions to the evaluations in a response to a Pawprints petition. The results of these questions would be made available to students through the OpenEvals website.
“The students take it in a very adult, mature way,” Spaull said. “I have a different impression from what little I know of Rate My Professors.”
OpenEvals is designed to avoid bias. Results are not released if the response rate is under 65 percent, if it is one of the professor's first two semesters at RIT or if it is the professor’s first time teaching the course. This is to allow a professor to acclimate to RIT and to a new course.
Yacci said that students get a better idea of a professor just from word of mouth. OpenEvals does benefit from having the opinion of at least 65 percent of students, whereas word of mouth may come from just a few vocal students.
Overall, student evaluations are an efficient way of collecting a large amount of responses to give an overview of a professor's teaching ability. However, they do not paint the full picture. If a student has a comment or concern, they should bring it directly to the department heads to guarantee it is heard. On the SmartEvals information webpage, students are advised that evaluations are not the place to submit reports of gender-based harassment or discrimination. This text was added after August 31, 2017, according to the Internet Archive. The Reporter article about Kiwitt was published in October 2017.
Gender-based harassment and discrimination complaints can be placed here anonymously and identified complaints (which are easier to act upon) can be placed here.