MIT Scales Up Homework Reviews

• By Leila Meyer
• 03/30/15

MIT researchers have developed a new system to help instructors review computer science homework assignments in courses with hundreds or thousands of students.

While instructors can easily create automated tests to determine whether students' computer programs produce the correct results, it's challenging for instructors to conduct a detailed analysis of the programming methods the students used when a course has hundreds of students, or in the case of some online courses, thousands of students.

The new system, called OverCode, analyses and compares students' computer code, groups solutions that use the same techniques and creates a program template for each technique. The system then displays the templates side-by-side and highlights any differences between them. Instructors can view a list of student programs that use each template.

The MIT researchers evaluated OverCode in two usability studies and will present the results at the Association for Computing Machinery's Conference on Human Factors i... in April. The studies involved 24 experienced programmers reviewing three different programming assignments from thousands of students using both OverCode and another tool that displays solutions one at a time. "When assessing the simplest of the three assignments, the subjects analyzing raw code performed as well those using OverCode," according to information from MIT. "For the most difficult of the three assignments, however, the OverCode users covered about 45 percent of student responses, while the subjects analyzing raw data covered only about 9 percent."

According to Elena Glassman, an MIT graduate student in computer science and engineering and first author on the paper, the results of the study suggest that OverCode is particularly useful for analysing more complex computer programming assignments. The researchers also said they think the system could help instructors of large online courses provide generalized feedback to more students and use the information to improve both online and on-campus courses.