New publication: A Process for Improving Course Quality based on Mid-semester Feedback

In the article published in the first International Conference on Technology enhanced learning, reforming education and quality of teaching with title "A process for Improving Course Quality based on Mid-semester Feedback" we present our experience with a new formalism to gather very valuable information on how students perceive a course such that corrective measures (if needed) can be taken without waiting for the following edition

Last fall we deployed a totally new C programming course. Everything, objectives, methodology, labs, etc. was conceived from scratch. The reason for this process was to try to design a course with a set of objectives derived from the program objectives that included aspects such as "teamwork", "use development tools used in corporate environments" and "promote self-learning". Compared with other similar courses, the amount of aspects that changed made the endeavor very challenging.

We could not afford to wait until the next edition to make adjustments on the course structure. Besides, the type of feedback obtained from the teacher evaluation provided by the University is not detailed enough as to obtain a clear picture of what is happening in the course. We decided to use an extremely simple, yet very effective tool to obtain quick feedback that clearly pointed to the aspects to submit for consideration to the teaching staff.

After several valuable conversations with professor Miguel Valero, a questionnaire was deployed with two very simple questions: "Describe the most crucial positive/negative aspect you´ve seen in the course so far". The responses were obtained through a conventional HTML form with a mechanism to limit the text to only 300 characters.

Although simple, the scheme proved to be very useful. By asking only about the most crucial positive/negative aspects, students are forced to choose. It is not a lengthy set of questions with multiple choices. Students are asked to think first, then answer. By restricting the size of the answer, students are also forced to be succinct, which means much more readable answers and a faster post processing. By asking about the two aspects (positive and negative), some reflection is needed to obtain the two extremes.

But with a course with 250 students, deploying the questionnaire and obtaining the answers was the easy step (altough only 25% of the students answered). The tricky part came when analyzing the answers. It was in this step where the scheme really worked. Each answer was reviewed, but due to its size, they were quickly categorized in a set of 20 topics. For each topic, the number of times such topic appeared as the most positive and the most negative were counted. These topics where then categorized again based on the comparison of these two numbers.

Topics with a much higher number of positive appearances where clearly those most liked by the students. Those with a higher number of negative appearance were those that needed to be re-evaluated by the teaching staff to see if there any measure was required. Finally, those topics with an almost equal number of positive and negative appearances were declared "controversial" and submitted also for re-evaluation.

The results were impressive. The course had an important problem of work overload. The categorization of the obtained answers quickly showed this as the most negative aspect. Next, the evaluation scheme (continuous evaluation) and the class methodology (requiring an active student participation) came second and third but with a significant number of positive appearances. Measures were taken to redesign the workload and emphasize on the need of daily work to adapt to continuous evaluation and class participation.

The questionnaire was deployed three times during the semester. The evolution of the obtained results was also very enlightening. After the first measures to balance the workload were deployed, the following results showed a clear improvement on the student satisfaction. The same was true for the evaluation scheme and class participation. Besides, the second and third surveys showed that students were very pleased with collaborative scheme to work in teams that was deployed on the second half of the semester.

I guess the conclusion is too obvious. If you want to truly know what is going on in your course, you just need to ask. However, when asking to a large number of students, you want the answers to be brief and to the point so that your turn around time is brief.