Project introduction and background information
This project resulted in an overview of how teachers of Industrial Design write feedback and provides insights in its merits and shortcomings. The first step of the project was to analyze and mine a large database of teacher feedbacks for Industrial Design students. This corpus was sampled and analyzed. Important aspects of teacher feedback were identified, and based on relevant literature of feedback (e.g. Hattie and Timperley, 2007 and Shute, 2008) a framework was constituted which can be used to analyze the data in a structured way. This work was published in two scientific publications (one workshop paper, one journal paper).
These insights were used to develop a formative assessment tool to formatively assess the quality of written feedback. Results of these assessments provide a starting point for a feedback tool that enables written feedback conversations between teachers and students. In such conversations, it is possible for teachers to better align their feedback with how it is actually perceived by students and what aspects of their written feedback really contribute to students’ learning. On the other hand, students learn how to request feedback in a more focused way providing relevant input to the discussion from the start. This instrument can also be used as a tool for systematic quality assurance.
Also with the help of an external data science and visual analysis consultant and a screen design team the findings of this analysis were used to develop this tool in close cooperation with the educational staff of Industrial Design. The tool is a web-based application that allows for written feedback conversations between teachers and students with special focus on learning activities and strongly related feedback aspects. The tool was piloted in two learning activities of the Industrial Design department with promising results. A third scientific publication is pending, summarizing the results of the second pilot study and focusing on deriving both feedback quality metrics and summarized feedback content from written conversations facilitated by the tool.
Objective and expected outcomes
The purpose of the project is twofold. Firstly, to gain insight into how teachers of assignments and modules write their feedback. Secondly, based on these findings, to design, implement and re- design a (formative assessment or visual analysis) tool to be used by teachers of assignments and modules and their students to assess the quality of their written feedback and start conversations to about the feedback. In such meetings, it is possible for teachers to discover how their written feedback is actually perceived by students and what aspects of their written feedback really contribute to students’ learning. In the context of Industrial Designs’ educational quality assurance the tool can be used as a means to monitor and improve teachers written feedback on a more systematic and continuous base. This effort is of strategic importance not only for the future of education in Industrial Design, but also to future competence-centered education in the Bachelor College at large.
Understanding how teachers write feedback is an important prerequisite for improving their actual practice of providing written feedback to students. Apart from engineering aspects related to the analysis of a large feedback corpus, and the development of a production-grade web-based feedback tool, enabling Industrial Design teachers to give timely, high-quality feedback to Industrial Design students was a challenge. This was achieved by developing a theoretical feedback framework, validating the framework with feedback samples, then translating the knowledge into a feedback tool that runs reliably since two months and is currently used by several students and teachers for feedback conversations.
- Scientific dissemination
- Dissemination during 3TU CEE launch symposium
- Insights about how teachers actually write feedback will be presented at a 3TU-meeting of the Centre for Engineering Education (CCE).
- Participation in the 3Tu/e accreditation
- Tool fully operational and received well
- Tool development finalized as planned, and with lower costs than estimated
- Participation in the tool pilots was low due to many other obligations and a yet missing operational and structural interface with the education administration of Industrial Design (see points of interest for implementation).
Integration of Rubrics system is advisable
Results and learnings
The tool was received very well (both in pilots and dissemination), however, only with further integration a real impact can be made. The tool allows for integration with other educational systems. However, at the momenet these systems are being revised in the Industrial Design department to better align with the Bachelor College / Graduate School and other external constraints. Therefore a final assessment on how feasible the integration will be needs to be made when the revision process is finalized.