Part of the 4TU.Federation
4TU.
Centre for
Engineering Education

EdTech Project to evaluate educational technology innovations at WUR

Wednesday, 12 May 2021

WUR aims to support teachers who want to experiment with new IT tools in their education and to improve the quality of teaching and learning. However, these educational tools first need to be tested in a pilot study before scaling them up at a WUR-level. This is why a Community for Education Innovation with EdTech was created, a new way for innovating education via Education Technology.

The ā€œEdTech evaluationā€ project, which is part of the Community for Education Innovation with EdTech, seeks to collect, analyze and report data on the effectiveness of the experimented educational tools. The evaluation report is critical to decide on whether to keep funding these tools and implementing it in a larger context at WUR or not. Furthermore, suggestions for improvements of the tools are also provided.

Reliable evaluation

Due to rapid growth of online learning, a huge number of educational tools are developed and offered by companies. It is difficult for educational institutions to decide which one is best to use. By evaluating these tools through a pedagogical and educational perspective by educational technology experts, the evaluation becomes more reliable. It provides the university a better judgment on whether or not to use the educational tools. So far, one tool called ā€œComprovedā€ has been evaluated. Three other educational tools are under evaluation at the moment, including TrainTool, CodeGrade, and Grasple.

Comproved tool

Comproved is an assessment tool which works based on comparative judgments. This tool structures comparisons of studentsā€™ work while aiming to be a valid, reliable and user-friendly instrument. The main users of Comproved are teachers. At WUR, Comproved was used in one of the courses for assessing and grading students. After experimenting with the tool, data was collected in both quantitative and qualitative formats from teachers and students through focus group discussions and surveys and analysed. The overall conclusion of the tool shows that the assessment is more valid (different teachers) and less biased and that the tool is technically easy to use by teachers, but not pleasant emotionally. The teachersā€™ satisfaction rate was 5.8 out of 10. Studentsā€™ satisfaction rate was 6.2 out of 10. Overall, teachers and students were dissatisfied with the tool. They found that the feedback is more general and superficial, it increased the workload of teachers and they had to invest a lot of time in it. The tool is not successful in attaining its goals.

The full results and recommendations of evaluating Comproved can be found here.

More information

To read more about this project please visit Evaluation of Educational Technology Innovations at WUR on the 4TU.CEE Innovation Map. Alternatively you can contact the researchers involved: