Curricula have changed significantly, and specialized software is part of many courses (e.g. programming, math). A project is not suitable for assessing all technical skills. Students should be able to carry out test assignments independently and under time constraints. A project does not offer those restrictions. Until now, the University of Twente did not have an option to use specialized software during exams. The University of Twente does not have dedicated digital assessment facilities.
However, students do carry their own laptops where this software is installed. But student laptops also contain programmes that should not be available during an exam: e-mail, Dropbox, and other communication tools.
Pilots were conducted with a USB assessment system in the 2015/2016 course.
Digital assessment:A digital test can be more effective as a solution for the situation described above. The department of Advanced Technology (AT) and Applied Physics (TN) initially developed a solution (further developed by UNTESO) which allows the student laptops to run under a controlled operating environment by means of a USB stick.
Bring Your Own Device (BYOD):These laptops were started with a distributed boot, which limited their functionality. Only a few programmes were available, which were necessary for the test, such as MatLab and LibreOffice Writer. The Internet connection was controlled, making it possible to deliver the results via the Internet, while making all other channels of communication (local and remote) inaccessible.
Pilots were conducted with the system in the 2015/2016 course.
1.Over 100 students were safely assessed with this method.
2.The system is technically stable (apart from Matlab-related issues, which were mainly solved in the last session with updated OpenGL drivers).
3.Lecturers are very satisfied with the solution.
4.We recommend using one dedicated location for digital testing with sufficient power connections (Therm).
5.The USB sticks’ bar codes are scanned. Logistics around this worked fine.
6.Booting the USB stick did not cause trouble with students, thanks to a dedicated instruction and test session.
7.The exam submission portal works well.
During the pilot conducted in 2016, we had the following points of concern and lessons learned.
Points of concern:
1.It takes time for students to get used to the Linux GUI. For example copy / pasting materials is different.
2.It is important to only give students with technical issues a spare laptop. When given the choice, students preferred to use a spare laptop. This resulted in a large number of spare laptops needed (30%).
3.The system does not work on MacBooks.
4.Spare laptops should be available when implemented campus wide.
5.Logistics around spare laptops definitely need to be resolved.
6.General organization and communication should not be underestimated.
We will take action on the following points:
•A policy with clear instructions for invigilators needs further attention.
•Instructions for configuration and start up need to be provided at an early stage.
•Clear information on time schedule.
•Set up a test protocol.
•Test documents need to be made available on a server.