Will digital assessment make your life as a lecturer easier?

Within the course 'strategic HR Analytics we conducted a pilot on digital summative assessment. The lecturers were looking for an innovative way to make the assessment process more efficient. The main reason was to improve the grading process by making it more reliable and less dependent on illegible student handwriting.

The pilot group consisted of 20 master students in an exam, and a resit with essay questions.  The solution provided was a combination of the assessment function in BlackBoard combined with Respondus Lockdown Browser.

Organization

For the exam and resit, the students were allowed to bring their own device (BYOD). Students needed to download an external program called 'Respondus Lockdown Browser' to ensure secure access to the test environment in BlackBoard.  

Since the students were already enrolled in the BlackBoard course, they were acquainted with this system.

The following precautionary measures were taken:

To let the students feel comfortable with the systems, a trial session was scheduled during the lectures.

For any technical issues which couldn’t be solved during the exam, spare laptops were provided.

Students who were unable to bring an appropriate device could also use a spare laptop.

If all else failed, a paper-based version of the exam was available.

The technical and organizational lay-out of the lecture hall was inspected and prepared before the session.

In collaboration with functional application management, the test settings in the systems were re-checked.

Overall  experience:

Trial session:

During the trial session no real problems occurred.  Almost all students were able to access the systems.

Examination:

During the actual exam, students working with iOS experienced some difficulties with the screen size and ability to scroll down with the mouse.   Because of this issue 5 students were unable to finish the exam on their own device. One of the students experienced technical problems with 'Respondus' and wasn't able to finish the exam on a laptop.

Resit:

None of the he above issues occurred in the resit.  All students were able to finish the exam.

 

Experiences

Experience of the lecturers:

The lecturers were satisfied with the process of this pilot.  Digital assessment helps to make the assessment more objective.  They recommend to use this solution in small student groups and involve support staff in the process.

 Experience of the students:

See Figure: Evaluation results of students

Experience of support staff:

Setting up this pilot was an organizational challenge; the initial time investment was high.  Overall we are satisfied with the results.

Points to consider

  • Always check the grading settings upfront
  • Make sure that the trial session is long enough so that students experience all aspects (and issues) of the exam
  • Ask support services to re-check the settings of the exam
  • A lot of organizational challenges arise with the use of spare laptops. Consider using only a paper-based backup
  • Always check the grading settings upfront
  • Make sure that the trial session is long enough so that students experience all aspects (and issues) of the exam
  • Ask support services to re-check the settings of the exam