University of Oxford

For this case study, Inspera interviewed the following people at the University of Oxford in 2022:

Jenny Roberts – Head of Examinations and Assessments
Samantha Hutton – Submissions and Research Degrees Manager
Charlotte Rowland – Online Examinations Manager
Naomi Hood – Senior Project Manager

Research and Project Planning

In May 2020, the University established the E-Assessment Project with funding to oversee the plan for the next 12 to 18 months. The Project Board was made up of senior academics and administrative leads who were able to represent different interests within the University.

The Project’s first task was to identify a preferred software solution to support exams and submissions. Although the University of Oxford already had a relationship with Inspera from the 2018 pilot, the E-Assessment Project team researched different platforms before making a decision. However, after consulting with exam software providers, they felt that many of them catered primarily for end-point assessments rather than their use case. Furthermore, they felt that other coursework submission solutions, including VLEs, didn’t offer the same anonymity features as Inspera.

The E-Assessment project team consulted with Inspera to ensure that they could meet their use-case requirements, both for exams and coursework. Although the exams and coursework teams had worked separately until this point, whilst ensuring some crossover, they now began to work more closely when the decision was made to work with Inspera.

During this period, consultations were vital because the University of Oxford is a devolved institution. Each department and exam board within the University needed to be able to assess their students in the way that worked best for them.
The E-Assessment Project brought in change managers and communications experts as well as hosting roadshows and briefing sessions to keep each department up-to-date on the project. Because of the structure of the University, the team delivered regular communications cascading information via the departments.

Assessment Structure and Requirements

Exams

The standard exam format that The University of Oxford used in Inspera was a timed, invigilated, essay-based exam. Some changes to this model were required to accommodate the unusual circumstances caused by the pandemic. The majority of assessments had to be carried out virtually, with students working remotely at home, which could have been anywhere in the world. When planning modes of completion, the exams team considered the different ways in which students may need to produce their answers: typed directly into Inspera only, typed directly into Inspera with supplementary uploads (e.g. PDF files of handwritten graphs or diagrams) or upload of files (eg scans of handwritten answers) only. If the students needed to upload part of their exam, they were given an extra half an hour of technical time. Although the majority of students were expected to start their exam at a set time (typically 9:30AM), the exams were open for a 25 hour period so that students in different parts of the world could work during daytime hours. Upon commencement of the exam, students were then subject to the appropriate time limit relevant to the exam they were sitting and whether any reasonable adjustment needed to be made for individual students.

Other departments, including the English department, tried a different approach. In this case, they used a ‘working day model’ in which students had eight hours to complete their exams. During this time, students were encouraged to pace themselves and take time to eat and rest so that, in total, they worked for about three hours. This format was designed to be more inclusive for everyone because there wasn’t any need for extra time for disabilities or medical conditions.

Using the lessons learned from the online delivery during the Covid pandemic, the E-Assessment Project team explored alternative methods of online exams for the 21/22 and 22/23 academic years. As an institution, University of Oxford moved away from the 25-hour exam window in favour of more set start times with flexibility provided on a case-by-case basis relative to the needs of the programme or student cohort. Also, within 21/22 more open-book exams took place with an 8-hour, 48-hour and in a small number of cases 72-hour duration.

Coursework Submissions

At the time of implementation, coursework was more static than exams: PDF upload was the only format used and the coursework team was not planning to change the process. During the Pandemic, they moved away from hard-copy submissions which was initially an adjustment for students, academics and administrators, but has now become the accepted norm.

Jenny Roberts said “Inspera is a secure method for sharing the off-system completed file,” insomuch as students complete their work outside of Inspera and only use the platform for submissions, therefore not spending much time on the platform. However, it was important that Inspera made it easy for students to verify which submission they were uploading, to upload the file itself and submit it, and to review their submission. For administrators, ease of use was also important so that it was simple to review the date of the submission and whether it was before or after the deadline, before sharing it with examiners in an efficient way.

A significant priority for coursework submissions was the ability to anonymise submissions and facilitate the Oxford approach to late submissions. It is University policy that all assessments must remain open for a minimum of 14 days after the original deadline. Students can apply for extensions either in advance or retrospectively which is then granted or not granted through the formal decision-making channels. Students may be given an extension of more than 14 days and so the platform had to be flexible to allow this.

Communication and Support

In the first term, the exams team was very focused on student support. They set up an online helpdesk for students as well as a phone line that students could use to get urgent help. Students were also encouraged to do a practice exam so that they could make sure that their device was set up properly and resolve any login issues prior to the real exam.

For staff, there was information on the website as well as large roadshows, attended by several hundred people, before the exam period to share advice and updates. In addition, they held twice weekly drop-in sessions which were helpful for people who needed individual support because they could find out how Inspera could adapt to their own needs. It was also important to ensure that policies were updated to reflect the new processes. For example, the exams team had to consider whether or not submissions would be accepted after the cut-off.

In case of technical problems during exams, the exams team used an online query management system to handle issues. They also used the messaging feature to get live key messages out to students sitting their exams.

When it came to coursework, it wasn’t feasible to do in-person training across departments because of timing but also because each department had a very different approach to coursework. Some departments used group submissions while others used individual submission options and each had its own timings throughout the year. Therefore there was no ‘one-size-fits-all’ training. Instead, the submissions team elected to use a self-help tool called Whatfix which provided in-app guidance. They worked with Inspera to incorporate the tool and provide an instantly available guide to the platform. This meant that staff could get support on-demand, in the areas where they needed it most.

However, there were in-person options available because they found that staff had varying preferences for support and training. In 2021, there were drop-in sessions four times a week in which anyone could ask about anything. In the following year they were providing pre-bookable support sessions which helped to manage the submission team’s capacity. During the initial implementation, the submissions team was supported by additional project management resources including an Implementation Coordinator who was appointed to answer queries. Over time, the project team and implementation coordinator have scaled back and the business-as-usual submissions team has been able to handle support themselves.

Methods for Collecting Feedback

The exams team took the Summer of 2021 to reflect on the first term. They spoke to both staff and students and gathered quantitative data from surveys as well as qualitative data from focus groups. This feedback was taken to a working group of administrators from across the University. They reviewed the information and decided on ways to improve exams for the future including putting together a case for more funding to allow for enhancements and considering whether or not the exam strategy should be changed.