In 2017/18 academic year the University launched a project that aims to make online summative assessment possible for a wide range of assessment types and in non-cluster environments, which is one of the objectives in the TEL Roadmap.

The project focuses on two key areas:

• Expanding the types of online exams that we can deliver here at Newcastle
• Exploring the possibility of students using their own laptops to take secure online exams

The first pilot exams

Online exam software called WISEflow was used for 3 pilot exams in semester 2 2017/18.  All the students who took a pilot exam were given a briefing about how their exam would work, and had the opportunity to take a practice test so that they could try out the software before their exam.

Typed written/essay exams

Two online exams that included a typed essay part took place successfully in semester 2 assessment period.  The exams were taken by Stage 1 Biology and Stage 2 Psychology students, and all took place in university computer clusters.

The exam software has word processing functionality that allows the student to

• use text formatting options including numbering, bullet points, and showing emphasis with italics, font size, type and colour
• edit and reorder their work as they write
• insert headings and generate an automated contents page, which they can use to navigate between sections of a long piece of writing
• draw basic diagrams
• see earlier versions of their work, and revert back to an earlier version if needed

The academic setting the exam can also choose whether to allow their students to use spellcheck during the exam.

When they had finished their pilot exam all students were invited to complete an evaluation survey.  This chart shows how students rated their experience of typing an essay exam:

Bring Your Own Device

The first Bring Your Own Device pilot was a formative multiple choice test for Stage 1 Biomedical Sciences students, which took place during the semester.  Students had the opportunity to opt in to taking the exam on their own laptops at an invigilated venue, instead of in a computer cluster.

Before the day of the exam students who opted to take it on their laptop downloaded and installed the FLOWlock Browser that they would use to take the exam.  The software locks down a student’s laptop in the same way that university computers are locked down for an OLAF exam, and means that they can’t access anything else on their machine or online during the exam.  The students took a practice test on their laptops, so that they could confirm ahead of the exam that everything was working correctly.

Everyone using their own laptop was asked to bring it to the exam fully charged, and power sockets were available in case anyone needed them.  The exam was 45 minutes long, and no one needed to make use of the power sockets.  A bank of University laptops were also available in case anyone had problems with their laptop, and 2 students made use of these.

This chart shows the evaluation survey responses from students who opted to take the exam on their own laptops.

National context

The Heads of eLearning Forum (HeLF) recently published the key findings from its 2018 survey on Electronic Management of Assessment (EMA) Digital Exams in UK Higher Education (HE).

The survey found that in the majority of universities (61%) digital exams are replacing traditional exams in only 1 or 2 modules.  Here at Newcastle the Online Assessment and Feedback (OLAF) Service supported 162 online exams in 2017/18 academic year.  Alongside this Numbas and Optical Mark Reader (OMR) systems are used to deliver specialist digital exams in certain subject areas, and the diversification of online exam provision is set to see the number and range of online exams we deliver increase further in the coming years.

The HeLF survey findings about barriers to online exam adoption make interesting reading, with 44% of respondents citing space as the biggest barrier, a factor that we need to consider here.  45% of those introducing online exams are using or considering using laptops, but only 13% are focusing solely on Bring Your Own Device.  A larger proportion, 28%, are focusing equally on delivering online exams using university-provided hardware and students own laptops.

Did you know that the OLAF Service launched as an initial pilot in 2007/08, and transitioned to a centrally supported service in 2010/11?  This  means that 2018/19 will be OLAF’s 12th academic year.  The HeLF survey found that only 14% of UK HEIs that responded have been undertaking online exams for 10 or more years, so that’s definitely something to celebrate!

What next for online exams at Newcastle University?

Following the success of the initial pilot exams, the project expanding and diversifying online exam provision will continue for a further 4 years, with more pilot exams using WISEflow planned for 2018/19.  The next phase of pilots will aim to include:

• typed written answer/essay exams from a wider range of disciplines, and which students taken on their own laptops
• Bring Your Own Device exams using an ‘opt out’ rather than ‘opt in’ approach, where students are expected to use their own laptop  unless they submit a request to use a university machine
• new system marked question types offered in WISEflow that are not currently available in Blackboard for OLAF exams
• testing and potential pilots of ‘whitelisting’ links, which could allow students to use an online resource or application during a secure locked down exam.

If you are interested in participating in the pilot please get in touch with LTDS@ncl.ac.uk