Transition to the Digital Exams Service: A Timeline

Following our October 2019 post introducing the digital exam service, we have a progress update and some news about what’s happening next.  Centrally supported digital exam provision (including the OLAF Service, and the Diversifying online exam provision project) is being combined into a single service, and we are reviewing our requirements ready to tender for a system that meets our needs. 

February 2020 

Requirement Mapping Workshops will be taking place. The outcomes of these sessions will help to inform the requirements that we will take to system providers.  All academic and professional services staff with an interest in digital exams are invited to contribute.  Please sign up via the link to have your say! 

March 2020 

Tender for digital exam system (30-35 days response time). A set of final requirements will be issued. 

April – May 2020 

Scoring of tender submissions against requirements will take place alongside user testing of software that meets our mandatory requirements.  Look out for updates about how to get involved. 

June  July 2020 

A provider will be awarded the contract to supply a digital exam system to the University.  

Following this, work will be undertaken to move as much of existing digital exam questions and content into the new system as is possible. 

August 2020 

The new system will be vigorously tested and integrated with University systems. User guidance and training for all stakeholder will be developed. 

August assessment period

Any exam deferrals and resits in the August assessment period will need to be completed/submitted in Canvas. The Blackboard license ends on July 31st and from that point no staff or students will be able to access that system.

Schools should adopt the same method of assessment that was used in Semester 2 for any resits/deferrals in the August assessment period. If a Blackboard test was used in the Semester 2 assessment period, then a Canvas quiz should be used in the August assessment period.

If you ran an OLAF exam in Semester 1 you can either deliver the resit using a Canvas quiz or a Turnitin submission.

Information and support is available via the Education Continuity webpages.

September 2020 

Digital Exam Service launches with new software – OLAF is no more. 

All digital exams previously taken in both Blackboard as part of the OLAF service and in WISEflow as part of the Diversifying Online Exam Provision project will be delivered using the chosen software. 

Training will be offered to all academic and professional services staff involved in delivering digital exams, and briefing information will be available for students. 

Introducing the digital exams service

Building on the solid foundations of OLAF provision, and the successful first 2 years of the Diversifying and Expanding Online Exam Provision project, the University’s Technology Enhanced Learning Sub-Committee have approved the launch of a new combined Digital Exams service.

The story so far …

Newcastle University’s Online Assessment and Feedback (OLAF) Service has been running high stakes secure online exams using Blackboard’s test tool since 2007/08. The 13 years since that first exam have seen OLAF come of age, supported by well-established institutional processes that ensured all 132 OLAF exams in 2018/19 went smoothly.

In 2017/18 the Diversifying and Expanding Online Exam Provision project was launched, and the first of some new types of digital exams were piloted using software called WISEflow. Bring Your Own Device was introduced, enabling students to use their own laptops to sit a secure digital exam. Alongside this, moving essay and long written answer exam questions from paper to online has also become possible for the first time.

Continue reading “Introducing the digital exams service”

OLAF Service Capacity

Due to the success of the OLAF service and the capacity to support online exams across LTDS, Exams and Awards and NUIT, the service is currently not available to support new online exams for fewer than 30 students during the assessment periods.  New exams for more than 30 students will go onto a waiting list and be considered based on student numbers, with exams with highest numbers of students being prioritised.

Outside the formal exam period there is no capacity for additional in semester assessments, however The OLAF Service will continue to support all online exams that have previously used the service.

Resit exams  cannot be supported through the OLAF service if there are less than 15 students, the OLAF service will not be available for these exams. The exams could still be run through Blackboard, although this would be without the use of the locked down browser or the University invigilators. Staff can utilise self-help resources well in advance of the resit assessment periods if they wish to run the exam online themselves.

Expanding and diversifying online exam provision

Student working on a computer

In 2017/18 academic year the University launched a project that aims to make online summative assessment possible for a wide range of assessment types and in non-cluster environments, which is one of the objectives in the TEL Roadmap.

The project focuses on two key areas:

  • Expanding the types of online exams that we can deliver here at Newcastle
  • Exploring the possibility of students using their own laptops to take secure online exams

Continue reading “Expanding and diversifying online exam provision”

Student views on feedback forms

Group of students

To find out more a student intern, working with staff in LTDS,  evaluated existing feedback forms and gathered opinions from students to identify what works and what could be improved. The project considered a total of 66 forms from 19 different schools and included focus groups and interviews with individual students.

What did they find?

These are a few key findings and you can find full details in the project report.

Form Design

Have clear, separate sections showing:

  • Strengths and areas for improvement
  • Clear advice for future work

Only use tick boxes for objective areas of the marking criteria, such as grammar. When tick boxes were used for subjective areas, such as argument, students found this unhelpful.

Look at your feedback forms and consider whether these should be redesigned. Consult with the students in your school as part of the process.

Utilising the form

Type feedback, wherever possible.

Introduce structured opportunities to help students understand:

  • expectations of the marking criteria
  • the ways in which this is reflected in the feedback sheet

Discuss how you use marking sheets with your colleagues. Try to develop a consistent approach to:

  • the volume of feedback
  • the use of notes in margins

For more information get in touch with LTDS@ncl.ac.uk

Transforming Assessment Webinars

Dr Mathew Hillier, Monash Education Academy, Monash University, Australia and Professor Geoffrey Crisp, PVC-Education, University of New South Wales, Australia will be hosting a series of webinars over the coming months focusing on transforming assessment with  topics such as digital literacy, written and audio feedback and blended simulation-based learning. Take a look at the further details below. Continue reading “Transforming Assessment Webinars”

Turnitin UK Academic Integrity Summit 2017

I recently attended the Turnitin UK Academic Integrity Summit 2017 held in Newcastle Upon Tyne.  This was a very timely conference following the release of the QAA report into contract cheating.  I was concerned that this would be a day-long sales pitch from Turnitin but was pleasantly surprised to find the opposite. There were many presentations from institutions around the world, but very little ‘grandstanding’ from Turnitin.

Stephen Gow, Academic Integrity Coordinator, University of York

The first session I attended was a look at the approach from the University of York towards academic integrity. They discussed the importance of the language used at the University, moving away from terms such as “plagiarism” towards “academic integrity”. All their students have a mandatory academic integrity online tutorial they must complete in Semester 1 of Stage 1. They are working closely with the student union on their “integrity week” and are also working more closely with staff, including on their Postgraduate Certificate in Academic Practice (PGCAP).

Turnitin Data Workflows

The second session was a discussion session with the Turnitin staff exploring the types of data and statistics institutions would like to get out of Turnitin. This included reports on feedback return time, statistics around number of students receiving extensions, archiving, learning analytics, and reporting on the various functions used. We hope Turnitin will use this in the further developments of the software.

Bill Loller, Turnitin

The third session was facilitated by Bill Loller, Chief Product Officer at Jobvite, who is working on a technical solution to expose contract cheating for Turnitin. They are using expertise from the field of forensic linguistics to develop a product. Forensic linguistics may be used in a court case to determine whether a person did, or did not, write a document. They are currently testing their modelling and developing a report that will provide a confidence score.

Bill continued this theme into a larger session with the group, showing some of the contract cheating/essay mills websites prevalent online. He admitted that Turnitin may have helped with this problem – “crack down on plagiarism and students will look elsewhere”. These websites offer 10,000 words for approximately £300.

Simon Bullock, QAA

Simon Bullock from the QAA was next to discuss his recent publication “Contracting to Cheat in Higher Education – How to Address Contract Cheating, the Use of Third-Party Services and Essay Mills.”  He discussed the risks to the public if students were obtaining their degrees through cheating but that despite attempts it is not yet illegal to offer essay mill services online. The QAA is exploring as many non-legislative methods as possible.

Irene Glendinning, Coventry University

Irene Glendinning of Coventry University presented her research work analysing the impact of policies for plagiarism in Higher Education across Europe. She highlighted the UK and Ireland as being some way ahead of many other countries in Europe. They have developed an academic integrity maturity model, a tool to compare the results of the impact analysis across 27 EU member states.

Cath Ellis, New South Wales

The presentation that had the most impact on me was from Cath Ellis from the University of New South Wales. Cath reported that there was too much anecdotal information forming decisions, and not enough hard data.

To find out how many students are using contract cheating services, Cath asked them anonymously. Out of the 14,096 students surveyed around 6% (n=814) admitted to cheating in some form during their programme. The vast majority of this cheating comes in the form of assistance from other or former students. It is not commercially driven. The cheating group’s attitudes show they are less likely to think it’s wrong, although there was no discernible difference between English and non-English speaking students. Non-English speaking students are as likely to think cheating is wrong as English speaking students. Other findings of the study showed that when there are perceptions that there are a lot of opportunities to cheat, cheating goes up. And when there is dissatisfaction with the teaching environment, cheating goes up.

Cath discussed the need for students to have “ethical fitness” – we should not try to remove every opportunity to cheat as students need to be ethical.

She then discussed the various types of contract cheating and review some of the typical websites.

Assessment design is widely advocated as a possible solution to contract cheating, but Cath argued that this is a myth. We should not change our assessment design because of a small percentage of cheaters. Reduced assessment time (shorter deadlines) will actually drive students towards essay mills.

Cath noted that we are not having the correct conversations with students and advised us to discuss contract cheating with them. Part of the study looked at the perceptions of how prevalent contract cheating is, compared to how damaging it is. The study showed that students in the cheating group thought that a lot of students were doing it and it was not that serious. Staff members thought it was not very common but it was very serious. Students in the non-cheating group followed the same path as the students in the cheating group. They also thought that lots of students were doing it while it was not very serious.

Professor Phil Newton – Swansea University

The last presentation was given by Professor Phil Newton from Swansea University. He presented various research projects that explored academic integrity.

I found the event extremely useful and I have reflected since on the way Newcastle University approaches academic integrity. The presentation from Cath Ellis convinced me that we should not be changing any approaches to assessment to attempt to counter the small number of cheating students, but we should be minimising their opportunities to cheat. We also need to be having more conversations with staff and students about the promotion of academic integrity, and the impact contract cheating could have on their career.


Assessment & Feedback Event – 1st Nov – all welcome

Assessment and feedback continue to be a source of student dissatisfaction across the sector. In particular student surveys highlight concerns about the alignment of feedback to marking criteria and inconsistencies in both the application of criteria and quality of feedback received.

The HaSS Faculty will be holding a workshop, to discuss these issues in the context of student transitions, on:

Wednesday 1st November 2017
1300-1600 (lunch will be provided)
Lindisfarne Room, Hadrian Building

Though hosted by HaSS, anyone interested in this event from any faculty is welcome to attend – please feel free to disseminate details of this event to your colleagues.

The session will explore the following questions:

  • How can we better understand assessment and feedback in the context of student transitions?
  • What are the assumptions inherent in our assessment criteria and feedback?
  • Can electronic assessment and feedback tools enhance students’ academic literacy?
  • Can we develop a ‘student as partners’ approach in assessment and feedback?

It will be particularly relevant for Degree Programme Directors and Module Leaders, whose input will help identify some key priorities for further action.

It will include presentations from Rowan South (Education Officer, Newcastle Union Students Union), Graeme Redshaw-Boxwell (Learning and Teaching Development Service) and Sarah Graham (Combined Honours).

To attend, please complete the booking form.

If you have any queries in the run up to the event, please contact susan.mclean@ncl.ac.uk.

Essay Mills and Contract Cheating

Wednesday 18th October 2017 marks the second International Day of Action against Contract Cheating, promoted by the International Center for Academic Integrity (ICAI).

Contract cheating occurs when one person completes academic work, such as an essay, assignment, test or exam, for another who then submits it for academic credit. This behaviour undermines academic standards and devalues the qualifications of those who do not cheat.

In 2016, in response to serious concerns among higher education providers and from government, the Quality Assurance Agency (QAA) published a report on the growing threat of third parties helping students to cheat, known as custom essay writing services, or ‘essay mills’. Following this, QAA has now this month published guidance for students and universities on tackling this problem and promoting academic integrity in higher education. It sets out the steps that universities can take to deal with forms of contract cheating, and recommends:

  • clear information for students on the risks of cheating, including academic misconduct being reported to relevant professional bodies
  • support for students to develop independent study skills, including academic writing
  • using a range of assessment methods to limit opportunities for cheating
  • blocking essay mill sites and taking action against essay mill advertising on campus
  • smarter detection, including new software and greater familiarity with students’ personal styles and capabilities
  • appropriate support for whistle blowing – to protect accuser as well as accused
  • student involvement on academic misconduct policies and panels.

Newcastle University is committed to defending academic integrity and freedom, and will be considering how best to take account of this guidance within its own policies and procedures.

The International Center for Academic Integrity has published an institutional toolkit against contract cheating that can be downloaded here.

STAR CASE STUDY – Saving Sim-Man

Are you struggling to offer active and experiential learning to large numbers of students?  SimMan could save the day.

SimMan is a high-fidelity patient simulator who can be programmed to display a wide range of physiological and pathophysiological signs and respond appropriately to treatment, be it physical, e.g. cardiopulmonary resuscitation, or therapeutic, e.g. administration of drugs.

But surely only a few students can make use of SimMan at a time?

Clare Guilding (Lecturer in the School of Medical Education) has developed an effective way of using SimMan along with interactive voting technology to provide an engaging learning experience for a lecture theatre full of students.

Clare explains, ‘To enable the entire class to engage in clinical decision-making, split-screen and interactive voting technologies are employed.’

One of the screens projects the physiological readouts from SimMan such as his blood pressure, ECG heart trace and oxygen saturation; the other screen is linked to a TurningPoint interactive quiz.

Each student is supplied with a TurningPoint handset and at a series of key clinical points throughout the scenario, the students are asked to vote individually and anonymously on the most appropriate course of action (e.g. initial patient management steps, which drug should be administered etc.).

The option with the most votes, (whether or not this was the correct) is applied to SimMan and the students then observe the physiological effects this has in real time.

Clare said: ’In the online end of unit evaluation 76% of respondents agreed or strongly agreed that SimMan had enhanced their learning experience.’

It also enabled students to see how their lectures applied to clinical practice:

One commented that ‘the lecture using SimMan at the end was really good, especially using TurningPoint so that we could try to ‘treat’ SimMan. It kept the lecture clinically-focussed and enabled us to see how the information would come in useful in practice’.

To find out more about SimMan and read about medical students’ repeated attempts to save his life, read the full case study on the Case Study database.

Or if you have your own example of really effective teaching practice in your School do get in touch with ltds@ncl.ac.uk.