Category Archives: Assessment

Introducing the digital exams service

Building on the solid foundations of OLAF provision, and the successful first 2 years of the Diversifying and Expanding Online Exam Provision project, the University’s Technology Enhanced Learning Sub-Committee have approved the launch of a new combined Digital Exams service.

The story so far …

Newcastle University’s Online Assessment and Feedback (OLAF) Service has been running high stakes secure online exams using Blackboard’s test tool since 2007/08. The 13 years since that first exam have seen OLAF come of age, supported by well-established institutional processes that ensured all 132 OLAF exams in 2018/19 went smoothly.

In 2017/18 the Diversifying and Expanding Online Exam Provision project was launched, and the first of some new types of digital exams were piloted using software called WISEflow. Bring Your Own Device was introduced, enabling students to use their own laptops to sit a secure digital exam. Alongside this, moving essay and long written answer exam questions from paper to online has also become possible for the first time.

Continue reading Introducing the digital exams service

Expanding and diversifying online exam provision

In 2017/18 academic year the University launched a project that aims to make online summative assessment possible for a wide range of assessment types and in non-cluster environments, which is one of the objectives in the TEL Roadmap.

The project focuses on two key areas:

  • Expanding the types of online exams that we can deliver here at Newcastle
  • Exploring the possibility of students using their own laptops to take secure online exams

Continue reading Expanding and diversifying online exam provision

Student views on feedback forms

To find out more a student intern, working with staff in LTDS,  evaluated existing feedback forms and gathered opinions from students to identify what works and what could be improved. The project considered a total of 66 forms from 19 different schools and included focus groups and interviews with individual students.

What did they find?

These are a few key findings and you can find full details in the project report.

Form Design

Have clear, separate sections showing:

  • Strengths and areas for improvement
  • Clear advice for future work

Only use tick boxes for objective areas of the marking criteria, such as grammar. When tick boxes were used for subjective areas, such as argument, students found this unhelpful.

Look at your feedback forms and consider whether these should be redesigned. Consult with the students in your school as part of the process.

Utilising the form

Type feedback, wherever possible.

Introduce structured opportunities to help students understand:

  • expectations of the marking criteria
  • the ways in which this is reflected in the feedback sheet

Discuss how you use marking sheets with your colleagues. Try to develop a consistent approach to:

  • the volume of feedback
  • the use of notes in margins

For more information get in touch with LTDS@ncl.ac.uk

Transforming Assessment Webinars

Dr Mathew Hillier, Monash Education Academy, Monash University, Australia and Professor Geoffrey Crisp, PVC-Education, University of New South Wales, Australia will be hosting a series of webinars over the coming months focusing on transforming assessment with  topics such as digital literacy, written and audio feedback and blended simulation-based learning. Take a look at the further details below. Continue reading Transforming Assessment Webinars

Turnitin UK Academic Integrity Summit 2017

I recently attended the Turnitin UK Academic Integrity Summit 2017 held in Newcastle Upon Tyne.  This was a very timely conference following the release of the QAA report into contract cheating.  I was concerned that this would be a day-long sales pitch from Turnitin but was pleasantly surprised to find the opposite. There were many presentations from institutions around the world, but very little ‘grandstanding’ from Turnitin.

Stephen Gow, Academic Integrity Coordinator, University of York

The first session I attended was a look at the approach from the University of York towards academic integrity. They discussed the importance of the language used at the University, moving away from terms such as “plagiarism” towards “academic integrity”. All their students have a mandatory academic integrity online tutorial they must complete in Semester 1 of Stage 1. They are working closely with the student union on their “integrity week” and are also working more closely with staff, including on their Postgraduate Certificate in Academic Practice (PGCAP).

Turnitin Data Workflows

The second session was a discussion session with the Turnitin staff exploring the types of data and statistics institutions would like to get out of Turnitin. This included reports on feedback return time, statistics around number of students receiving extensions, archiving, learning analytics, and reporting on the various functions used. We hope Turnitin will use this in the further developments of the software.

Bill Loller, Turnitin

The third session was facilitated by Bill Loller, Chief Product Officer at Jobvite, who is working on a technical solution to expose contract cheating for Turnitin. They are using expertise from the field of forensic linguistics to develop a product. Forensic linguistics may be used in a court case to determine whether a person did, or did not, write a document. They are currently testing their modelling and developing a report that will provide a confidence score.

Bill continued this theme into a larger session with the group, showing some of the contract cheating/essay mills websites prevalent online. He admitted that Turnitin may have helped with this problem – “crack down on plagiarism and students will look elsewhere”. These websites offer 10,000 words for approximately £300.

Simon Bullock, QAA

Simon Bullock from the QAA was next to discuss his recent publication “Contracting to Cheat in Higher Education – How to Address Contract Cheating, the Use of Third-Party Services and Essay Mills.”  He discussed the risks to the public if students were obtaining their degrees through cheating but that despite attempts it is not yet illegal to offer essay mill services online. The QAA is exploring as many non-legislative methods as possible.

Irene Glendinning, Coventry University

Irene Glendinning of Coventry University presented her research work analysing the impact of policies for plagiarism in Higher Education across Europe. She highlighted the UK and Ireland as being some way ahead of many other countries in Europe. They have developed an academic integrity maturity model, a tool to compare the results of the impact analysis across 27 EU member states.

Cath Ellis, New South Wales

The presentation that had the most impact on me was from Cath Ellis from the University of New South Wales. Cath reported that there was too much anecdotal information forming decisions, and not enough hard data.

To find out how many students are using contract cheating services, Cath asked them anonymously. Out of the 14,096 students surveyed around 6% (n=814) admitted to cheating in some form during their programme. The vast majority of this cheating comes in the form of assistance from other or former students. It is not commercially driven. The cheating group’s attitudes show they are less likely to think it’s wrong, although there was no discernible difference between English and non-English speaking students. Non-English speaking students are as likely to think cheating is wrong as English speaking students. Other findings of the study showed that when there are perceptions that there are a lot of opportunities to cheat, cheating goes up. And when there is dissatisfaction with the teaching environment, cheating goes up.

Cath discussed the need for students to have “ethical fitness” – we should not try to remove every opportunity to cheat as students need to be ethical.

She then discussed the various types of contract cheating and review some of the typical websites.

Assessment design is widely advocated as a possible solution to contract cheating, but Cath argued that this is a myth. We should not change our assessment design because of a small percentage of cheaters. Reduced assessment time (shorter deadlines) will actually drive students towards essay mills.

Cath noted that we are not having the correct conversations with students and advised us to discuss contract cheating with them. Part of the study looked at the perceptions of how prevalent contract cheating is, compared to how damaging it is. The study showed that students in the cheating group thought that a lot of students were doing it and it was not that serious. Staff members thought it was not very common but it was very serious. Students in the non-cheating group followed the same path as the students in the cheating group. They also thought that lots of students were doing it while it was not very serious.

Professor Phil Newton – Swansea University

The last presentation was given by Professor Phil Newton from Swansea University. He presented various research projects that explored academic integrity.

I found the event extremely useful and I have reflected since on the way Newcastle University approaches academic integrity. The presentation from Cath Ellis convinced me that we should not be changing any approaches to assessment to attempt to counter the small number of cheating students, but we should be minimising their opportunities to cheat. We also need to be having more conversations with staff and students about the promotion of academic integrity, and the impact contract cheating could have on their career.


Assessment & Feedback Event – 1st Nov – all welcome

Assessment and feedback continue to be a source of student dissatisfaction across the sector. In particular student surveys highlight concerns about the alignment of feedback to marking criteria and inconsistencies in both the application of criteria and quality of feedback received.

The HaSS Faculty will be holding a workshop, to discuss these issues in the context of student transitions, on:

Wednesday 1st November 2017
1300-1600 (lunch will be provided)
Lindisfarne Room, Hadrian Building

Though hosted by HaSS, anyone interested in this event from any faculty is welcome to attend – please feel free to disseminate details of this event to your colleagues.

The session will explore the following questions:

  • How can we better understand assessment and feedback in the context of student transitions?
  • What are the assumptions inherent in our assessment criteria and feedback?
  • Can electronic assessment and feedback tools enhance students’ academic literacy?
  • Can we develop a ‘student as partners’ approach in assessment and feedback?

It will be particularly relevant for Degree Programme Directors and Module Leaders, whose input will help identify some key priorities for further action.

It will include presentations from Rowan South (Education Officer, Newcastle Union Students Union), Graeme Redshaw-Boxwell (Learning and Teaching Development Service) and Sarah Graham (Combined Honours).

To attend, please complete the booking form.

If you have any queries in the run up to the event, please contact susan.mclean@ncl.ac.uk.

Essay Mills and Contract Cheating

Wednesday 18th October 2017 marks the second International Day of Action against Contract Cheating, promoted by the International Center for Academic Integrity (ICAI).

Contract cheating occurs when one person completes academic work, such as an essay, assignment, test or exam, for another who then submits it for academic credit. This behaviour undermines academic standards and devalues the qualifications of those who do not cheat.

In 2016, in response to serious concerns among higher education providers and from government, the Quality Assurance Agency (QAA) published a report on the growing threat of third parties helping students to cheat, known as custom essay writing services, or ‘essay mills’. Following this, QAA has now this month published guidance for students and universities on tackling this problem and promoting academic integrity in higher education. It sets out the steps that universities can take to deal with forms of contract cheating, and recommends:

  • clear information for students on the risks of cheating, including academic misconduct being reported to relevant professional bodies
  • support for students to develop independent study skills, including academic writing
  • using a range of assessment methods to limit opportunities for cheating
  • blocking essay mill sites and taking action against essay mill advertising on campus
  • smarter detection, including new software and greater familiarity with students’ personal styles and capabilities
  • appropriate support for whistle blowing – to protect accuser as well as accused
  • student involvement on academic misconduct policies and panels.

Newcastle University is committed to defending academic integrity and freedom, and will be considering how best to take account of this guidance within its own policies and procedures.

The International Center for Academic Integrity has published an institutional toolkit against contract cheating that can be downloaded here.

STAR CASE STUDY – Saving Sim-Man

Are you struggling to offer active and experiential learning to large numbers of students?  SimMan could save the day.

SimMan is a high-fidelity patient simulator who can be programmed to display a wide range of physiological and pathophysiological signs and respond appropriately to treatment, be it physical, e.g. cardiopulmonary resuscitation, or therapeutic, e.g. administration of drugs.

But surely only a few students can make use of SimMan at a time?

Clare Guilding (Lecturer in the School of Medical Education) has developed an effective way of using SimMan along with interactive voting technology to provide an engaging learning experience for a lecture theatre full of students.

Clare explains, ‘To enable the entire class to engage in clinical decision-making, split-screen and interactive voting technologies are employed.’

One of the screens projects the physiological readouts from SimMan such as his blood pressure, ECG heart trace and oxygen saturation; the other screen is linked to a TurningPoint interactive quiz.

Each student is supplied with a TurningPoint handset and at a series of key clinical points throughout the scenario, the students are asked to vote individually and anonymously on the most appropriate course of action (e.g. initial patient management steps, which drug should be administered etc.).

The option with the most votes, (whether or not this was the correct) is applied to SimMan and the students then observe the physiological effects this has in real time.

Clare said: ’In the online end of unit evaluation 76% of respondents agreed or strongly agreed that SimMan had enhanced their learning experience.’

It also enabled students to see how their lectures applied to clinical practice:

One commented that ‘the lecture using SimMan at the end was really good, especially using TurningPoint so that we could try to ‘treat’ SimMan. It kept the lecture clinically-focussed and enabled us to see how the information would come in useful in practice’.

To find out more about SimMan and read about medical students’ repeated attempts to save his life, read the full case study on the Case Study database.

Or if you have your own example of really effective teaching practice in your School do get in touch with ltds@ncl.ac.uk.

 

Star Case Study – Mock Viva Video in Politics

Doing your viva in Politics has been revolutionised by a new mock viva video for Ph.D. students.

 

Politics PGR Director Professor Tony Zito and Kate Manzo (Geography) realised that students often were not attending or were not paying attention to more traditional approaches to preparing students for their viva, so he decided to show them exactly what to expect by making a video.

Tony said: ‘A lot of students were just not coming to the sessions I was running about vivas and what their viva would be like.

‘I think for those who were in first or second year of their Ph.D.s their viva seemed very far away and for those nearing the end of their project the viva had become something too scary to think about.

‘So we decided to make sure that there would be something online that they could always access, perhaps even in the middle of the night when they were worrying about an upcoming viva.’

Tony enlisted the help of politics student Russell Foster, himself preparing for an upcoming viva.

Russell agreed to be filmed during a mock viva with Tony and Kate taking the roles of internal and external examiners.

The mock viva was kept very formal, with Russell entering the room in a suit to greet his examiners, just like in the real thing.

‘It was great of Russell to agree to do that on camera because it’s a pretty scary thing but he was happy to help other students.

‘The video worked really well and will hopefully give other students an idea of what to expect as their viva looms.

‘You can see Russell go through the whole process so hopefully it will be helpful to them to see the whole thing so clearly laid out.’

The video was posted on Youtube, with Russell’s permission and is used frequently as a resource for students in GPS approaching the end of their Ph.D.s.

Tony said: ‘We’re not sure of the impact yet. We did this in October 2013 and Ph.D.s are a slow process so we’ve not had that many students through yet to notice any particular trends but we expect that it can only have a positive impact.’

Thinking of doing a similar thing in your school? Contact LTDS@ncl.ac.uk  for more information or for technical support.

For more examples of good practice in teaching and learning from across Newcastle have a look at the Case Studies Database.