Probity in Online Exams – Success at TEPARG for SDS and FMS TEL

This post highlights the work done by SDS and FMS TEL to support remote exams over the past few years. Successful techniques for keeping students informed and supported are discussed, including student voice.

Work done by the School of Dentistry and FMS TEL Team won first prize at the Trans-European Pedagogic Anatomical Research Group Hybrid Conference this year, for the presentation on ‘Adopting a flexible approach to professional anatomy spotter exams during COVID’. You can read the internal news item on Sharepoint.

This work was also subsequently presented at the Newcastle University Learning and Teaching Conference 2022. Newcastle Staff can view the poster here for an overview.

This work centres on how exams subject to oversight from professional bodies – in this case the General Dental Council – could be run with adequate probity when these could not be undertaken in person.

The first element of this is to run the exams ‘live’ rather than as a 24h format. This meant that teaching staff and professional staff could be on hand to resolve any technical difficulties or make invigilation decisions.

A variety of question designs were also thought through. The final format was a ‘stimulus’ question type on Canvas, allowing an image to be shown with answer options alongside. Now this question type can also be used with Inspera exams.

The support of students with SSPs was also a key consideration, and rest breaks were granted across the board when exams were very lengthy. Students were asked what they thought about the probity measures put in place, such as the use of an exam declaration, and checking responses which may have been copy-pasted. Overall, the response was positive.

A key element in the smooth running of these exams was the preparation offered to students beforehand, such as clear messaging via email and the opportunity to practice with the exam environment before undertaking their summative assessments. 99% of students agreed that they had been well-supported throughout the process and during the exams themselves via the Zoom exam hotline. Most calls were students double-checking their responses had been submitted, rather than having technical issues.

Learning from these exams has already been carried forward for digital exams running through the new Inspera exam system, and the confidence staff and students have in the procedure means that any future changes to circumstances will be much easier to navigate for these teams and cohorts.

Developing audio commentated interviews to support student learning – NULT2022

As part of the conference, Susan Lennie and Eleanor Gordon presented this video detailing the student response to the introduction.

The written summary of the development of the intervention can be accessed below.

Case Study – Audio Commentaries on Client Consultations – Susan Lennie

Instant Feedback – Lindsey Ferrie

This post is about the use of Menti – a pretty polling tool that can show responses in real time on screen. Lindsey Ferrie used this to gather student feedback during an in-person event. Link to guides included.

In Biomedical, Nutritional and Sport Sciences this year there is a focus on improving assessment and feedback. Including student voice in this development work is key. As well as the normal routes for student voice, such as staff-student committees and module evaluation forms, there were certain questions that staff had in mind which could reveal a lot about how students were feeling about this aspect of their course.

The formal module evaluation forms don’t always capture a large sample from the student body, and as with any survey, generating a high response rate is very difficult, and can sometimes end up only revealing the most polarised responses. Informal live response tools were a potential way to gather large amounts of more representative data, as well as demonstrating that feedback is wanted, heard, and acted upon within a shorter cycle. It is also much more convenient to capture a response immediately when students are in the room.

Having seen Menti in use during the FMS TEL Conference, the decision was taken to try it out in a much bigger setting – the inductions for all programmes in the School. Overall, this includes around 1500 students.

A useful aspect of Menti in this situation was the capacity to provide a wide range of question types, as well as the ability to see answers as they are coming in. Naturally, this has some disadvantages too – students’ responses may be coloured by what they see on screen if this is displayed. Possibly due to its anonymous nature, students provided some genuinely challenging feedback which proved very useful.

When in use, Menti allowed for the sessions to be more interactive as sharing live responses is quite immersive. It also allowed for the presenters to directly react and respond to things as they came in and broke the ice between students and staff. This can be useful for quick questions about content as well as student voice activities, for example polling the level of understanding of a concept at the start and end of a teaching session.

In terms of providing evidence to share with other colleagues, or at committees, Menti retains the responses for you to review, and can generate lists of entered comments. It isn’t designed as a statistical analysis tool, so comments don’t come in a convenient format for analysis. Graphs can be screenshotted for simple sharing if needed. It is of course possible to read through the comments and type up an overview of common themes yourself if needed.

The instant feedback received highlighted that some students weren’t entirely satisfied with the feedback they were receiving on their lab reports. Based on the responses, the School was able to put on a dedicated feedback session to further explore this, and to find out what could be improved, and have already been able to implement some changes.

Tips

Consider adding a delay before sharing answers on screen
For free text answers, it may be prudent to summarise rather than share the screen
Consider numbers when selecting your question types – it’s not easy to read 30+ typed responses quickly.
Don’t overuse it – keep use to key points so the novelty doesn’t wear off
Just try it – and be prepared for the honesty in the responses!
Menti works best in the moment in a session, or when you want to gather responses instantly

Can I try it?

You can create a free account on Mentimeter.com. This free account has limits on how many individual questions you can make, but you can always make multiple presentations and switch to them when sharing your screen. The university does not have any subscription to this service. As such, do not ask students to input any personal or sensitive information as this won’t be covered by our data policies.

When presenting, simply ask the audience to visit the site and enter the provided code that you show on the screen. When you’re ready, you can display the results to your audience. See the FMS TEL Community for a full walkthrough.

Other Ideas for Use

Gauging the experience already in the room before starting a session (free text or poll)
A quick litmus test on an opinion – yes/no/not sure (could be done at the beginning and end to see how things change)
Challenging students to define a term
Choices and branching scenarios
Feeding back results on a solo exercise
Anonymous feedback / rating for an aspect of the session

Contacts

Lindsey Ferrie, Senior Lecturer, School of Biomedical, Nutritional and Sport Sciences

Resources

Mentimeter presenter site
Example of a text answer with summary. Newcastle staff only – via Recap, start at 7:40.
Example of a multiple choice Menti with shared result screen. Newcastle staff only – via Recap, start at 28:20.
Menti guide on the FMS TEL Community on Canvas or the MLE.

Case Study – Enhanced Personal Tutoring: Embedding Academic Mentorship

A recent case study by David Kennedy about personal tutoring in the School of Medical Sciences and its impact on student satisfaction.

Dr David Kennedy of the School of Medical Education recently shared his work on personal tutoring and academic mentorship via the LTDS blog. The case study details what changes were made, and shows the massive increase in student satisfaction that resulted.

“As a School, we had a desire to improve academic and professional development, as well as pastoral support, for all of our students to enable them to achieve their true potential and support their transition to the workplace.”
David Kennedy, SME

This case study will be of interest to anyone who is interested in mentoring and pastoral support, and student experience.

Enhanced Personal Tutoring: Embedding Academic Mentorship