June 2022 – FMS TEL

Case Study: Virtual Oral Presentations as a summative assessment

How do oral presentations work for 100% online modules?

Presentations helps students put across an idea while expressing their personalities, which is hard to do in an essay.

Introduction

Oral presentations are a popular choice of assessment in the Faculty of Medical Sciences, especially in our e-Learning modules. Students are asked to submit a pre-recorded presentation to Canvas and the markers watch the presentations at a time and place that suits them.

Diarmuid Coughlan, module leader for ONC8028 Practical Health Economics for Cancer, has kindly agreed to walk us through how the Virtual Oral Presentation element works on his module.

The Assessment

This year we had 14 students on the module. We asked the students to create a 15 minute presentation using either Zoom, Panopto (Recap) or PowerPoint.

We informed the students right at the start of the module that an oral presentation was part of the assessment and 4 weeks into the module we provided a formative assessment. The formative assessment allowed students to familiarise themselves with their chosen software, gain experience talking to a camera and also get some limited feedback on their presentation skills.

The submissions are double marked by 2 markers. Marking is completed separately by each marker outside of Canvas, then markers meet to discuss which marks/comments would be entered into Canvas and made visible to each student.

The Set Up

We provided 2 submission points in Canvas:

Recording Submission Point:

This area was used for the marking. It was set up as Media Recording for MP4 uploads (max of 500 mb) with a Text Entry option for Panopto users (no size limit).

We allowed students to choose which technology they were most comfortable with and provided video and written instructions for Panopto and Zoom. PowerPoint instructions were added later as an option with links to guidance provided by Microsoft.

We also provided some instructions so students could crop their recordings to comply with the 15 minute time limit.

You are limited by time so remember to edit your recording so it is no longer than 15 minutes. Instructions: Windows | Mac | Panopto

Slide Submission Point:

This area had a 0 point value. It was set up as a File upload area for students to submit their slides as .ppt or .pdf, this allowed us to get a turnitin plagiarism score for each presentation as well as a reference copy of the slides, should anything be unclear in the video recordings.

How did it go?

There was a lot of fear from students initially. We encouraged students to give it a go, informing them that we were not trying to trick them. We provided clear guidance on what we expected and provided a rubric with a breakdown of points, clearly showing only a small percentage of the grade would be based on their presentation style and delivery. The content of the presentation was the most important part!

The use of technology was varied:

As markers we also had to overcome our fears of technology.

PowerPoint is easier once you know how to access recordings (you have to download the file, then click start slideshow).

Sometimes the Panopto recordings were hard to find, especially if students had experience of using the technology in Blackboard and did not follow the Canvas instructions correctly.

What are your next steps?

We only provided grades with a short feedback comment last year, we plan to provide more extensive feedback going forward
We will add more video content into the module as examples of how to create engaging slides and showcase our presentation styles – hopefully leading by example
We would also like to provide examples of a good presentation vs a bad presentation

Captioning and Transcribing – What Standard Should I Aim for?

When captioning and transcribing, what is meant by ‘accuracy’? When are captions good enough?

In FMS TEL and LTDS many team members regularly work with captioning videos, in particular for our own instructional videos or webinars. Recently a few of us have been talking about how we caption videos and how we decide what to correct. After discovering we all had differences of opinion about what to keep and what to edit, it seemed like a good idea to think through the issues.

This webinar from the University of Kent features Nigel Megitt from the BBC talking about priorities when captioning and audio describing TV programme. It includes research on how people with different levels of hearing feel about captions.

Note: These discussions refer to materials created for staff training and other internal uses. For student materials, please see the university policies on captioning materials for students and the captioning disclaimer to help with your decision-making.

Different Types of Captioning and Transcription

Commercial captioning companies offer a range of levels of detail. We do not outsource these tasks, but the predefined service levels can help clarify what decisions are made when captioning. Is verbatim captioning better than a lightly edited video? An accurate set of captions or transcript should include hesitations and false starts, but a more readable one might remove these for fast comprehensibility and more closely resemble the script of a speech.

Key Considerations

Destination – who is the audience? What do they need?
Speaker(s) – how can they be best represented? How do they feel about you editing their speech for clarity (e.g. removing filler words) vs correcting captions to verbatim?
Timescale – how fast do you need to turn this around? Longer videos and heavier editing takes longer.
Longevity – will this resource be around for a long time and reach a wider audience? If so it may merit extra polish.

Once you have broadly decided on the above, you can deal with the nitty-gritty of deciding what to fix, edit or remove. Deciding on your approach to these common issues means you won’t have to make a decision each time you find an error in your transcript. If working with a few other colleagues on a larger project you might want to agree with each other what standard you are aiming for to create uniformity.

Editing Decisions

The ASR occasionally misunderstands speech and adds incorrect captions that may be distracting, embarrassing or inappropriate, for example adding swearing or discriminatory language that the speaker has not in fact used. Checking the captions for these is a great start, and is likely to be appreciated by all speakers!

We don’t usually speak in the same way we write. Normal speech is full of little quirks that don’t appear in text. Some of these include…

False starts (If we take… no actually let’s start with… yes, OK, if we take question 4 next…)
Hesitations (um….ah…)
Filler Words (you know, like, so…)
Repeated words (You can do this by… by reading the text)

Other Considerations for Captioning

Remember that captions will be read on screen at the pace of the video. This means that anything that you can do to increase readability may be useful for the viewer. This includes simple things like…

Fixing initialisms and acronyms (PGR not p g r, SAgE not sage)
Fixing web and email addresses (abc1@ncl.ac.uk, not A B C One At Newcastle Dot A See Dot UK)
Adding quotation marks around quotes.

You may also consider…

Presenting numbers using figures rather than words (99% not ninety-nine percent)
Removing awkward breaks (When Panopto separates a final word from its sentence.)
Fixing inaccurate punctuation like full stops in the wrong places, or commas and apostrophes (this is quite time consuming).

Considerations for Transcription

As well as the editing and tidying jobs above, before beginning to work with your file, consider whether or not the timing points are going to be important, and how you are going to denote different speakers, or break up the text. For example, for an interview you may need to denote various speakers very clearly. By contrast, for a training webinar, even if there are two presenters it might not be crucial to distinguish them. Instead it might be better to add headings for each slide so that the two resources can be used side by side.

Once you have decided on what to edit and what to ignore, your process will move along much faster as you won’t need to decide on the fly.

Keep an eye on the blog over the next few weeks for tips on how to quickly manage and edit your caption and transcription files.

Personal Tutoring, an example of rapid application development?

What is Personal Tutoring?

Personal Tutoring is the process of assigning the availability of university staff for student tutoring. It is not actually the process of assigning individual students to individual staff. Staff in the faculty are assigned to programme groupings/pots (not individual programmes), these groupings have a lead administrator that can then work out by looking at the expected student intake whether they need more staff resource, or can free it up for others.

The faculty admin team have undertaken this process for many years using excel spreadsheets and email communications to pass the information back and forward. This kinds of activity is both time consuming and prone to errors caused by duplicate copies of data and missed communications.

This is the perfect example of a process that can be done better using a web application, the kind of work the Technologies Developers in FMS TEL undertake all the time.

No time to plan properly

Sometimes a project comes to the unit that needs to be completed in a short time frame. Ideally the amount of time spent on a new website would be evenly spread between specifications, design, implementation, testing and support/improving., it may even go through many loops of these processes.

When a project does not have the luxury of time, then all these steps need to be compressed and decisions made on which steps need to be prioritised . In the case of personal tutoring the design phase and specifications where collaborated from the old Excel spreadsheets and turned into a simple tabular wireframe display. These spreadsheets where also identified as the origin of the data the site would be based off and import scripts planned accordingly. No complex interface features where offered, just a clean display of the data, with filters and stats to help the tutoring assignments. As for the implementation of the site, we decided to host the new site on top of another, saving time on hosting framework and infrastructure. We chose a site that had similar tools (FMS Projects, which has a statistics section) and tools we could utilise. We also based the core of the new site off knowledge and experience the team was used to (API’s and data tables, spreadsheet importing / exporting). Finally the testing and support side was compressed, keeping the interface simple, reducing the need for support and documentation. The limited number of users the site may have, also helps support as we can offer short term direct guidance.

With all these measures, we managed to reduce a development process that could take 6 months down to 3.

The Results

The Personal Tutoring site is due to go live in July 2022. We managed to write and get a functional version of the site done in roughly 2 months. This left 1 month planning the release, testing and show casing the site to the customers and making improvements from their feedback. Overall we are pleased with the structure and quality of the site. The code design is based on solid principles and should offer a degree of flexibility when Personal Tutoring gets used and the inevitable suggested improvements come through.

If you are interested in this topic and wish to learn more, please contact:

Dan Plummer, Learning Technologies Developer, dan.plummer@newcastle.ac.uk

Probity in Online Exams – Success at TEPARG for SDS and FMS TEL

This post highlights the work done by SDS and FMS TEL to support remote exams over the past few years. Successful techniques for keeping students informed and supported are discussed, including student voice.

Work done by the School of Dentistry and FMS TEL Team won first prize at the Trans-European Pedagogic Anatomical Research Group Hybrid Conference this year, for the presentation on ‘Adopting a flexible approach to professional anatomy spotter exams during COVID’. You can read the internal news item on Sharepoint.

This work was also subsequently presented at the Newcastle University Learning and Teaching Conference 2022. Newcastle Staff can view the poster here for an overview.

This work centres on how exams subject to oversight from professional bodies – in this case the General Dental Council – could be run with adequate probity when these could not be undertaken in person.

The first element of this is to run the exams ‘live’ rather than as a 24h format. This meant that teaching staff and professional staff could be on hand to resolve any technical difficulties or make invigilation decisions.

A variety of question designs were also thought through. The final format was a ‘stimulus’ question type on Canvas, allowing an image to be shown with answer options alongside. Now this question type can also be used with Inspera exams.

The support of students with SSPs was also a key consideration, and rest breaks were granted across the board when exams were very lengthy. Students were asked what they thought about the probity measures put in place, such as the use of an exam declaration, and checking responses which may have been copy-pasted. Overall, the response was positive.

A key element in the smooth running of these exams was the preparation offered to students beforehand, such as clear messaging via email and the opportunity to practice with the exam environment before undertaking their summative assessments. 99% of students agreed that they had been well-supported throughout the process and during the exams themselves via the Zoom exam hotline. Most calls were students double-checking their responses had been submitted, rather than having technical issues.

Learning from these exams has already been carried forward for digital exams running through the new Inspera exam system, and the confidence staff and students have in the procedure means that any future changes to circumstances will be much easier to navigate for these teams and cohorts.