Author: Eleanor

Captioning and Transcribing – What Standard Should I Aim for?

When captioning and transcribing, what is meant by ‘accuracy’? When are captions good enough?

In FMS TEL and LTDS many team members regularly work with captioning videos, in particular for our own instructional videos or webinars. Recently a few of us have been talking about how we caption videos and how we decide what to correct. After discovering we all had differences of opinion about what to keep and what to edit, it seemed like a good idea to think through the issues.

This webinar from the University of Kent features Nigel Megitt from the BBC talking about priorities when captioning and audio describing TV programme. It includes research on how people with different levels of hearing feel about captions.

Note: These discussions refer to materials created for staff training and other internal uses. For student materials, please see the university policies on captioning materials for students and the captioning disclaimer to help with your decision-making.

Different Types of Captioning and Transcription

Commercial captioning companies offer a range of levels of detail. We do not outsource these tasks, but the predefined service levels can help clarify what decisions are made when captioning. Is verbatim captioning better than a lightly edited video? An accurate set of captions or transcript should include hesitations and false starts, but a more readable one might remove these for fast comprehensibility and more closely resemble the script of a speech.

Key Considerations

  • Destination – who is the audience? What do they need?
  • Speaker(s) – how can they be best represented? How do they feel about you editing their speech for clarity (e.g. removing filler words) vs correcting captions to verbatim?
  • Timescale – how fast do you need to turn this around? Longer videos and heavier editing takes longer.
  • Longevity – will this resource be around for a long time and reach a wider audience? If so it may merit extra polish.

Once you have broadly decided on the above, you can deal with the nitty-gritty of deciding what to fix, edit or remove. Deciding on your approach to these common issues means you won’t have to make a decision each time you find an error in your transcript. If working with a few other colleagues on a larger project you might want to agree with each other what standard you are aiming for to create uniformity.

Editing Decisions

The ASR occasionally misunderstands speech and adds incorrect captions that may be distracting, embarrassing or inappropriate, for example adding swearing or discriminatory language that the speaker has not in fact used. Checking the captions for these is a great start, and is likely to be appreciated by all speakers!

We don’t usually speak in the same way we write. Normal speech is full of little quirks that don’t appear in text. Some of these include…

  • False starts (If we take… no actually let’s start with… yes, OK, if we take question 4 next…)
  • Hesitations (um….ah…)
  • Filler Words (you know, like, so…)
  • Repeated words (You can do this by… by reading the text)

Other Considerations for Captioning

Remember that captions will be read on screen at the pace of the video. This means that anything that you can do to increase readability may be useful for the viewer. This includes simple things like…

  • Fixing initialisms and acronyms (PGR not p g r, SAgE not sage)
  • Fixing web and email addresses (abc1@ncl.ac.uk, not A B C One At Newcastle Dot A See Dot UK)
  • Adding quotation marks around quotes.

You may also consider…

  • Presenting numbers using figures rather than words (99% not ninety-nine percent)
  • Removing awkward breaks (When Panopto separates a final word from its sentence.)
  • Fixing inaccurate punctuation like full stops in the wrong places, or commas and apostrophes (this is quite time consuming).

Considerations for Transcription

As well as the editing and tidying jobs above, before beginning to work with your file, consider whether or not the timing points are going to be important, and how you are going to denote different speakers, or break up the text. For example, for an interview you may need to denote various speakers very clearly. By contrast, for a training webinar, even if there are two presenters it might not be crucial to distinguish them. Instead it might be better to add headings for each slide so that the two resources can be used side by side.

Once you have decided on what to edit and what to ignore, your process will move along much faster as you won’t need to decide on the fly.

Keep an eye on the blog over the next few weeks for tips on how to quickly manage and edit your caption and transcription files.

Probity in Online Exams – Success at TEPARG for SDS and FMS TEL

This post highlights the work done by SDS and FMS TEL to support remote exams over the past few years. Successful techniques for keeping students informed and supported are discussed, including student voice.

Work done by the School of Dentistry and FMS TEL Team won first prize at the Trans-European Pedagogic Anatomical Research Group Hybrid Conference this year, for the presentation on ‘Adopting a flexible approach to professional anatomy spotter exams during COVID’. You can read the internal news item on Sharepoint.

This work was also subsequently presented at the Newcastle University Learning and Teaching Conference 2022. Newcastle Staff can view the poster here for an overview.

This work centres on how exams subject to oversight from professional bodies – in this case the General Dental Council – could be run with adequate probity when these could not be undertaken in person.

The first element of this is to run the exams ‘live’ rather than as a 24h format. This meant that teaching staff and professional staff could be on hand to resolve any technical difficulties or make invigilation decisions.

A variety of question designs were also thought through. The final format was a ‘stimulus’ question type on Canvas, allowing an image to be shown with answer options alongside. Now this question type can also be used with Inspera exams.

The support of students with SSPs was also a key consideration, and rest breaks were granted across the board when exams were very lengthy. Students were asked what they thought about the probity measures put in place, such as the use of an exam declaration, and checking responses which may have been copy-pasted. Overall, the response was positive.

A key element in the smooth running of these exams was the preparation offered to students beforehand, such as clear messaging via email and the opportunity to practice with the exam environment before undertaking their summative assessments. 99% of students agreed that they had been well-supported throughout the process and during the exams themselves via the Zoom exam hotline. Most calls were students double-checking their responses had been submitted, rather than having technical issues.

Learning from these exams has already been carried forward for digital exams running through the new Inspera exam system, and the confidence staff and students have in the procedure means that any future changes to circumstances will be much easier to navigate for these teams and cohorts.

The Unessay – NULTConf

Dr Stephanie Holton presented her Unessay assessment task – what could you assess with an Unessay?

At the Learning and Teaching Conference Dr Stephanie Holton, along with two of her students, presented their experiences trialling a new approach to assessment – giving students freedom in how to present their learning, rather than setting a traditional essay task. This work was done in a module examining Ancient Greek texts.

What is an ‘unessay’ – and how exactly does it work? This talk explores the increasingly popular unessay as alternative assessment type, taking as a case study its implementation across several compulsory language modules in the School of History, Classics & Archaeology during 2020-21. Delivered by both the staff and students involved, it highlights the wide range of benefits – as well as the challenges – of diversifying assessment in some of our most traditional modules.

Dr Stephanie Holton

The full talk can be viewed here.

It was fantastic to see how the students were able to approach this new assessment methodology, and the outputs themselves were diverse, including models and digitally-created choose-your-own adventure books. The support provided included workshops and one-to-one sessions so that students knew what to expect, and were confident they were on the right track. While somewhat time-intensive in terms of support, the level of engagement and ownership the students felt around their creations was a clear benefit of this methodology.

Students were able to share their extra curricular creative skills, and the diversity of their approaches meant that they were prompted to explore their texts in new ways – for example researching clothing colours for their model characters that would be appropriate for the time. The need for these extra details prompted students to do more research around their text that they may not have otherwise done, broadening their subject knowledge.

These kinds of assessments open the door to students using and expanding their digital skills, even though this isn’t the focus of the assessment. The element of choice allows them to choose their own level of comfort with how they’d like to present their project – whether this is in a digital format or physical.

What could you assess with an Unessay?

Exploring 3D Anatomy – The University of Cape Town Launch

This week students from the University of Cape Town have joined the pilot of the Exploring 3D Anatomy course.

The second pilot of the Exploring 3D Anatomy course is now live, with students participating at the University of Cape Town in South Africa.

Leonard Shapiro presented an introduction to the course on 18th March to a class of second-year medical students – many of whom joined the class remotely via Teams.

Photo of a presentation Exploring 2D Anatomy.
Leonard presents the course
Photo of a classroom with Zoom call showing on a projection screen.
An example of one of the course exercises is shared
Photo of presenter and project screen showing exercises.
Leonard presents the course exercises
Dr Iain Keenan in the course trailer
Dr Iain Keenan in the course trailer

The course trailer was also shown, and students were invited to sign up using a QR code posted on the slide show as well as on posters in the lecture theatre. Senior lecturer in Anatomy, Dr Geney Gunston, also posted the sign-up link for the students on VULA, the university messaging system.

Many students have signed up, and we look forward to working with them over the next few weeks as they Explore 3D Anatomy from home.

After this pilot and a further development phase, we hope to make the course more widely available to many more students.

Images provided by UCT medical student Mulaudzi God-mother Matodzi-muswa and Leonard Shapiro.

Developing audio commentated interviews to support student learning – NULT2022

As part of the conference, Susan Lennie and Eleanor Gordon presented this video detailing the student response to the introduction.

The written summary of the development of the intervention can be accessed below.

Case Study – Audio Commentaries on Client Consultations – Susan Lennie

Designing Convertible Teaching – NULT2022

Conference Materials – Designing Convertible Teaching

All of our posts about this conference can be seen under the tag NULTConf2022.

This workshop was presented in person for the first time at the Learning and Teaching Conference 2022. Newcastle University staff wishing to access the resources and the recording of the online version can do so here.

View Recording And Resources (canvas)
View Recording and resources (MLE)

NULT Conference 2022

Looking forward to seeing you all at the Learning and Teaching Conference!

FMS TEL are well-represented at the University Learning and Teaching Conference 2022. We look forward to seeing you at the conference, and hearing what you think of our sessions, videos and posters! All of our posts about this conference can be seen under the tag NULTConf2022.

To book your place at the conference, or find out more, please visit the Learning and Teaching Conference homepage.

Designing Effective Rubrics

A written summary of our training on using rubrics with links to the full webinar resources.

We have recently delivered some training for BNS (School of Biomedical, Nutritional and Sport Sciences) in collaboration with Rebecca Gill and Susan Barfield from LTDS. The two sessions covered Rubrics – both their design and how they can be implemented in Turnitin. You can access the training recordings and resources at the foot of this page.

Examples covered included:

  • a rubric with very few criteria and letter grading
  • a rubric with weighted criteria and bands
  • a very fine-grained rubric that awarded numerical points based on ten different criteria.

What are Rubrics for?

Rubrics can be used to evaluate assessments, whether you use a quantitative rubric to calculate marks, or a qualitative one with more wiggle room. Using a rubric makes it easier to identify strengths and weaknesses in a submission, and creates common framework and assessment language for staff and students to use. This in turn can help make learning expectations explicit to learners, and assist in the provision of effective feedback.

What is the best way?

There is no one way to design a perfect rubric, as assessments are very individual.

Before you begin you may want to consider how you can design your rubric to lessen the marking or feedback workload. Quantitative rubrics can reduce decision-making difficulties as this means you don’t need to consider what mark to give within a band. On the other hand, you may need this flexibility to use professional judgement. A detailed rubric with less wiggle room per descriptor also acts as detailed feedback for students, reducing the need for writing long additional comments, but also takes longer to design.

Descriptors

When writing descriptors, ensure that there is enough clear and objective difference between each band. You may find that aligning your descriptors with an external framework helps you write them. This is critical for secure marking, and is helpful for students receiving that feedback. Using positive language also helps make this feedback easier to digest, and allows students to see what they need to include to improve.

Rubric Workflow

When creating a rubric, you can follow this basic process. At every stage it is important to consult local assessment guidelines and discuss progress with your colleagues for constructive feedback.

  1. Determine your assessment criteria – ideally these should be aligned with the learning outcomes of the task.
  2. Consider the weighting of each element, if required – is presentation as important as content?
  3. Decide whether you will need defined marks or flexible ranges. This may be partly determined by your in-house guidelines.
  4. How do marks in various criteria interact with or depend upon one another? For example, if there is a very low mark in a content criterion, does that mean that the assessment can never be a pass?
  5. Try to write out individual descriptors – if you’re having difficulty discriminating between bands you may need to adjust your structure.
  6. Test your rubric against former or dummy submissions and adjust as necessary. Does it work for a lower level of mastery as well as a middle-scoring and high-scoring submission? If you had difficulty deciding between criteria, or discover a double-credit/penalty, you will need to adjust.

Technical Setup

Turnitin allows for the use of Grading Forms and Rubrics. You can watch how to implement these in the Using Turnitin video in the session resources below.

Turnitin grading forms can be created to assist with marking assignments, allowing you to add marks and feedback under various criteria. When using these forms, the highest mark entered will become the grade for the assignment. You can also use this without scoring to give feedback.

Turnitin rubrics allow for marking under multiple criteria and bands. You can have standard rubrics that calculate grades, or qualitative rubrics that do not include scoring. Custom rubrics can be used for more flexibility within a band.

An alternative to using Turnitin is to integrate a rubric into the assignment itself by using a coversheet. (see the ‘Effective Rubrics – Using Turnitin’ video at 28m25s, link in the Canvas below).

Resources

A Quick Trial of 360° Cameras in a Practical Session

Find out the pros and cons of 360 degree meeting room cameras in practical sessions.

Roisin Devaney, Susan Lennie, Andrea McGrattan

Situation

This practical session was based around students creating a meal to align with the recommendations of the EatWell guide, using ingredients provided in the food handling lab. The session involved a short introduction, followed by time for the activity, and feedback. At the last moment there was a request to record the practical session, and therefore a 360° camera was used as a quick solution – an ideal opportunity to test the camera’s capabilities.

These rooms are not set up for ReCap, so having this portable method of recording allowed for the session to be captured. The camera was connected to Zoom so that a remote student could watch the session or review the recording. As it happened, no one watched the stream live, so there was no interaction with a remote participant. As shown from the screenshots below, the camera could cover the whole room without needing to be moved. It can also zoom in and out to focus on where the action was taking place.

Method

The camera was quick and simple to set up, and lecturers used lapel mics to capture sound when they were addressing the class. The camera was placed on the demonstrators’ bench at the front of the room, where it could see the kitchen spaces as well as the session leads, and the whiteboard.

All present were made aware of the device. It was relatively unobtrusive and didn’t get in the way of the session – most people just forgot about it. This is a bonus as it means the lecturers don’t have to consider what to capture at various points during the session. They could also walk around the room naturally rather than being stuck at the fixed point of presenting from a particular spot at the front as the camera would follow them.

Results

As a quick solution, it was easy to set up, and the predetermined settings allowed for much of the session to be captured. It provided a record of the practical session, though due to the level of automation in where the camera chose to focus, some detail and some sound was lost. For example, on some occasions, the camera would choose to focus based on an incidental sound in the room, like a pot being scraped. This kind of auto-selection would work better in a situation such as a seminar where everyone sits around one table with less background noise and more obvious turn-taking when speaking.

The camera also captured incidental discussions – as it was placed at the front where students were collecting ingredients, it captured discussions between students about what they were choosing to include in their meal.

It was also able to capture two areas simultaneously and display them side by side, allowing for an overview of the room as well as capturing individual discussions.

Other Applications

There are a range of applications for this type of device in a practical setting. For example, for a remote student, the device could be placed in a group of students so that some collaboration would be possible. This would allow them to be more involved in the practical aspect of the session, though of course it would not be a complete substitution for in-person attendance.

Though the session recordings are relatively long, once uploaded to Panopto, bookmarks could be added to allow students to quickly navigate to various parts of the session, for example the input and feedback.

Contacts

Roisin Devaney, Susan Lennie and Andrea McGrattan, Nutrition and Dietetics, School of Biomedical, Nutritional and Sports Sciences

Resources

Owl Camera

Art of the Possible Lightning Talk including a brief overview of this work at 24:22.

2021 Showcase

A showcase of some of our work from 2021 and our submissions to the Learning and Teaching Conference 2022.

The FMS TEL team have collaborated on a wide range of projects throughout 2021, some of which we have submitted to the internal Learning and Teaching Conference 2022.

Adopting a flexible approach to professional written exams during COVID

Staff from the School of Dental Sciences and FMS-TEL have collaborated throughout the pandemic to ensure summative assessments continue in a robust fashion to satisfy regulatory requirements. This required a flexible approach to online assessment which continues to be important as the uncertainty continues. This presentation details the challenges we have faced and solutions we have utilised, offering practical advice for those who may wish to run online exams in the future on any platform.

Sarah Rolland and Luisa Wakeling, School of Dental Sciences

Audio Commentary and Structured Asynchronous Teaching of Communication Skills in Dietetics and Nutrition

Role-play and peer observation are widely used as educational methods for teaching communication skills in healthcare education, which has become a greater challenge to implement during the pandemic. In the context of dietetics and nutrition, our students developed communication skills using innovative audio resources. This video will discuss student voice survey findings regarding engagement and quality perceptions of the resources, and advise those wishing to create similar tasks, including task setup and technical support.

Susan Lennie, Biomedical, Nutritional and Sport Sciences

Can Scaffolding help Reflective Practice?

This workshop sets out the aims of incorporating reflective practice with scaffolded approaches in your teaching contexts and invites you to explore their potential value in your programmes or modules. Structured reflective templates are one of the key developments of the new NU Reflect system, which are currently being piloted. This session will also offer case studies to demonstrate how colleagues have implemented reflective templates to support the student reflective process within their contexts.

Patrick Rosenkranz, Simon Cotterill, Sam Flowers, David Gillies, David Teasdale

Content Evolution

A brief look at the FMS bespoke VLE Ngage, and its special features which were designed for our distance learning programmes. This session will cover how we migrated our content into Canvas without losing functionality and the challenges we encountered. We will also discuss how the content has evolved during the pandemic and the changes made to enhance the student experience.

Emily Smith, FMS TEL

Creating and Using Animations to Explain Concepts

This video demonstrates a how animations can be used to demonstrate concepts in teaching. It showcases advanced animations that are used to show complex concepts and allow for dynamic and interactive content to be shared on screen. Should you not have access to high-end software, the video also shows what can be achieved using PowerPoint animations and shows some tools that can be used to develop understanding beyond static diagrams or simple videos.

Ashley Reynolds, FMS TEL

Designing Convertible Teaching

This workshop provides colleagues with practical tips on how to design teaching that can be converted between online and offline, and synchronous and asynchronous delivery styles with minimal effort. Participants will think through resource design using examples, and apply this knowledge to their own resources brought to the session. Participants will take away technological shortcuts that reduce the burden when changing delivery styles, as well as an understanding of how to design inherently flexible resources.

Eleanor Gordon, FMS TEL

Exploring 3D Anatomy: Collaborative Development of an Inclusive Online Course Supporting Universal Enhancement of Transferable Observational Skills

This talk covers the development of a MOOC which develops students’ skills in 3D Spatial Awareness in the context of the study of anatomy. This collaboration between Newcastle University and the University of Cape Town focuses on specific art-based and technology enhanced learning exercises to develop skills that will enhance students’ capabilities in clinical observation, diagnosis, and surgical training. Prior research and development of specific observation methodologies, their deployment in an online environment, and students’ learning outcomes will be shared.

Iain Keenan, School of Medical Education

Flipping case-based learning online in response to the pandemic

This presentation shares practice from rapid changes made to ‘flip’ learning and teaching in MB BS, in response to the pandemic. This is in the context of a new Clinical Decision Making course, originally designed to use online cases to supplement primarily face-to-face learning and teaching in both UK and NuMed. The pandemic gave need to rapidly shift to predominantly online delivery with more asynchronous delivery. We will demonstrate adaptions made and discuss lessons learned.

John Moss, David Kennedy, Dan Plummer

Staff development: collaboration across continents for using TEL

A study across campuses of Newcastle University into perceptions of TEL use found that staff in the NUMed and Singapore campuses felt they lagged behind in training for using TEL. The FMSTEL team responded with an online conference, collaborating with NUMed, at a time accessible to all. Various topics included specific technologies and embedding online transnational cross-campus teaching in the curriculum. Successful, it may become an annual event.

Alison Clapp, David Kennedy, Ruth Valentine, John Moss and Bhavani Veasuvalingam, FMS and NUMed

Using Canvas Commons to Supply and Support Student Learning Opportunities

What is Canvas Commons, how does it work, and why is it useful for disseminating online learning material? I will present examples of tutorials I have shared and induction material. By using Commons we can promote our material and the University to the wider Canvas Community. This will address the theme of Changing Practice through the Pandemic by focusing on the use of Canvas to provide excellent learning opportunities.

Michelle Miller, FMS TEL