Case Study: Virtual Oral Presentations as a summative assessment

How do oral presentations work for 100% online modules?

Presentations helps students put across an idea while expressing their personalities, which is hard to do in an essay.

Introduction

Oral presentations are a popular choice of assessment in the Faculty of Medical Sciences, especially in our e-Learning modules. Students are asked to submit a pre-recorded presentation to Canvas and the markers watch the presentations at a time and place that suits them.

Diarmuid Coughlan, module leader for ONC8028 Practical Health Economics for Cancer, has kindly agreed to walk us through how the Virtual Oral Presentation element works on his module.

The Assessment

This year we had 14 students on the module. We asked the students to create a 15 minute presentation using either Zoom, Panopto (Recap) or PowerPoint.

We informed the students right at the start of the module that an oral presentation was part of the assessment and 4 weeks into the module we provided a formative assessment. The formative assessment allowed students to familiarise themselves with their chosen software, gain experience talking to a camera and also get some limited feedback on their presentation skills.

The submissions are double marked by 2 markers. Marking is completed separately by each marker outside of Canvas, then markers meet to discuss which marks/comments would be entered into Canvas and made visible to each student.

The Set Up

We provided 2 submission points in Canvas:

Recording Submission Point:

This area was used for the marking. It was set up as Media Recording for MP4 uploads (max of 500 mb) with a Text Entry option for Panopto users (no size limit).

We allowed students to choose which technology they were most comfortable with and provided video and written instructions for Panopto and Zoom. PowerPoint instructions were added later as an option with links to guidance provided by Microsoft.

View of instructions in Canvas

We also provided some instructions so students could crop their recordings to comply with the 15 minute time limit.

You are limited by time so remember to edit your recording so it is no longer than 15 minutes. Instructions: Windows | Mac | Panopto

Slide Submission Point:

This area had a 0 point value. It was set up as a File upload area for students to submit their slides as .ppt or .pdf, this allowed us to get a turnitin plagiarism score for each presentation as well as a reference copy of the slides, should anything be unclear in the video recordings.

How did it go?

There was a lot of fear from students initially. We encouraged students to give it a go, informing them that we were not trying to trick them. We provided clear guidance on what we expected and provided a rubric with a breakdown of points, clearly showing only a small percentage of the grade would be based on their presentation style and delivery. The content of the presentation was the most important part!

The use of technology was varied:

As markers we also had to overcome our fears of technology.

PowerPoint is easier once you know how to access recordings (you have to download the file, then click start slideshow).

Sometimes the Panopto recordings were hard to find, especially if students had experience of using the technology in Blackboard and did not follow the Canvas instructions correctly.

What are your next steps?

  • We only provided grades with a short feedback comment last year, we plan to provide more extensive feedback going forward
  • We will add more video content into the module as examples of how to create engaging slides and showcase our presentation styles – hopefully leading by example
  • We would also like to provide examples of a good presentation vs a bad presentation

Probity in Online Exams – Success at TEPARG for SDS and FMS TEL

This post highlights the work done by SDS and FMS TEL to support remote exams over the past few years. Successful techniques for keeping students informed and supported are discussed, including student voice.

Work done by the School of Dentistry and FMS TEL Team won first prize at the Trans-European Pedagogic Anatomical Research Group Hybrid Conference this year, for the presentation on ‘Adopting a flexible approach to professional anatomy spotter exams during COVID’. You can read the internal news item on Sharepoint.

This work was also subsequently presented at the Newcastle University Learning and Teaching Conference 2022. Newcastle Staff can view the poster here for an overview.

This work centres on how exams subject to oversight from professional bodies – in this case the General Dental Council – could be run with adequate probity when these could not be undertaken in person.

The first element of this is to run the exams ‘live’ rather than as a 24h format. This meant that teaching staff and professional staff could be on hand to resolve any technical difficulties or make invigilation decisions.

A variety of question designs were also thought through. The final format was a ‘stimulus’ question type on Canvas, allowing an image to be shown with answer options alongside. Now this question type can also be used with Inspera exams.

The support of students with SSPs was also a key consideration, and rest breaks were granted across the board when exams were very lengthy. Students were asked what they thought about the probity measures put in place, such as the use of an exam declaration, and checking responses which may have been copy-pasted. Overall, the response was positive.

A key element in the smooth running of these exams was the preparation offered to students beforehand, such as clear messaging via email and the opportunity to practice with the exam environment before undertaking their summative assessments. 99% of students agreed that they had been well-supported throughout the process and during the exams themselves via the Zoom exam hotline. Most calls were students double-checking their responses had been submitted, rather than having technical issues.

Learning from these exams has already been carried forward for digital exams running through the new Inspera exam system, and the confidence staff and students have in the procedure means that any future changes to circumstances will be much easier to navigate for these teams and cohorts.

The Unessay – NULTConf

Dr Stephanie Holton presented her Unessay assessment task – what could you assess with an Unessay?

At the Learning and Teaching Conference Dr Stephanie Holton, along with two of her students, presented their experiences trialling a new approach to assessment – giving students freedom in how to present their learning, rather than setting a traditional essay task. This work was done in a module examining Ancient Greek texts.

What is an ‘unessay’ – and how exactly does it work? This talk explores the increasingly popular unessay as alternative assessment type, taking as a case study its implementation across several compulsory language modules in the School of History, Classics & Archaeology during 2020-21. Delivered by both the staff and students involved, it highlights the wide range of benefits – as well as the challenges – of diversifying assessment in some of our most traditional modules.

Dr Stephanie Holton

The full talk can be viewed here.

It was fantastic to see how the students were able to approach this new assessment methodology, and the outputs themselves were diverse, including models and digitally-created choose-your-own adventure books. The support provided included workshops and one-to-one sessions so that students knew what to expect, and were confident they were on the right track. While somewhat time-intensive in terms of support, the level of engagement and ownership the students felt around their creations was a clear benefit of this methodology.

Students were able to share their extra curricular creative skills, and the diversity of their approaches meant that they were prompted to explore their texts in new ways – for example researching clothing colours for their model characters that would be appropriate for the time. The need for these extra details prompted students to do more research around their text that they may not have otherwise done, broadening their subject knowledge.

These kinds of assessments open the door to students using and expanding their digital skills, even though this isn’t the focus of the assessment. The element of choice allows them to choose their own level of comfort with how they’d like to present their project – whether this is in a digital format or physical.

What could you assess with an Unessay?

Designing Effective Rubrics

A written summary of our training on using rubrics with links to the full webinar resources.

We have recently delivered some training for BNS (School of Biomedical, Nutritional and Sport Sciences) in collaboration with Rebecca Gill and Susan Barfield from LTDS. The two sessions covered Rubrics – both their design and how they can be implemented in Turnitin. You can access the training recordings and resources at the foot of this page.

Examples covered included:

  • a rubric with very few criteria and letter grading
  • a rubric with weighted criteria and bands
  • a very fine-grained rubric that awarded numerical points based on ten different criteria.

What are Rubrics for?

Rubrics can be used to evaluate assessments, whether you use a quantitative rubric to calculate marks, or a qualitative one with more wiggle room. Using a rubric makes it easier to identify strengths and weaknesses in a submission, and creates common framework and assessment language for staff and students to use. This in turn can help make learning expectations explicit to learners, and assist in the provision of effective feedback.

What is the best way?

There is no one way to design a perfect rubric, as assessments are very individual.

Before you begin you may want to consider how you can design your rubric to lessen the marking or feedback workload. Quantitative rubrics can reduce decision-making difficulties as this means you don’t need to consider what mark to give within a band. On the other hand, you may need this flexibility to use professional judgement. A detailed rubric with less wiggle room per descriptor also acts as detailed feedback for students, reducing the need for writing long additional comments, but also takes longer to design.

Descriptors

When writing descriptors, ensure that there is enough clear and objective difference between each band. You may find that aligning your descriptors with an external framework helps you write them. This is critical for secure marking, and is helpful for students receiving that feedback. Using positive language also helps make this feedback easier to digest, and allows students to see what they need to include to improve.

Rubric Workflow

When creating a rubric, you can follow this basic process. At every stage it is important to consult local assessment guidelines and discuss progress with your colleagues for constructive feedback.

  1. Determine your assessment criteria – ideally these should be aligned with the learning outcomes of the task.
  2. Consider the weighting of each element, if required – is presentation as important as content?
  3. Decide whether you will need defined marks or flexible ranges. This may be partly determined by your in-house guidelines.
  4. How do marks in various criteria interact with or depend upon one another? For example, if there is a very low mark in a content criterion, does that mean that the assessment can never be a pass?
  5. Try to write out individual descriptors – if you’re having difficulty discriminating between bands you may need to adjust your structure.
  6. Test your rubric against former or dummy submissions and adjust as necessary. Does it work for a lower level of mastery as well as a middle-scoring and high-scoring submission? If you had difficulty deciding between criteria, or discover a double-credit/penalty, you will need to adjust.

Technical Setup

Turnitin allows for the use of Grading Forms and Rubrics. You can watch how to implement these in the Using Turnitin video in the session resources below.

Turnitin grading forms can be created to assist with marking assignments, allowing you to add marks and feedback under various criteria. When using these forms, the highest mark entered will become the grade for the assignment. You can also use this without scoring to give feedback.

Turnitin rubrics allow for marking under multiple criteria and bands. You can have standard rubrics that calculate grades, or qualitative rubrics that do not include scoring. Custom rubrics can be used for more flexibility within a band.

An alternative to using Turnitin is to integrate a rubric into the assignment itself by using a coversheet. (see the ‘Effective Rubrics – Using Turnitin’ video at 28m25s, link in the Canvas below).

Resources

Authentic Assessment

Having a new assessment system (Inspera) offers the opportunity to redesign assessments, and there are many examples of authentic assessment already running throughout FMS.

What is Authentic Assessment?

Authentic assessment covers a wide range of assessment techniques, including setting tasks that students may be asked to undertake in their future careers. It also covers the setting of other purposeful tasks – assessments that have value beyond simply testing knowledge, such as reflective tasks.

During the FMS TEL conference, a session was run that covered several examples of authentic assessment. Examples came from the School of Dentistry, presented by Ruth Valentine and Chris Penlington, the School of Psychology’s Psychology of Religion module led by Patrick Rosenkranz, and the School of Biomedical Sciences, presented by Lindsey Ferrie. The resources can be accessed via the links below.

  • Video and Audio via Panopto directly.
  • Full set of resources on the FMS TEL Community on Canvas and the MLE.

Inspera Resources

Check out our previous posts on Inspera for more information, or read more on the LTDS website and Inspera’s corporate homepage.

Further Reading

Inspera Launch Events – Updates

Inspera is the new Digital Exam System available at the University. Inspera offers a wide range of features covering a large variety of exam styles. Colleagues wishing to learn more about Inspera are strongly encouraged to attend the event below, and explore the online guidance available on the LTDS website.

Further to our previous post, there have been some changes to the launch events as below.

  • 12 October 2021 10:00 AM – 11:30 AM UK, 5:00 PM-18:30 PM Malaysia (via Zoom)
  • 9 November 2021 2:00 PM – 3:30 PM UK (Ridley 2)
  • 10 November 2021 2:00 PM – 3:30 PM UK (NUBS)

Find out more about Inspera

Digital Exams with Inspera Assessment webpages.  

Inspera Guidance Canvas course, simply click the link to self-enrol.  

LTDS blog post about Inspera

If you have any questions about the launch events or Inspera Assessment, please email digital.exams@newcastle.ac.uk

New Digital Exams System: Inspera

Inspera is the new Digital Exam System available at the University. Inspera offers a wide range of features covering a large variety of exam styles. Colleagues wishing to learn more about Inspera are strongly encouraged to attend the event below, and explore the online guidance available on the LTDS website.

Have you seen the latest updates on Inspera?

Come along to one of the online events for live demonstrations, the chance to speak with our Inspera colleagues and to find out more about uses for digital assessments.

The first rollout will involve modules already using digital assessments.

Find out more

Digital Exams with Inspera Assessment webpages.  

Inspera Guidance Canvas course, simply click the link to self-enrol.  

LTDS blog post about Inspera

If you have any questions about the launch events or Inspera Assessment, please email digital.exams@newcastle.ac.uk

New Digital Exams Software: Inspera

Inspera Assessment was selected as our digital exam system following a rigorous procurement process, which began with requirements mapping workshops in February 2020, attended by over 60 academic and professional services staff.

In 2021/22 Inspera will be the standard tool for centrally supported digital exams, and will be used for all modules that have a present-in-person digital exam in MOFS (with the exception of any specialist digital exams that require Numbas).

Inspera will be available for additional new digital exams from 2022/23 onwards. There will be opportunities during the academic year to see demonstrations of the software and learn more about the new types of assessment it makes possible. To discuss a potential new Inspera digital exam please contact digital.exams@newcastle.ac.uk.

Further information on preparing for exams in 2021/22 and the types of assessment possible with Inspera is now available on the Inspera pages of the Learning and Teaching website. This also includes information on how to prepare students for Inspera digital exams and links to student facing resources on the Academic Skills Kit webpages.

Webinars and in person workshops are now open for bookings.

For any queries regarding Inspera please contact digital.exams@newcastle.ac.uk.