Tag: online assessment

Probity in Online Exams – Success at TEPARG for SDS and FMS TEL

This post highlights the work done by SDS and FMS TEL to support remote exams over the past few years. Successful techniques for keeping students informed and supported are discussed, including student voice.

Work done by the School of Dentistry and FMS TEL Team won first prize at the Trans-European Pedagogic Anatomical Research Group Hybrid Conference this year, for the presentation on ‘Adopting a flexible approach to professional anatomy spotter exams during COVID’. You can read the internal news item on Sharepoint.

This work was also subsequently presented at the Newcastle University Learning and Teaching Conference 2022. Newcastle Staff can view the poster here for an overview.

This work centres on how exams subject to oversight from professional bodies – in this case the General Dental Council – could be run with adequate probity when these could not be undertaken in person.

The first element of this is to run the exams ‘live’ rather than as a 24h format. This meant that teaching staff and professional staff could be on hand to resolve any technical difficulties or make invigilation decisions.

A variety of question designs were also thought through. The final format was a ‘stimulus’ question type on Canvas, allowing an image to be shown with answer options alongside. Now this question type can also be used with Inspera exams.

The support of students with SSPs was also a key consideration, and rest breaks were granted across the board when exams were very lengthy. Students were asked what they thought about the probity measures put in place, such as the use of an exam declaration, and checking responses which may have been copy-pasted. Overall, the response was positive.

A key element in the smooth running of these exams was the preparation offered to students beforehand, such as clear messaging via email and the opportunity to practice with the exam environment before undertaking their summative assessments. 99% of students agreed that they had been well-supported throughout the process and during the exams themselves via the Zoom exam hotline. Most calls were students double-checking their responses had been submitted, rather than having technical issues.

Learning from these exams has already been carried forward for digital exams running through the new Inspera exam system, and the confidence staff and students have in the procedure means that any future changes to circumstances will be much easier to navigate for these teams and cohorts.

Designing Effective Rubrics

A written summary of our training on using rubrics with links to the full webinar resources.

We have recently delivered some training for BNS (School of Biomedical, Nutritional and Sport Sciences) in collaboration with Rebecca Gill and Susan Barfield from LTDS. The two sessions covered Rubrics – both their design and how they can be implemented in Turnitin. You can access the training recordings and resources at the foot of this page.

Examples covered included:

  • a rubric with very few criteria and letter grading
  • a rubric with weighted criteria and bands
  • a very fine-grained rubric that awarded numerical points based on ten different criteria.

What are Rubrics for?

Rubrics can be used to evaluate assessments, whether you use a quantitative rubric to calculate marks, or a qualitative one with more wiggle room. Using a rubric makes it easier to identify strengths and weaknesses in a submission, and creates common framework and assessment language for staff and students to use. This in turn can help make learning expectations explicit to learners, and assist in the provision of effective feedback.

What is the best way?

There is no one way to design a perfect rubric, as assessments are very individual.

Before you begin you may want to consider how you can design your rubric to lessen the marking or feedback workload. Quantitative rubrics can reduce decision-making difficulties as this means you don’t need to consider what mark to give within a band. On the other hand, you may need this flexibility to use professional judgement. A detailed rubric with less wiggle room per descriptor also acts as detailed feedback for students, reducing the need for writing long additional comments, but also takes longer to design.

Descriptors

When writing descriptors, ensure that there is enough clear and objective difference between each band. You may find that aligning your descriptors with an external framework helps you write them. This is critical for secure marking, and is helpful for students receiving that feedback. Using positive language also helps make this feedback easier to digest, and allows students to see what they need to include to improve.

Rubric Workflow

When creating a rubric, you can follow this basic process. At every stage it is important to consult local assessment guidelines and discuss progress with your colleagues for constructive feedback.

  1. Determine your assessment criteria – ideally these should be aligned with the learning outcomes of the task.
  2. Consider the weighting of each element, if required – is presentation as important as content?
  3. Decide whether you will need defined marks or flexible ranges. This may be partly determined by your in-house guidelines.
  4. How do marks in various criteria interact with or depend upon one another? For example, if there is a very low mark in a content criterion, does that mean that the assessment can never be a pass?
  5. Try to write out individual descriptors – if you’re having difficulty discriminating between bands you may need to adjust your structure.
  6. Test your rubric against former or dummy submissions and adjust as necessary. Does it work for a lower level of mastery as well as a middle-scoring and high-scoring submission? If you had difficulty deciding between criteria, or discover a double-credit/penalty, you will need to adjust.

Technical Setup

Turnitin allows for the use of Grading Forms and Rubrics. You can watch how to implement these in the Using Turnitin video in the session resources below.

Turnitin grading forms can be created to assist with marking assignments, allowing you to add marks and feedback under various criteria. When using these forms, the highest mark entered will become the grade for the assignment. You can also use this without scoring to give feedback.

Turnitin rubrics allow for marking under multiple criteria and bands. You can have standard rubrics that calculate grades, or qualitative rubrics that do not include scoring. Custom rubrics can be used for more flexibility within a band.

An alternative to using Turnitin is to integrate a rubric into the assignment itself by using a coversheet. (see the ‘Effective Rubrics – Using Turnitin’ video at 28m25s, link in the Canvas below).

Resources

New Digital Exams System: Inspera

Inspera is the new Digital Exam System available at the University. Inspera offers a wide range of features covering a large variety of exam styles. Colleagues wishing to learn more about Inspera are strongly encouraged to attend the event below, and explore the online guidance available on the LTDS website.

Have you seen the latest updates on Inspera?

Inspera Assessment – Digital Exams Launch Events: 12 October, 9 and 10 November 2021

Come along to one of the online events for live demonstrations, the chance to speak with our Inspera colleagues and to find out more about uses for digital assessments.

Book a place

The first rollout will involve modules already using digital assessments.

Find out more

Digital Exams with Inspera Assessment webpages.  

Inspera Guidance Canvas course, simply click the link to self-enrol.  

LTDS blog post about Inspera

If you have any questions about the launch events or Inspera Assessment, please email digital.exams@newcastle.ac.uk.