Category Archives: Decisions

Other online assessment and feedback tools

There are a range of online assessment and feedback tools that may be better suited to your needs (and those of your students).

Group Work

  • WebPA allows students to assess each others contribution to group work. Contact OLAF Admin.

Essays

Turnitin now provides tools for giving students feedback and facilitating Peer feedback on written work in addition to plagiarism checking:

  • GradeMark – Mark essays and access feedback online. Includes reusable comments and marking Rubrics.
  • PeerMark – Allows students to score and review the work of their peers.

Maths, Statistics and other disciplines

NUMBAs – Whilst some mathematical testing is possible though Blackboard, NUMBAs, developed by the School of Mathematics and Statistics here at Newcastle, offers more sophisticated assessment. It is used by several schools across the institution. See also http://numbas.mathcentre.ac.uk/

OLAF Service level description and user roles

OLAF (Online Assessment and Feedback) is a University supported service to allow schools to set online examinations for students to sit under invigilated examination conditions.

The OLAF Service Level Description outlines:

  • The OLAF system for delivering secure online examinations (see section 1. Service Description and section 3. Service Provision)
  • The responsibilities for staff as customers using the service (see section 2. User’s role in running an exam in OLAF) and the targets for delivering the service (see section 4. Service Targets)

This service is available to all academic units. The service is for examinations where the mark counts towards a module or course final grade. If you want to set up tests in Blackboard where the grade does not count, please contact olaf.admin@ncl.ac.uk for advice.

Negative Marking

It is possible to use negative marking through Blackboard. Before deciding to use negative marking you need to:

  • Consider whether negative marking tests student’s strategy to risk taking rather than their knowledge of your subject. See Times Higher Education article.
  • Read up on the literature that suggests there may be a gender bias. Example study. 

If you decide you are going to use negative marking you should:

  • Ensure your students are fully aware of the marking and the consequences of guessing. Ideally, they will have experienced a negatively marked practice exam.
  • Understand that the negative marking is not part of our conversion process from a Word document through Respondus. We will work with you to set up the negative marking the first time you use it. You will need to allow us more time to set this up. We will ask you to then amend the online version of the exam in future years, rather than start from a Word document.

Randomised tests

On request, we can randomise the order in which the exam questions appear to students. This can be:

  • Completely randomised. Students would see the test in an entirely different order.
  • Randomised in sections. For example, there might be a section on Marx followed by a section on Hegel. Students would see the sections in the same order, but questions within each section in a different order

Randomising the test does prohibit certain post exam reporting, particularly the Attempt Statistics report if you randomise in blocks. It also makes checking the exam before it takes place trickier. You might need to allow extra set up time if using this option.

Will OLAF save me time?

The answer to this question is that it will depend on:

  • What  you are comparing OLAF to
  • How complex your exam is and how many students are there
  • How you manage the process in your school

What comparison are you using?

As a rule of thumb, OLAF will save lots of time compared to marking on paper. Even so, schools do need to put in some work up from to make sure things are correct, particularly the first year the system is used. There is an upfront investment in year 1, and there will be more time next year if the exam does not change significantly.

If you have used another automated marking process (eg the now-withdrawn Data-prep Service or an Optical Mark Reading system like Speedwell) you need to be prepared for the additional checking of the online version. In this case, some of the onus is moved from the admin team to the academic to check. Having said this, there may be means of managing the process to the benefit of all.

How complex is the exam?

An exam consisting of simple multiple choice questions will convert easily from Word – IF provided in the correct format.

Exams with images, tables, equations, special formatting and characters as well as varied question types will require more checking. It may be better to build the exam directly in Blackboard online. For revisions of a previous OLAF exam where small changes are required, it may be easier to  edit online. Alternatively Contact LTDS for a compatible Word document version to edit.

How many students do you have?

For low student number (less than 30), there is less advantage. It would be easier to mark by hand.

How will you manage the process in your school?

  • If setting an exam online the first time, particularly a complex one, please start thinking about the format in good time. Please give us some sample questions so that we can show you what they will look like online. This allows us to make adjustments in good time. We will need the final version a month before the exam if you want us to convert it for you.
  • Involve administrators in the logistics and checking, as you would for other exams. They can do some of the online checking to ensure things look correct.Subject knowledge will be needed for the final check.
  • If repeating an exam from the previous year with one or two changes it may be easier to edit the exam from the previous year online. You can do this yourself or tell us the changes.
  • If there are a few more changes to make, it may be better for us to download a Word version of last year’s exam for you to edit. The format we download will be easier for us to convert and will require less checking than if starting from scratch.
  • Over time, you will build up a bank of usable online questions from previous. The Dental School now use their bank of questions as a resource to build their tests. You can tag questions with topics, difficulty level etc. You can then use the ‘reuse question’ feature to create a new exam with the appropriate coverage/level.