Category Archives: OLAF

Technology Enhanced Learning Drop-ins

Academic and professional services colleagues: Have you got a question about a particular learning technology? Do you want to find out if technology can enhance an existing teaching and learning activity? Do you have a great idea and want to know if technology can help you deliver it?

There is no need to book, just turn up and our learning enhancement and technology advisers will be on hand to help you. No question is too small!

Come along on:
Wednesday 5 December
12:00-14:00
Courtyard entrance, Old Library Building

No need to book, just turn up on the day.

If you can’t make this one, don’t worry,  there will be more over the next few months at locations across campus. Further details will be published here.

If your questions relates to School (or Faculty) specific software we may not be able to help, but we hope we can signpost you to the correct support.

Any questions? Get in touch ltds@ncl.ac.uk

Format for conversion from a Word document

The Learning and Teaching Development Service uses software called Respondus to convert the exam and also to produce print versions of the exam.  Respondus is also now available for University staff to use, find out more here.

If you would like LTDS to convert an exam from Word, or if you are going to convert it yourself using Respondus, it needs to be in the format shown in the linked document. Respondus format. See simple example below.

Example Exam Question:

Title: Speed of Light. 

3) Who determined the exact speed of light?

a. Albert Einstein

*b. Albert Michelson

c. Thomas Edison

d. Guglielmo Marconi

NOTE: Respondus only converts the following question types:

  • multiple choice
  • true-false
  • essay – not currently supported in locked down exams
  • fill in the blank
  • jumbled sentence
  • ordering questions
  • matching
  • multiple answer questions

Respondus cannot convert calculated formula questions and has no support for Hot Spot questions. These types need to be created online. Hot spot questions will need to be added manually into the paper version of the exam. The other questions can be pulled down from Blackboard to Respondus for printing (with answers for external examiners to check and without answers as a backup for students).

If you are using a paper based exam that is currently very different to this, please get in touch with OLAF Admin in good time  and we can look at  how it will work.

How is an OLAF exam run?

  1. The Exams Office arrange invigilation and technical support.
  2. Students log into Blackboard using Respondus LockDown Browser and navigate to the exam and await further instructions.
  3. When the exam starts, students are instructed to turn over a printed slip to find a secret password that allows them to start the exam itself.
  4. Each question is displayed on a separate page.
  5. Students can move backwards and forwards through the exam using the next and previous buttons or the ‘Question Completion Status‘ bar.
  6. If students have any problems during the exam, the invigilators and computing officers assist.

Other online assessment and feedback tools

There are a range of online assessment and feedback tools that may be better suited to your needs (and those of your students).

Group Work

  • WebPA allows students to assess each others contribution to group work. Contact OLAF Admin.

Essays

Turnitin now provides tools for giving students feedback and facilitating Peer feedback on written work in addition to plagiarism checking:

  • Turnitin Feedback Studio – Mark essays and access feedback online. Includes reusable comments and marking Rubrics.
  • PeerMark – Allows students to score and review the work of their peers.

Maths, Statistics and other disciplines

NUMBAs – Whilst some mathematical testing is possible though Blackboard, NUMBAs, developed by the School of Mathematics and Statistics here at Newcastle, offers more sophisticated assessment. It is used by several schools across the institution. See also http://numbas.mathcentre.ac.uk/

Checking an OLAF test and settings using the ‘checkexam’ web page

As a member of staff who is running an OLAF exam, you will be sent an email containing the link to the exam check page when your exam is ready for you to check. The exam check page makes it easier to check OLAF exam questions and settings (randomisation and feedback to students). Exams will appear on this page if they have the word OLAF in the title (e.g. ‘BIO1001 23rd May 2017 OLAF’) and the Available from setting is for a date in the future.

Please read the notes below before using the page.

Notes:

  • If you click the links on the page you will need to log into Blackboard with your standard username and password.
  • If you believe the settings are wrong, please contact OLAF Admin so that we can make the changes for you and ensure they are not reset later incorrectly. Some of the options we use for exams may seem counter-intuitive. They may be set as they are for a reason.
  • The start and end times are deliberately set to be available early (30 minutes before the start) and become unavailable 60 minutes after the finish. This is to allow students to get logged in and ready to enter the exam password at the start time.The extra time allows for students entitled to extra time and empowers the invigilator to give extensions if the group or individual students experience technical issues. The invigilator tells the students when to stop.
  • The test is only visible to students during the Available from and Available to period. Even then, a password is required. The password is changed after the exam starts.

 Overview

The page consists of a table with data about the exam and links to Edit and View the test and settings. Click the image below to expand and see the explanations.

To access the page use the link that has been emailed to you by the OLAF team.

Checking settings:

  • Please check the Available from and Available to columns to check the dates are correct (and that the time is correct with half an hour before, and 45 minutes after the official start and end times)
  • Please check the columns for Randomise question and Scores are automatically released to students to make sure they are correct for your exam.
  • Please check the Title of the exam. This is what the students will see in the exam room. Please contact olafop@ncl.ac.uk if you think this will not be meaningful to them. This is particularly important if there are exams in similar subject areas taking place, or if the same is exam is happening under two module codes. Delays can occur if students click the wrong link.

To edit the test or see the test in it’s entirety, click Edit the test. Sign in with your standard (Blackboard) username and password.

To see the test as a student would see it, you need to know the password:

  1. Click View the test options, log in with your Blackboard username and scroll down the screen to find and note the password.
  2. Return to the exam check page (the link that has been emailed to you by the OLAF team) and click Check student view.
  3. Log in with your standard username and password.
  4. A screen will open showing the instructions for the exam (this includes general information about the exam, as well as any specific instructions that have been created).  Click Continue to move to the password screen.
  5. Enter the exam password you noted earlier, and click Submit.
  6. Work through the questions using the forward and backward arrows.

 

Regrading questions individually

If a question needs to be removed or the answer key changed and regraded for all students who have taken a test, you can do this by editing the test and regrading (separate post).

It is also possible to manually overwrite the automatic grades on a question for each student. This is particularly useful if you have used text entry questions (e.g. fill in the blank). For fill in the blank questions, you may just want to focus on the students who have submitted answers that are marked as incorrect and think about updating the test for the following year to account for different spellings etc.

To overwrite the automatic grade for a question:

  1. Go to the module/organisation in Blackboard and click Grade Centre, then Full Grade Centre under Control Panel. 
  2. Find the column for your exam/test. You may need to scroll to the right to find it.
  3. Use the drop down menu on the Column Header for your exam to access the Grade Questions feature.
  4. You will see a list of the questions. Click the response number on the right hand side to start grading that question.
  5. You can choose to regrade anonymously by ticking the box at the top left. Click Edit next to any given answer to make the change.

Changing the instructions students see in a test

You can add some instructions to a test to explain how it will be marked, how to enlarge text and how to manipulate image sizes etc.

Note: The instructions appear at the top of each question. Avoid having too much instruction as this will push the questions down the page.

How to add/edit instructions.

1. Either

Go to the course and the area where the test is deployed. Use the drop-down menu to the right of the test name to Edit the test; 

OR

Go to Course Tools under Control Panel, Tests, Test Canvas and click the drop-down to the right of the test name and click Edit).

2. Click the drop-down menu next to the test name at the top of the screen.

3. Scroll down to find the instructions box. Type the instructions and click Submit.

 

Student advice: Checking your score and viewing feedback

For OLAF Exams the scores are not released immediately after the exam.

Once the school decides to release the scores (and feedback if applicable) they should follow the steps below.

 

Student guide to accessing the score:

  1. Go to any computer with Internet access.
  2. Go to http://blackboard.ncl.ac.uk.
  3. Log in with your Newcastle University username and password.
  4. On the first screen that you come to after logging in (My Institution), click My Grades (under Tools)
  5. In My Grades, select the module for the exam you have taken.
  6. Find the row showing the grade for the exam. (If the test was an OLAF exam the exam name will include the word OLAF and the date the exam took place).
Image of My Grades screen
Click on the image to view full size

Student guide to viewing feedback:

In some cases, particularly practice tests, you can click the score and see more specific feedback.  Your tutor will let you know if this is available for your test.

  1. Click on the name of the test in the My Grades screen (see image above).
  2. In the View Attempts screen click on the grade
Image of View Attempts screen
Click on the image to view full size

Different types of feedback available are different for different tests.  Your feedback may include some or all of the following:

  • Your score for each question
  • Your answer, marked with a green tick if it is correct, or with a red cross if it is incorrect
  • The answer options, with a green tick next to the correct answer or answers
  • Feedback from the instructor about your answer

You can see examples of each of these types of feedback in the image below:

Image of example feedback types
Click on the image to view full size

 

Changing the password for an exam

For an official OLAF exam, the password is changed after half an hour. This means that if a student leaves the room, they cannot access the test with the original password (they need to talk to the invigilator).

If you are managing the test yourselves, e.g. for a resit or extraordinary sitting, you may wish to do this yourself. (Contact olaf.admin@ncl.ac.uk first if you need help setting up a copy of the exam).

To change the password:

1. Locate the test in the course.

2. Hover over the drop down menu to the right of the assignment name. Use the drop down arrow to the right of the test name to access the  ‘Edit Settings‘ option.

3. Scroll down to 2.Test Availability, the last item is the password (see screenshot below). Make the change and scroll down and click Submit.

 

 

Checking scores and post exam reports

Scores

To see the scores:

  1. Click ‘Grade Centre’ in the Control Panel (bottom left of the screen).
  2. Choose ‘Tests‘.
  3. Scroll to the right until you see the column for the exam. You can hover over the column headers to see the test’s full name.
  4. Scroll down to see individual student scores.
  5. Click a cell and click on the down arrow, choose attempts to see the individual student’s answers.

To save the scores to a spreadsheet (CSV file)

  1. From the Grade Centre, click Work offline (near top right) and then Download. 
  2. Choose selected column and then find the column for the exam form the drop-down list.
  3. Where it says delimiter type, choose Comma. 
  4. Scroll down and click Submit. 

Reporting

There are a number of reports built into Blackboard. Some do not work for exams with randomised question sets.

Attempts statistics

This report is helpful to check immediately after the exam to ensure the questions and answer key were correct. To see the Attempts Statistics report:

  1. Click the column header and then click the downward pointing arrow to bring up the drop down menu.
  2. Click Attempts statistics. 

Please note, this report may take a while to process.

The report will appear onscreen. Scroll down to see each questions, and the percentage of students that choose each answer. If you find that a large number of students have chosen the wrong answer, you might want to check the question is correct.  It is possible to re-score a question or test.  Get in touch with OLAF Admin if you need to do this.

Other settings/reports.

General statistics

  1. Click the column header and then click the downward pointing arrow to bring up the drop down menu
  2. Click Column statistics.

This will display score distributions and averages.

Setting the column to display a percentage instead of  a raw score

  1. Click the column header and then click the downward pointing arrow to bring up the drop down menu.
  2. Click Edit Column settings. 
  3. Change the primary display to percentage.
  4. You can also change the secondary display to score if you want to see both. The secondary display appears in brackets.

Item analysis

See this separate post for information about automated Item Analysis reports for Blackboard tests.