Category Archives: OLAF

Checking an OLAF test and settings using the ‘checkexam’ web page

As a member of staff who is running an OLAF exam, you will be sent an email containing the link to the exam check page when your exam is ready for you to check. The exam check page makes it easier to check OLAF exam questions and settings (randomisation and feedback to students). Exams will appear on this page if they have the word OLAF in the title (e.g. ‘BIO1001 23rd May 2017 OLAF’) and the Available from setting is for a date in the future.

Please read the notes below before using the page.

Notes:

  • If you click the links on the page you will need to log into Blackboard with your standard username and password.
  • If you believe the settings are wrong, please contact OLAF Admin so that we can make the changes for you and ensure they are not reset later incorrectly. Some of the options we use for exams may seem counter-intuitive. They may be set as they are for a reason.
  • The start and end times are deliberately set to be available early (30 minutes before the start) and become unavailable 45 minutes after the finish. This is to allow students to get logged in and ready to enter the exam password at the start time.The extra time allows for students entitled to extra time and empowers the invigilator to give extensions if the group or individual students experience technical issues. The invigilator tells the students when to stop.
  • The test is only visible to students during the Available from and Available to period. Even then, a password is required. The password is changed after the exam starts.

 Overview

The page consists of a table with data about the exam and links to Edit and View the test and settings. Click the image below to expand and see the explanations.

To access the page use the link that has been emailed to you by the OLAF team.

Checking settings:

  • Please check the Available from and Available to columns to check the dates are correct (and that the time is correct with half an hour before, and 45 minutes after the official start and end times)
  • Please check the columns for Randomise question and Scores are automatically released to students to make sure they are correct for your exam.
  • Please check the Title of the exam. This is what the students will see in the exam room. Please contact olafop@ncl.ac.uk if you think this will not be meaningful to them. This is particularly important if there are exams in similar subject areas taking place, or if the same is exam is happening under two module codes. Delays can occur if students click the wrong link.

To edit the test or see the test in it’s entirety, click Edit the test. Sign in with your standard (Blackboard) username and password.

To see the test as a student would see it, you need to know the password:

  1. Click View the test options, log in with your Blackboard username and scroll down the screen to find and note the password.
  2. Return to the exam check page (the link that has been emailed to you by the OLAF team) and click Check student view.
  3. Log in with your standard username and password.
  4. A screen will open showing the instructions for the exam (this includes general information about the exam, as well as any specific instructions that have been created).  Click Continue to move to the password screen.
  5. Enter the exam password you noted earlier, and click Submit.
  6. Work through the questions using the forward and backward arrows.

 

Technology Workshops for 2016/2017

We’re pleased to announce that we’ve totally overhauled our workshop booking system.

It will now be much easier to select the workshop or webinar you want, be it an introduction to Blackboard or ePortfolio or something more advanced.

You can then select a date, rather than scrolling through a long list!

We hope it will make it much easier to get started and even simpler to develop your skills across the technologies available at Newcastle.

We’ll also still be offering bespoke one-on-one sessions if you want to do something in particular with your teaching and need help or advice on how to achieve it.

Just get in touch.

Sessions will be available to get you started with all of our technologies including Blackboard, ePortfolio, Turnitin, using online assessment, ReCap and the University’s new responseware system, Ombea, which allows you to collect student responses to questions and check their progress in large learning environments through mobile devices and tablets.

There are also a range of more advanced workshops to make use of other features such as discussion boards or blogs on Blackboard, Personal Tutoring and Supervision Groups on ePortfolio or even Writing Effective Online Test Questions.

You can read more about the webinars and workshops which are available and book up on our website.

 

 

Amending or deleting a question from a test after it has taken place

We often get queries about whether it’s possible to amend or delete questions from a test in Blackboard when the test has already taken place.  The answer is yes – and if the test uses question types that have automatic grading Blackboard will automatically regrade the test to reflect the changes you make.

That said, there is a bug in Blackboard which on rare occasions can cause problems with students’ test attempts when you amend or delete a question.  To help safeguard against this we recommend that you take the steps below before you make any changes to a test after it has taken place.

Close the test and make sure that there are no attempts in progress

Ensure that the test is no longer available to students, so that there is no danger that they might be taking the test when you make the change.  To check this:

  1. Go to the Content Area where the test is deployed in the module, and hover over the name of the test.  Click on the dropdown arrow that appears, and select Edit test options:Edit test options
  2. EITHER under Test Availability make sure that for Make the link available the No option is selected:         Make link avail No
  3. OR/AND scroll down to the Display Until date, and make sure this is set to a date and time in the past (this can be 5 minutes from the current time):Display Until date & time
  4. Scroll to the bottom of the test options screen and click Submit to save any changes you have made.
  5. Open the Grade Centre by clicking on Control Panel > Grade Centre > Full Grade Centre.
  6. Find the column for the test, and scroll down to check whether there are any attempts in progress, shown with the ‘Attempt in progress’ icon of a blue clock:Attempt in progress icon
  7. If there are any attempts in progress submit them on behalf of the student if appropriate.  See ‘Attempt in progress’ grade centre icon for an explanation and instructions .

Export grade centre

  1. In the module, open the Full Grade Centre.  In menu bar at the top of the screen you’ll see the option Work Offline on the right hand side.  Click the down arrow and select Download:Work Offline
  2. In the Download Grades screen leave the default options selected, and click Submit.
  3. On the next screen click the Download button and save the file to an appropriate location (e.g. your H: drive or shared file space for your school).

Download test results

  1. In the Grade Centre find the column for the test, and click the dropdown arrow to the right of the column name.  Select Download Results from the menu:Download results
  2. In the Download Results screen leave the default options selected.  Click the button to download results, and again save to an appropriate location.

Note: both of these actions produce a .csv file, which you can open and amend in Excel as required.

Amend or delete a test question and regrade attempts

Once you have completed the steps above  you can go ahead and make the required changes to the test.

  1. Open the test canvas.  You can do this via the Control Panel > Tests, Surveys and Pools > Tests, then hover over the name of the exam and click the dropdown arrow.  Select the Edit Test option.
  2. Scroll to the question that you need to amend or delete, and hover over the title.  Click the dropdown arrow that appears:Edit question dropdown
  3. To amend a question select Edit from the dropdown menu.  Make the required changes.  Click Submit and Update Attempts.  Blackboard will display a warning message; click OK, and the test will automatically be regraded:amending question warning
  4. To delete a question select Delete and Regrade from the dropdown menu.  Blackboard will display a warning message.  Click OK, and the test will automatically be regraded:deleting question warning
  5. Once you have amended or deleted all of the required questions, check the Grade Centre column for the test to ensure that the attempts have been automatically graded as expected.

Blackboard module rollover for 2015/16

All 2015/16 modules will be available in Blackboard by Monday 27 July 2015.  There is no Blackboard downtime scheduled and there will be no changes to Blackboard for the new academic year.  However, you will notice some changes in the way that assessments are copied over from existing 2014/15 modules into the new 2015/16 modules.  These changes have been made in order to tidy up the grade centre, and to avoid potential issues with assessments being accidentally made available to students.

Blackboard tests are copied over, but not deployed

This applies to all Blackboard tests, including OLAF exams.  An existing test will not be available to students, and there will not be a column for the test in the grade centre.  As an instructor you can access the test to edit it, and you can deploy it to make it available again to the 2015/16 cohort.  The steps to do this are outlined below:

1. To view and edit an existing test: click on Course Tools in the Control Panel.  Select ‘Tests, Surveys and Pools’ from the drop down menu, then click on Tests.

tests 1

2. The tests area lists all tests that exist in the module.  Locate the test you wish to view/edit in the list (you can sort items in the list by Name and Date Last Edited).  Hover over the name of the test and click on the drop down arrow.  Click ‘Edit’ to open the test for editing.

test 2 find and edit

 

3. To deploy an existing test: go to the area in the module where you want to make the test available to students.  Click on the Assessment menu and select Test.

test 3 deploy

 

4. See the information on Blackboard Help for instructions on making the test available.  You may also find the help page on test options useful.

Turnitin assignments are copied over but hidden from students

It is not possible to reuse an existing Turnitin assignment, so you do not have an option to deploy an assignment again.  However, as an instructor you can view hidden assignments if you need to refer to them (for example to check the settings that you used for Turnitin assignments run in 2014/15).

Note that no student data is rolled forward from one year to the next, so a rolled over Turnitin assignment in a 2015/16 module will not contain any student work.  If you need to refer back to 2014/15 students’ work you will need to access this in the 2014/15 version of the module.

Useful Tips

Date Management Tool

If there is any content in a module which has Date/Time Restrictions set up for it, at the Blackboard rollover the content is copied and the date restrictions rolled forward by one calendar year (e.g. an item that was made available from 1 November 2014 will now be set to become available from 1 November 2015).

Blackboard has a Date Management Tool that allows you to easily amend Date/Time Restrictions for all content in a module in one place.  The tool can be used to amend all content simultaneously, or choose to amend dates/times for individual items.  This Blackboard Help page explains how to use the different options in the tool.

Content areas are hidden by default

When content is copied from an existing 2014/15 module into a 2015/16 module by default it is hidden from students at the top level.  This is done to allow instructors to update/amend content if required, before making it available to the new cohort of students.

Hidden content areas are identified by an icon of a white box with a black line through it, like this: hidden area

To make a content area available to students, hover over it in the menu, click the drop down arrow, and select ‘Show Link’:

show link

 

Remember that you can double check what students can see in a module using the student preview mode (explained in this post).

 

Last year’s ReCap recordings are not copied over

A reminder that ReCap recordings are not copied when modules are rolled over in Blackboard.  If the 2014/15 version of a module included a ReCap content area this will be copied into the 2015/16 module, but it will not contain any recordings until new ones are created during the module.

 

 

How could Respondus help you to run online exams and tests?

Learning and Teaching Development Service’s Rebecca Gill answers your questions about making this software work for you….

1) What is Respondus for?

Respondus is a piece of software that can be used with Blackboard to help set up online tests or exams. It is widely used by the Online Assessment and Feedback (OLAF) Service, and it can also be very useful to academic and support staff who are running their own formative or summative online tests.

2) How might staff be able to use it?

The software converts test questions written in Microsoft Word into Blackboard test questions. Respondus allows staff to quickly and easily create large sets of questions, to build exams or question pools in Blackboard. It works with a range of question types, including multiple choice, multiple answer, matching, jumbled sentence and fill in the blanks questions. Respondus can also be used to download exams from Blackboard with or without answers.

3) How might it benefit students?

Respondus makes it easy to create large sets of questions in Blackboard, which means that as well as building summative assessments, staff can more easily create practice tests for their students to use in revision. It also has the option to add feedback for correct and incorrect answers to each question, making it easier for staff to provide detailed feedback on exams – this is particularly useful for large group teaching, because the amount of time needed to add the feedback is the same whether you are setting an exam for 20 or 200 students.

4) How might it benefit staff?

Respondus can help save time when setting up Blackboard tests, and streamlines the process of deploying a test with required settings. The ability to download Blackboard tests to Word means that you can edit existing test questions in Word and easily reimport them to Blackboard. Once you have set up a series of test questions they can easily be edited and reused in future years.

The software can also be used to create a paper version of a test without answers, which is a helpful backup if any students have problems logging in when you are running an online test or exam. In addition some Schools use Respondus to download Blackboard tests so that they can easily be made available to their external examiners.

5) What will the workshops do?

The workshop starts by introducing Blackboard tests and question pools. You will then learn how to create exam questions in Microsoft Word using Respondus format, and will practice importing the questions into Respondus, and uploading them to Blackboard to create a test or question pool. The workshop will also cover using test settings to control availability, question display, and releasing scores and feedback to students. Finally you’ll practice downloading tests from Blackboard to create a copy in Word.

6) Who might/should be interested?

Any member of academic or administrative support staff who is responsible for creating Blackboard tests/exams, and would like to learn how Respondus can help. The workshops are suitable for anyone who has general experience of Blackboard.

Book a workshop on using Respondus to create Blackboard tests.

‘Attempt in Progress’ icon in grade centre

Why this happens:

After you have run a Blackboard test or an OLAF exam that is automatically marked, the students’ grades appear in the grade centre as soon as they press the ‘Save and Submit’ button at the end of the test.  If a student forgets to press the button, instead of their grade in the grade centre you will see an ‘attempt in progress’ icon of a blue clock.   An example of this is shown below:

Gradecentre attempt in progress

To override this you can submit the attempt on behalf of the student.

 

To submit a test attempt on behalf of a student:

1. Hover your mouse over the grade centre cell with the attempt in progress icon, and click on the dropdown arrow.  Click on ‘Attempt’, as shown below:

Gradecentre click on attempt

2. The student’s attempt will open.  Click on the arrow at the right of the ‘Test Information’ bar to expand it.

3. The Status will show ‘In Progress’.  Click on the ‘Submit Attempt’ button, circled in the example below:

Expand test info, submit attempt button

4. A popup will appear asking you to confirm that you want to submit the attempt on behalf of the student.  Click OK.

5. The Status will now show ‘Completed’.  Either click ‘Exit’ to return to the grade centre, or use the left and right arrows on either side of the student’s name near the top of the screen to scroll to the next student’s attempt.

Respondus – software for converting tests to and from Microsoft Word

About Respondus

We now have a site licence for Respondus. This allows you to:

  • Download tests to print (with and without an answer key)
  • Convert correctly formatted Word documents to online test format
  • Create tests offline
  • Copy tests between courses
  • Deployment to one or multiple courses
  • Download tests from publishers signed up to the test bank network

Installing and Activating Respondus

  1. Get in touch with your computing officer to arrange for installation. They should also provide the required account information and  password.
  2. When you first run Respondus, you will be prompted to enter the information.
  3. The info you need to get started is
    Institution Name:   Newcastle University
    Local Support Contact(s):  Rebecca Gill, e-mail: rebecca.gill@ncl.ac.uk; Marc Bennett, e-mail: marc.bennett@ncl.ac.uk
    The installation  password (which your IT officer should provide. We cannot publicise it through this blog)

Connecting to a server

The first time you download or upload questions, you will need to set up the connection with the server From the Publish wizard:

  1. Choose ‘Add new server’
  2. Select ‘Check for predefined settings
  3. Enter your Blackboard user id and password
  4. Do not check the box to ‘Test the connection‘ as this does not always work (even when the settings are correct)

If you have any problems, ensure that the box in the settings called ‘Login Page’ says ‘relogin’ rather than login at the end, eg “/webapps/logon/?action=relogin”

Help and Guides

Demonstration Movies – Short demonstration movies are available at: http://www.respondus.com/products/respondus/demos.shtml . A full-length training video is available at http://respondus.com/products/respondus/training.shtml . These are great resources for introducing and training instructors on Respondus 4.0.

Quick Start Guide – A Quick Start Guide for Respondus 4.0 is available for download from www.respondus.com/quickstart . This Quick Start Guide is 3 pages in length and contains the essential information for getting started with Respondus 4.0.

User Guide – If you prefer a printed user guide, go to http://www.respondus.com/products/userguide.shtml . These user guides are provided in MS Word format so that institutions can easily edit/add information that pertains to their institution.

Online Help – Online help is available from within the Respondus 4.0 application. Context-sensitive help is available by pressing the F1 key. Or, go to Help > Help Topics to access the main help system.

Respondus Test Bank Network –The test banks are in a ready-to-use format and can be accessed with Respondus 4.0 for free by instructors who adopt a participating textbook.  More information is available at http://www.respondus.com/testbank .

 

 

 

 

 

New Item analysis tool

See also the video guide from Blackboard on Demand

Item analysis tool Blackboard guide pages

Use this:

  • Immediately after an exam to highlight questions that need checking for errors
  • Before updating the exam for the following year to check for errors and assess how difficult your questions are and how well individual questions discriminate between high and low performers

To run an item analysis report:

  1. Go to the module and the Grade Centre (‘Tests‘ or ‘Full Grade Centre’).
  2. Find the column relating to your exam/test
  3. Click the column header and access the drop down menu to find ‘Item Analysis
  4. Select the test and click Run. It may take a while depending on the complexity of the exam, numbers of examinees and the load on the system. You will see a message telling you that the report is running. You can check back later.
  5. When the analysis has run, you will be able to select it from the list.

Interpreting the report

A summary at the top of the page shows you how many questions are good discriminators (distinguish between low and  high scoring students on the test overall). It also shows you the number of Easy, Medium and Hard questions.

Further down, you can see Discrimination and Difficulty scores for the individual questions. It also indicates questions that may need to be reviewed. See the key on the page.

Questions marked with a red dot have poor discrimination scores and/or easy or hard difficulty values. Click the blue question text to get more info on that individual question.

  • A higher discrimination score number suggests a question is a good discriminator. A low or negative score is a poor discriminator
  • The higher the difficulty score, the ‘easier’ the question was for the students taking the test

Click Edit Test to make a change to the question. If you change the correct answer, you can then ‘Re score’ the test. This will make the change for future tests as well.
See also the video guide from Blackboard on Demand

Item analysis tool Blackboard guide page

 

Will OLAF save me time?

The answer to this question is that it will depend on:

  • What  you are comparing OLAF to
  • How complex your exam is and how many students are there
  • How you manage the process in your school

What comparison are you using?

As a rule of thumb, OLAF will save lots of time compared to marking on paper. Even so, schools do need to put in some work up from to make sure things are correct, particularly the first year the system is used. There is an upfront investment in year 1, and there will be more time next year if the exam does not change significantly.

If you have used another automated marking process (eg the now-withdrawn Data-prep Service or an Optical Mark Reading system like Speedwell) you need to be prepared for the additional checking of the online version. In this case, some of the onus is moved from the admin team to the academic to check. Having said this, there may be means of managing the process to the benefit of all.

How complex is the exam?

An exam consisting of simple multiple choice questions will convert easily from Word – IF provided in the correct format.

Exams with images, tables, equations, special formatting and characters as well as varied question types will require more checking. It may be better to build the exam directly in Blackboard online. For revisions of a previous OLAF exam where small changes are required, it may be easier to  edit online. Alternatively Contact LTDS for a compatible Word document version to edit.

How many students do you have?

For low student number (less than 30), there is less advantage. It would be easier to mark by hand.

How will you manage the process in your school?

  • If setting an exam online the first time, particularly a complex one, please start thinking about the format in good time. Please give us some sample questions so that we can show you what they will look like online. This allows us to make adjustments in good time. We will need the final version a month before the exam if you want us to convert it for you.
  • Involve administrators in the logistics and checking, as you would for other exams. They can do some of the online checking to ensure things look correct.Subject knowledge will be needed for the final check.
  • If repeating an exam from the previous year with one or two changes it may be easier to edit the exam from the previous year online. You can do this yourself or tell us the changes.
  • If there are a few more changes to make, it may be better for us to download a Word version of last year’s exam for you to edit. The format we download will be easier for us to convert and will require less checking than if starting from scratch.
  • Over time, you will build up a bank of usable online questions from previous. The Dental School now use their bank of questions as a resource to build their tests. You can tag questions with topics, difficulty level etc. You can then use the ‘reuse question’ feature to create a new exam with the appropriate coverage/level.

Using Images

One powerful feature of the online tests is that they can include images both in the question and as answers. Students can zoom in to the images if they need to see them better (by pressing ‘ctrl’ and ‘+’ on the keyboard or with the mouse wheel.)

There are some things that you do need to consider when using images:

  • Please remember that more checking of the online version of the exam  may be required to ensure the images appear correctly
  • Screens may vary slightly in terms of contrast and resolution. Ensure that images are very clear, particularly where detail is important. Make sure that arrows or labels are suitable such that if the colour is slightly off on the monitor, or light is shining on the screen, the information can be ascertained. Use dark colours and reasonably thick lines if possible.
  • If you are supplying us with the paper for conversion from Word, please use the recommended format and include the image within the body of the question ie after the question number and before the first answer. For example:

1. What geometric term is represented by the image below?

a. a corner
*b. a cross-section
c. the circumference
d. the perimeter

  • Remember that the questions will be displayed ‘one at a time.’ Images that are required for more than one question will have to be repeated. Figure numbers are unnecessary.