Online assessment
Blackboard provides a range of tools to support online assessment:
Assessments: to create self-assessed or graded quizzes, and surveys
Assignments: to create assignments to be submitted, marked and managed online
Discussions: to create gradable online discussion topics such as private journals
Grade Book: allows you to manage grades and view student information
Grading forms: to use with assignments and graded discussions to allocate marks for specific criteria and levels of performance
Peer review: to enable students to grade and comment on each other's work, using either a simple rating scale or a grading form
Non-Blackboard DSO tools to support online assessment include:
Turnitin: to check student work for plagiarism
Respondus: to create quizzes, or convert quizzes created in Word, that can be published directly to your units in Blackboard
eLive: to enable students to make online audio-visual presentations and receive instant feedback; and also to support off-campus collaboration on assessment tasks.
Online assessment strategies
Assessing online can add diversity, efficiency and flexibility to your teaching/learning strategy, but it is not without its hazards. Technical issues such as unreliable or slow access and tricky submission procedures present challenges, as does dealing with the possibility for students to cheat, but these generally are not difficult to overcome. On the other hand, the advantages of assessing online include:
- instant, automated feedback
- group submissions
- automatic checks for plagiarism
- multimedia and web-based resources - in both questions and submissions
- delivery times of several days reduced to seconds.
DSO provides tools to support the assessment online of intellectual skills from the bottom to the top of the hierarchy identified by Bloom et al. (1964). That is, they can help you assess learning outcome types ranging from simple knowledge/comprehension through application and analysis to synthesis and evaluation. The development of generic skills may also be assessed and traced using online tools.
Knowledge and comprehension
Application and analysis
Synthesis
Evaluation
Generic skills
Further information
References
Knowledge and comprehension
Assessing students' knowledge and comprehension of specific facts such as definitions, dates, names, etc. is tedious and best automated. Blackboard's assessment tools are versatile and reliable for both formal, summative tests and informal self tests with feedback.
Summative assessment
The Blackboard quiz format will automatically mark questions and transfer these marks to the Blackboard grade book. Several types of questions are available, including multiple choice, true/false, fill in the gap and short answer. You can include formatting, pictures, web links, tables, etc. by using the built-in HTML creator. You can also provide feedback if you wish, and adjust the settings to allow one or more attempts. To increase the security of such tests, you can:
- set randomised questions
- release the quiz for a short period of time
- provide only one question per screen
- set a proctor code for an exam supervisor to enter
- limit access to specific computer numbers (eg only computers in a specific laboratory).
Formative assessment
The self test format provides the same range of question, formatting and feedback options as the quiz, but marks are not recorded in the grade book, so it is used purely for formative assessment - to let students know how well they have managed to remember the most important facts in a unit. The ability for students to self-assess whenever they feel ready, receive immediate, detailed feedback and repeat a test endlessly without penalty until they feel confident that they know the subject matter is seen as one of the major advantages of online assessment and online learning generally.
A Blackboard quiz with several different types of questions
Application and analysis
Quizzes and self tests
Universities have tended to use online assessment mainly to assess lower-level knowledge and comprehension skills. However, with a bit of thought it is possible to create quizzes and self tests that test students' ability to apply principles they are learning to different contexts (application); or sort fact from opinion and break information into parts (analysis). For example, you can create a quiz or self test that contains sets of multiple-choice questions based on short cases, or you can set short-answer questions (which will need to be manually marked, but this can also be done online).
Discussions
The Discussions tool can also be used to assess students' ability to apply and analyse important concepts. You can set up a threaded discussion, blog or journal to encourage students to share their thoughts on how certain principles could (or should) be applied in their own workplace, for example. Or you can get them to provide a short analysis of a website, newspaper article, television program or film of their choice that presents a particular point of view on a subject.
Even though you can provide formal marks for postings, which are transferred automatically to the grade book, for several reasons it is better to use discussions for feedback and formative assessment rather than summative assessment:
- Formally marking postings will generate boring discussions as students will focus on posting messages that meet your assessment criteria, rather than explore less conventional approaches, express honest opinions or engage their peers in conversation.
- Unless you set questions that require students to refer to their own context or experience, you may get multiple messages that are extremely similar, as it may be difficult for the later posters to find something different to say on the subject.
- Students may avoid giving too much away that may help their classmates (who suddenly become rivals) to good marks.
- Assessing students' discussions violates the safety of this learning space.
A better way to use the discussion tool to develop and assess application and analysis skills is to set a context- or case-related topic as described for quizzes and self tests above and ask students to post to this topic and respond to other postings there. The assessment task, however, is an essay or report that draws on a combination of students' own contributions to the discussion topic and those of others, all properly quoted and referenced.
Assignments
The Blackboard Assignment tool is very versatile and as such lends itself to assessing open-ended application and analysis-type learning outcomes. It allows you to set individual or group assignments that students can submit online as attachments in any software format you specify, eg Word, PowerPoint, Excel, Acrobat or HTML.
A Blackboard assignment
Assignments submitted through Blackboard appear in the unit instructors' Assignment Dropbox, where they can be marked online or downloaded, marked and uploaded again. Grading forms may be used with assignments to give feedback and marks on a pre-set, customisable range of criteria. Assignment marks are automatically transferred to the grade book and both marks and feedback can then be accessed by students immediately, using their My Grades or Assignments tools. It is also possible to make exemplary work available to other students by simply clicking a 'Publish' button in the Assignment Dropbox (with the creator's consent, of course).
A Blackboard grading form with customisable criteria
Turnitin
Plagiarism and collusion are increasingly common problems with assignments where students are required to locate and reproduce information on particular topics. Thankfully, they can be virtually 'designed out' of assignments that require students to use application and analysis skills to reflect on their own context. Turnitin is a DSO tool to identify text in electronic assignment submissions that is not original, and is used primarily to help detect plagiarism and collusion. It can also be a useful tool to help you assess a range of other aspects of your students' work. It can help you:
- check the accuracy of citations, which can help you judge the authoritativeness of the students' real sources for the level of research required
- identify the amount of text that is cited from various sources against the amount the student has written.
In this case study a Deakin University teacher shares her experiences and perspectives related to Turnitin, and talks of how her teaching was adapted accordingly.
For further information on Turnitin, including how to add a Turnitin assignment to your unit site, see the Turnitin guides. For further information on how to design assignments to minimise the temptation to plagiarise and collude, see the Institute of Teaching and Learning Plagiarism site.
A Turnitin originality report
Synthesis
'Synthesis' skills involve creating or blending elements in a different way to form something that is new and whole. Examples include designing a new model or program, or reconstructing an existing one for a new purpose. Blackboard's Assignment tool can be used to submit individual synthesis-type project work online, whether the attachment is a document, website, image, movie or spreadsheet.
Collaborative projects
Many synthesis projects are presented as collaborative problem-solving exercises, both to share the load of creating large-scale products and to introduce a realistic level of teamwork and communication into the learning outcomes. Blackboard's Group Manager can be used to set up virtual groups, to which you can then allocate private discussion topics and assignments submission areas. Drupal-SMF also provides a blogging and threaded discussion space that groups of students can use to discuss project work. MediaWiki provides an online space in which students can create documents collaboratively and edit each other's work.
More information on how you can use DSO to support your group work is contained in Online group work.
Authentic assessments
'Authentic' assessments involve collaborative activities that 'match as nearly as possible the real-world tasks of professionals in practice' and 'culminate in the creation of a whole product' (among many other characteristics) (Herrington 2002). For many educators, authentic assessments are the ultimate model for synthesis-type projects. Herrington describes several online tertiary courses designed around authentic web-based activities. She argues that it is necessary to match only the critical aspects of real-life locations and practices for the benefits of authentic learning to be available, and demonstrates that with thought and care it is highly likely that many authentic assessments could be created online.
MediaWiki and Drupal-SMF are being used increasingly at Deakin to create authentic learning environments. For more information and examples, see Authentic activities and learning environments.
Several Deakin units have made use of e-simulation technology to create an authentic learning and assessment environment. E-simulations involve expert programming, and are produced at Deakin by Knowledge Media Division. For more information, see the Authentic assessment (77 KB) page of the Institute of Teaching and Learning website or contact the ITL Support Service.
Evaluation
'Evaluation' skills involve judging the value of something, criticising, comparing and contrasting. The subject of the evaluation can be a model, theory or product, the work of peers or even a student's own development over the course of a unit.
Private journals
Private reflective journals (using a journal-type Discussion topic) are especially useful for getting students to reflect periodically on what they have learnt, what they are confused about or disagree with, how principles they have learnt may be applied in their own context, and what value they attach to things they have learnt. In a journal set up for 'private' access, students will see only their own postings, while those with an Instructor role in the unit will be able to both view and comment on all postings. This sets up an opportunity for you to provide informal, private coaching throughout the semester. If you wished to assess such a journal, rather than assessing a whole journal, which could discourage honest postings and take some time, you could ask students to write a final, summary posting for assessment. You could also ask them to self-assess their journal, based on certain criteria that you may prefer either to set or negotiate.
Peer review in public discussions
Peer and self assessment are almost essential for online group work, as discussed in the Online group work page. To assess one's peers and one's own contribution accurately can involve a good deal of reflection. Providing opportunities for students to practise assessing their peers and themselves can help them develop metacognitive skill (understanding about how and what they are learning). A procedure for online peer and self assessment in group work is described in Online group work.
You can also set up class-wide peer assessment using a Blackboard discussion topic set to 'public' access. Discussions that are open to the whole class present natural opportunities for formative peer assessment if students are encouraged to comment thoughtfully on each other's postings. You can also set up a grading system. The simple-scale peer review option will provide students with a simple rating and comments; or a grading form can be used to provide feedback on a customisable range of criteria.
For more general information on peer and self assessment, see the peer assessment page of the Institute of Teaching and Learning website.
A Blackboard discussion journal posting with peer review
Generic skills
Generic skills are skills that are not specific to particular disciplines, but are nevertheless important to students' success throughout their working lives. A list of these skills have been recognised as the Attributes of a Deakin Graduate, and faculties are obliged to progressively develop these attributes in students in their course curriculum design. They include such attributes as communication and information technology skills, personal organisation and management skills, collaborative skills, critical analysis, problem solving and creative thinking skills, ethical awareness and ability to work with a systematic body of knowledge. For more information on the attributes and the University's requirements, see the Attributes of a Deakin Graduate Procedure in The Guide.
As generic skills develop throughout a course, though not uniformly through all subjects, and as by their nature they are generally difficult to assess, the assessment of these skills can be problematic. The Developing and Assessing Graduate Attributes (912 KB) Guide (.pdf) provides a wealth of practical ideas, many of which can be implemented online using DSO tools.
Further information
For further information on online assessment, see the Australian Universities Teaching Committee website: Assessing learning in Australian universities.
For further general information on assessment, see the Institute of Teaching and Learning teaching and learning module Assessment.
References
Bloom, B.S., Mesia, B.B., Krathwohl, D.R. (1964), Taxonomy of educational objectives, David McKay, New York.
Herrington, J. (2002), 'Designing authentic activities in web-based courses' in M. Driscoll & T.C. Reeves (eds), Proceedings of e-learn 2002, Association for the Advancement of Computing in Education, Charlottesville, VA.
Vista and other Blackboard product names mentioned are either registered trademarks or trademarks of Blackboard Inc.
