Getting started with well…anything is often the hardest part of any project. It’s the same for reviewing your assessments and thinking how they can best work in a digital format. Whether that’s a simple migration from paper to digital input, or leveraging the opportunities that digital and online assessment afford you as an educator.

Assessment is sometimes seen as something related but external to the learning process. We firmly believe that integrating assessment into the learning experience design is essential for student success.

In this article, we’ll explore the best practices for effective digital and online assessment design that meet the needs of both educators and candidates. When combined, they create a powerful and cohesive strategy for seamless implementation.

We’ve broken these down into seven areas for consideration when designing your online assessment:

  1. Align with learning outcomes
  2. Identify audience requirements
  3. Choose appropriate assessment methods and question types
  4. Ensure validity and reliability
  5. Practice online assessment security and credibility
  6. Utilize technology effectively
  7. Trial your online assessment.

1. Align with Learning Outcomes

As educators know, modern pedagogy is much more focused on the learner and learner activity than on the teacher and the curriculum. While the inputs are important and make a difference, it’s widely accepted that what the learner does with that information and how they perform is more important than the content or the teaching methods. The outcome outweighs the means. And there is no such thing as passive learning.

So, if learner activity is the key element, why does assessment still seem to focus more on the content than on the course learning outcomes? Why do skills and attitudes often feel left out in the assessment process?

One possibility is that assessment is often constrained by available systems suited for knowledge testing, such as recognizing responses (tests) or producing written artifacts (assignments). Of course these formats can be and often are effectively used, but “true” assessment in real world situations feels missing from the equation.

Much like with traditional assessment, it’s still essential to apply a “backwards design” approach, as outlined by (Wiggins and McTighe, 1988, 2005) . By thinking first about the learning outcomes, we can then develop the rest of the elements. The backwards design approach follows a three step process:

 

Source: MIT Teaching and Learning Lab

It’s clear that the application of traditional assessment methods can be transposed to online assessment design here. By thinking of the desired results first, it becomes easier to define what acceptable evidence should be produced to demonstrate understanding and provide a better framework to design learning experiences and how to scaffold candidates through the process.

Backward design means that assessment is no longer a separate evaluation piece external to the learning process, but a core part that takes into account all elements of the learning experience and design towards achieving the learning outcomes.

With the data easily gained through online assessment, you can easily see where the gaps in learning may occur over a cohort or timeframe and develop the learning experiences accordingly (Martin and Ndoye 2016).

 

2. Identify Your Audience Requirements

Candidates come from all learning paths, backgrounds and skill sets. It’s important to include the learner’s perspective into the assessment design, offering appropriate scaffolding and accommodation.

In terms of student needs and expectations, elements such as their prior knowledge of the topic, disability status, veteran status, age, gender, work, family responsibilities can have an impact on the learner.

Using a digital assessment solution makes scaffolding the assessment process for diverse learners easier with added flexibility to include diverse learners. Creating meaningful assessments that can be tailored to accommodate student needs are viable options today that can be incorporated from the design stage, and not as an afterthought.

In terms of access, it is important to repeat that inclusion goes beyond accommodation of disability, while digital assessment opens the door for a lot of flexibility such as remote assessment taking with different devices (Burstein et al. 2022), knowledge of the target audience will also inform how to incorporate diverse learner needs and preferences in the assessment design.

 

3. Choose Appropriate Assessment Methods and Question Types

Different learning outcomes and assessment types, such as formative or open-book exams, require different methods. For instance, if we are considering a pure knowledge outcome, a closed book assessment can be deployed as an online assessment supported with a locked browser to prevent access to unauthorized resources.

More complex outcomes, as those related with application of knowledge or production, may require open book assessment or another authentic assessment format, and therefore, to make this suitable for online delivery, access to certain materials should be allowed and/or enforced.

Choosing an assessment method that is best suited to the desired learning outcomes goes beyond traditional methods. With online assessment, design takes on a new facet as alternative methods and question types become more readily available to assess consistently.

One way of doing so, for example, is combining traditional test questions with more visual question types, such as hotspot, drag-and-drop or even code-based questions that will specifically have an impact in STEM subjects.

Beyond this, more complex assessment elements may require that grades and feedback are more than just a number and a generic note. Rubrics offer a comprehensive guideline to expectations and can be shared with candidates ahead of time, bringing clarity about the grading process. Use of rubrics also means grades are more consistent and assessment criteria are maintained throughout the whole process Andrade (2005).

 

4. Ensure Validity and Reliability

Designing assessments that actually measure what they intend to measure and provide consistent results over time is crucial. A continuous improvement mindset must be adopted to ensure that assessment reflects achievement of the learning outcomes. This requires a hybrid approach with both quantitative and qualitative indicators (Haladyna and Rodriguez 2013).

On the quantitative side, results and trend analysis are important to define whether assessment is consistent and valid. One example is test item analysis, where variance is studied to ensure that questions are relevant. On the qualitative side, student feedback is a fundamental source of information about the clarity, practical elements and purpose of the assessment. This is used to understand how meaningful and authentic the assessment is for candidates.

 

5. Practice Online Assessment Security and Credibility

It’s also important to consider factors like the security and credibility of the assessment process. Online assessments can be conducted in a secure and reliable way, which is important, not just to the institution but also for the candidate putting effort towards achieving a certification and is looking for fair and authentic assessment.

One way of tackling these concerns is by incorporating online proctoring solutions to digital assessment. While there has been some conversation around the impacts, online proctoring is not essentially different from invigilating a face-to-face exam, provided that the right data protection safeguards are adopted.

In other words, if we accept that assessment can be done online and there are benefits to it, we should also accept that there has to be an adequate level of security surrounding the process. Proctoring is not just a way of monitoring, it can also perform:

  1. Student identity checks at the time of performing the assessment
  2. Setup, making sure that there are no unauthorized materials, behaviors or people present during the exam
  3. Support for candidates in the event of technical problems.

While proctoring tends to be used for high-stakes assessment, there are other simple ways to increase the integrity of digital assessment. For instance, applying time limits, or randomizing / polling the questions, which enables candidates to ensure papers are not identical. These are just a few examples of ways online assessment can be leveraged for more secure exams with integrity.

 

6. Utilize Technology Effectively

Choosing a user-friendly platform that meets security and accessibility standards underpins every other decision you make when it comes to designing an online assessment. A reliable online exam software should offer the necessary features to support various assessment methods while ensuring ease of use for both educators and students.

There are many online assessment providers to choose from. Research the capabilities and user experience of each, as well as the number and flexibility of question types to ensure you choose the provider appropriate for your requirements.

 

7. Trial Your Online Assessment

When adopting or improving any process, it is important to perform a careful review of the needs, current approach and desired outcomes. So, before fully implementing your online assessment, trialing your design is critical. This process helps identify potential issues and receive feedback from a small group of students, allowing you to make necessary adjustments. Effective pilot testing ensures a smoother rollout and better overall experience for all participants.

You can take this a step further with a pilot test for online assessment as a whole, covering all assessments for a trial period.

Final Thoughts

As you can see, many of these best practices apply in both traditional and online assessment. Essentially, your experience in teaching is your guide here. But taking this experience and applying it to digital assessment is a creative endeavor and can benefit from experimentation with your cohort and by knowing how they might best learn and demonstrate their understanding.

Some other things you should consider when planning your digital and online exams and assessments include:

The implementation of online assessment institution-wide is as complex as necessary, but that does not mean that you have to wait until you get everything right, otherwise you will probably never start!
Some form of digital assessment is likely already taking place at the institution. Try to analyze those practices in the light of the different elements we have discussed here, this will reveal actionable items for improvement.
Choose technologies that are purpose built over recycling current systems such as the LMS/VLE. Make sure to come up with a plan to pilot and deploy those online assessment solutions widely at the institution for maximum impact.

Finally, make sure to capture and analyze assessment data and feedback, that is the best way to refine and improve your institutional assessment strategy.

By following these best practices, you can create assessments that align with intended learning outcomes, meet the needs of your students, and maintain academic integrity.

 

Learn Why and How to Implement Online Assessment.