Welcome back to our #TLF Blog. We hope that our previous posts have been helpful to you as you navigate through this contemporary teaching contexts. As the mid-term assessment phase of the semester comes to an end, we would like to explore assessments and its sometimes favourable, and other times questionable, relationship with artificial intelligence (AI). From course curricula/design, to the implementation of learning activities and creation of teaching and learning artifacts, the integration of AI in Higher Education has profoundly transformed the way things are done, created or managed. While many educators have been uncomfortable with the “threats” AI can create, some educators have begun to view AI in Education (AIEd) as less of a threat and more as a pedagogical instrument or resource as such tools provide real-time feedback and meet personal learning needs (Selwyn, 2021, and Luckin et al, 2016). But perhaps most notably, the revolutionizing impact of AIEd can be seen in the domain of educational assessments.

A – Assessments and The AI Factor

Retrieved from: https://www.zabala.eu/news/artificial-intelligence-and-consultancy/
 

The power of AI in educational assessments lies in the benefits it continues to provide. Responsible and ethical use of AI can assist in designing various formal (graded) or informal (ungraded) assessment activities, creating course curricula or lesson plans, and decoding large amounts of assessment data. For learners, it is often used as a guide that highlights various points or bits of information for consideration in answering questions or completing tasks. In such cases, AI is used as a resource. But while AI may provide a plethora of benefits, it is important that users are guided by ethical guidelines that ensure academic integrity, as failure to do so may result in AI becoming a threat to the aforementioned benefits it provides.

B – The Benefits and Challenges of AI in Assessments

When it comes to assessments, AI has certainly revolutionized how they are designed by educators and how they are done by students. The use of AI in educational assessments provides the following benefits:

•  Personalized Learning: Assessments are customized to individual student needs and learning styles.
• Efficiency: Reduced administrative burden for teachers and faster turnaround on feedback.
• Improved Insights: High-resolution data helps teachers understand student progress

But on the other hand, AI can also give way to a series of serious concerns or challenges. These include the following:

•  Equity: Risk of increasing the digital divide.
• Ethical Concerns: Increasing occurrence of AI abuse and undermining academic integrity.

One thing we know, AI is here and it may be here for quite a while! The future of assessment is not without AI and its influence. Nevertheless, the future of AI should also focus on authentic assessments, which require students’ skills such as classroom presentations and discussions/debates, ethical reasoning, creative problem-solving (solving complex problems), that AI cannot replicate. To that end, it may be best for you to specify to students the extent to which AI can be used in a particular assessment. The aforementioned illustration provides a guide or scale in this regard. In your course assessments, try to include a balanced mix of AI integration and specific tasks (authentic components) where it is not required.
 

C – AI and Online Assessments

When it comes to online assessments, conventional avenues include lengthy essay questions, multiple choice quizzes, short-answer questions and the like. However, AI has vastly improved the ways in which online assessments are designed, delivered, marked/graded and more. Whether it is through automated item generation and marking, to personalized assessments and malpractice detection, AI continues to revolutionize the way we teach and asses, and the way students learn. The following are some ways in which AI can benefit your online assessments:
 

1. Practical Assessments: This may serve as a formal or informal online assessment activity, where students engage in role-play scenarios with AI-powered bots. These interactive and immersive experiences are almost likened to students completing a problem-based/project-based, proficiency-based or authentic assessment, as the AI technology can easily replicate real-world situations for a more impactful learning (and assessment) experience. This can be adapted to assessments and tasks in most fields including language learning, consultancy and other scenarios that the learner may eventually encounter in their future careers. In so doing, it helps to prepare the current learner for the future!

2. Automated Grading: You may require your learners to submit written and/or spoken (verbal) responses to various assessment items/questions e.g. essays, SAQs etc. There are AI-powered tools, equipped with Natural Language Processing (NLP) models and/or speech and linguistic criteria, that can accurately analyze and assess learners’ written or verbal responses for marking/grading purposes. This automation allows for a more efficient, consistent and more objective grading process.

3. Assessment Analysis: AI can perform various statistical and psychometric analyses which may provide you with substantive insight to the learners’ performance. In addition to interpreting results, it can provide visualizations of the learners’ performance, and thus allow you to see any possible strengths and weakness in their learning and assessment submissions. Moreover, it will highlight areas for improvement, thus allowing you to take a more data-driven approach to subsequent lesson planning, revision activities and other courses of action to address and/or mitigate the identified learning gaps.

4. Plagiarism Detection: There is currently a range of AI marking tools that can now detect instances of plagiarism in learners’ written submissions (e.g. SAQ and essay responses). One example of this in our UWI context is the Turn-it-in platform. Accessible through all MyCaveHill eLearning courses, this is an activity tool that facilitates students’ submission of text-based content/assessments. Moreover, it has the ability to detect plagiarized content and with recent upgrades, it can now detect AI-generated content that the students may have unethically used in their submissions. Although this may not be 100% fail-proof in its detection, it is still an avenue you may consider to ensure that students use AI technologies responsibly and ethically.

5. Test Construction: AI can now modify the ways in which assessments are created. No longer do you have to create assessment items manually via a Word document. AI allows for a more efficient process in which question items are created automatically, and in the case of MCQs for example, it can also generate distractors. How does this work? Well, it is based on AI algorithms that analyze existing question banks, find patterns and create new questions that fit within certain rules or conditions. It also has the potential to ensure validity in assessments by examining the quality and reliability of questions that comprise the assessment. It is recommended, however, that human-based quality assurance be in place to further avoid any possible issues relating to assessment validity and the like – in fact, we recommend that AI should always be monitored/reviewed by human input. Nevertheless, AI allows you to create your assessments in less time, and provide you with more time to do other things in your busy schedules.

D – Additional Resources

We hope these points are useful to you. While AIEd can provide an array of benefits to teaching, learning and assessments, it still requires human input or intervention to ensure validity in your assessment items, and academic integrity in students’ assessment submissions. With continued developments in AI, it will continue to give birth to various innovations and transformations in teaching, learning and assessments. As such, the following is the UWI’s Official policy for Artificial Intelligence. It provides a much need framework for the integration of artificial intelligence into teaching, research, and operations while emphasizing ethics, equity, and academic integrity. Among other things, the policy mandates authorized AI use, and requires proper citation of AI-generated content (even in assessments) to avoid misconduct.  
 
The following resources may help you in capitalizing on the power of AI in your assessments:
 

1. The UWI’s AI Policy/Framework
2. Assessment in the Age of AI and Beyond – Archived Cross-Campus CETL Workshop

 
 
We hope the aforementioned tips and resources are helpful to you.
 
Should you need any further assistance, please feel free to contact us.