Generative Artificial Intelligence and its Role Within Teaching, Learning and Assessment

• understand the significance of generative AI for their studies and future careers.
• recognise appropriate, and inappropriate, uses of generative AI in supporting learning and assessment.
• appreciate the strengths and limitations of generative AI when used as part of the learning experience and in the context of the development of their graduate attributes.
• develop the skills to ethically and successfully utilise generative AI tools to support learning and in appraising their own educational gain.

• Alter text to clarify and/or develop the ideas, arguments, and explanations.
• Correct the accuracy of the information.
• Develop or change the ideas and arguments presented.
• Translate text into, or from, the language being studied.
• Or for the purpose of reducing the length of the submission so as to comply with a word limit requirement.

• Correcting spelling and punctation.
• Ensuring text follows the conventions of grammar and syntax in written English.
• Shortening long sentences or reducing long paragraphs without changes to the overall content.
• Ensuring the consistency, formatting and ordering of page numbers, headers and footers, and footnotes and endnotes.
• Improving the positioning of tables and figures and the clarity, grammar, spelling and punctuation of any text in table or figure legends.

We will continue to review the developments associated with generative AI detection software and may allow its future use.

• This should be detailed within the course outline and all assessment and assignment briefs.
• Students should also have the position verbally outlined during relevant teaching sessions, via relevant module-specific Canvas pages, and course handbooks.
• It should include a dedicated and well-signposted Canvas page outlining the nature and rationale for their use, and the extent of the allowable role of generative AI within each assessment and assignment.

• they are familiar within their limitations and associated ethical issues, and that these are discussed with students. Examples include: privacy and data considerations; potential for bias; inaccuracy and mis-representation of information; ethics codes; plagiarism; and, exploitation.
• they are familiar with the specific privacy policies or user agreements relating to their use. Students should be explicitly alerted to these policies whenever generative AI is to be used.

• this should be clearly articulated within those learning materials or assessments.
• academic staff are individually responsible for ensuring the factual accuracy and quality of any materials created using generative AI tools.

• whether the use of generative AI tools is permitted.
• how its use should be acknowledged by students.

• Requiring students to include a pre-defined statement that explicitly indicates whether or not they have used generative AI tools.
• Asking students to share prompts used, outputs or modifications.
• Requiring students to upload a reflective component detailing how generative AI has been used and their experience of engaging with it.
• Appropriate or enhanced referencing (see for example, APA style 7th Edition which includes guidance on referencing generative AI tools).

• Reflect how the use of generative AI is being assessed.
• Proportionately reward successful demonstration of the higher-order thinking skills of Bloom’s Taxonomy (see for example) which generative AI currently finds difficult to replicate.

• 25 July 2023: Generative AI and the student perspective.
• 23 May 2023: Generative AI: what does it mean for teaching, learning and assessment?
• 9 May 2023: How do we approach the impact of Generative AI in our institution?

How do generative AI technologies work?
Generative AI technologies, such as ChatGPT, can best be thought of as ‘conversation prediction’ tools, very much like the prediction tool first seen on a smartphone keyboard. However, unlike the early attempts of such systems to predict the next word in a sentence, which were only coherent within a few words, ChatGPT is able to consider words and phrases that were written much earlier within the text. This allows it the ability to maintain the context of the conversation for much longer. It is also trained using large amounts of data and broadly continues conversations in a way that matches the previous texts and conversations that it was trained upon.  

What can generative AI actually do?
Generative AI has the ability to create quite detailed written responses on a particular topic by combining information from multiple sources. The key difference here, however, is that rather than simply copying the text from the datasets verbatim, they combine elements of many related texts in different ways each time dependent upon previous user inputs, thereby creating responses that appear unique and mimic those a human might make in relation to similar prompts. 

Considering current developments in generative AI in the context of Bloom’s Taxonomy, and dependent upon the material on which they have been trained, ChatGPT for example is generally able to replicate the lower-order thinking skills in terms of the recall of facts and basic concepts (Remember), and in creating the impression of being able to explain ideas and concepts (Understanding).  

What are the current limitations of generative AI?
Generative Al models are only as good as the information they are trained upon. ChatGPT was trained using text databases from the internet including data obtained from books, Wikipedia and online articles. But is not connected to the internet, so it cannot train itself based upon new information or in real-time. Its most recent training data is from September 2021 and so it is operating on outdated data set which means it may not be able to provide accurate or up-to-date information on more recent events or developments. Generative AI can create variations on existing content, but will struggle to create accurate and realistic content when there is little or no existing information available. Another area where they can also struggle is in the repetition of facts or quotations, and in differentiating between accurate references and fake content. They may generate material that appears real at the surface, but upon careful scrutiny by an expert, it is instead clearly wrong. 

Again considering ChatGPT in the context of Bloom’s Taxonomy, it cannot replicate the higher-order thinking skills such as producing new or original work (Create), justifying a position, decision or argument (Evaluate), or drawing connections between different ideas (Analyse). 

In the short term, are there any ways that I can modify my assessments because I am concerned that generative AI technologies may impact upon student learning?

• Assess the higher order aspects of Bloom’s Taxonomy: Whilst students are completing an assignment, incorporate a reflective element that asks them to explain and justify (evaluate) their ideas and approaches. For example, why did they take the approach they did? What other options or approaches did they consider? Why did they not pursue them?
• Incorporate assessment tasks into the classroom: Rather than using in-class time for the delivery of new content, consider ‘flipping’ your approach so that students cover new content independently prior to the session. In-class time can be used to draft, develop or revise assessment tasks. This allows the opportunity for students to discuss with you, and their peers, their work and ideas and answer any questions that might arise.
• Apply a local, recent or personal context: Generative AI models are currently trained using a broad, but still limited dataset. ChatGPT is based upon a training dataset from before September 2021 and so may not be accurate with more recent developments. Framing assessments in terms of more recent or local events, for example using case studies, may be effective.
• Use generative AI to personalise assessments: Rather than setting as assessment question for students to answer, use generative AI to present an answer to a question and ask the students to evaluate and improve the response. For example, this might involve providing students AI generated computer code or a mathematical proof and asking them to identify any mistakes or areas where this might be simplified/enhanced. 

What are the longer-term ways that I can mitigate against the potential impacts of generative AI technologies upon student learning?

• Diversify assessment types: Some assessment types are more resistant to the effects of generative AI than others and can also help students develop, and evidence, their wider graduate attributes. Oral assessments, including assessed seminars and group discussions, might be appropriate and provide an opportunity for students to demonstrate their knowledge, understanding and even skills in persuasion to an examiner and/or their peers. Similarly videos and podcasts allow students to demonstrate their skills in communication. 
• Consider a group-based approach: Make assessment tasks collaborative with work taking place during teaching sessions. Randomly allocated groups, where there is a natural peer-led review of work, can minimise the use of generative AI by students.
• Staging assessment tasks: Stagger assessment tasks over multiple weeks or assignments so that students are required to submit these as a series of smaller components that when combined form a solution to a larger problem. Students can receive feedback on these smaller components, which they are then required to embed in future iterations, and it will allow you to observe the evolution of their work and ideas.
• Encourage the use of AI: Allow students the choice of whether or not to use AI within an assessment. For example, if a student uses generative AI to develop an essay, they can demonstrate, through tracked changes and comments how it has been developed and why. A similar approach can be used in mathematical or scientific disciplines where students are required to fully justify and explain their methods. Such an approach is also likely to help students better understand how they can engage with AI.
• Consider Bloom’s Taxonomy: Assessments that require students to evaluate, analyse, or apply what they have learned will limit their ability to pass AI generated work off as their own. Consider how this might be aligned with research-led teaching in your discipline. Similarly advanced-level projects, where students are required to create new knowledge are also more immune to AI generated content.
• Assess synoptically: Explore how an assessment might integrate multiple ideas, concepts or approaches, perhaps spanning several modules within your discipline. This will reduce assessment load and encourage students to investigate and articulate how different disciplinary ideas are connected.
• Require engagement with specific research-led literature: Require students to robustly cite external, and where appropriate, modern disciplinarily research. This will help students better appreciate the quality and applicability of literature sources, and will also help enhance their skills in critical thinking and analysis.