Revolutionising Learning: AI and Group Work Unveil a New Approach to Reading Activities

Navigating through the extensive volume of reading material in certain modules can be a daunting task for students, often leaving them overwhelmed by the sheer magnitude of information. Recognising this challenge, the module leaders of ONC8017 took a pioneering approach to ease the burden on students. In a bold move towards innovation, they harnessed the power of artificial intelligence (AI) and embraced the collaborative spirit of group work to revolutionise the learning experience.

tablet showing research paper and a robot saying i can help with that
Image used in Discussion Board task

The Task

Article Allocation:

The first step involved compiling a comprehensive list of articles relevant to the module’s curriculum. Each article was carefully selected to contribute significantly to the students’ understanding of the subject matter. Subsequently, the articles were allocated to individual students, ensuring that each student had a unique piece of content to delve into. Students were asked to read and summarise their assigned article.

Student Autonomy:

To cater to diverse learning preferences, students were given the autonomy to choose their preferred approach in engaging with the assigned article. They could opt to read and summarise the content independently, a traditional method fostering individual comprehension and analysis. Alternatively, students had the option to choose an AI tool for summarisation, exploring the cutting-edge capabilities of technology to streamline the information extraction process.

Students who opted to use an AI tool were tasked with critiquing the summaries generated. This not only encouraged a deeper engagement with the material but also honed their analytical skills as they assessed the accuracy, coherence, and relevance of the AI-generated summaries.

Following consultations with the Newcastle University Library, we recommended the AI tools Scholarcy and TLDR This. However, students were able to choose any tool that best suited their preferences. The library, also provided valuable insights, including a copyright statement and links to AI Guidance, as well as the Uses and Limitations of AI.

If your allocated article is behind a sign in wall we kindly request that you do not upload or share this licensed material with third party AI tools

Copyright statement

Group Collaboration:

The students were asked to share their summaries to a discussion board and to look through the summaries posted by others. They could then identify which literature was most relevant to them and read the articles in depth themselves.

Recognising the significance of collaborative learning, the module leaders fostered a sense of community among students. Group discussions and collaborative sessions were encouraged, providing a platform for students to share insights, discuss varying perspectives, and collectively enhance their understanding of the subject matter. This collaborative element not only enriched the learning experience but also mirrored the collaborative environments students are likely to encounter in their future careers.

The Student Experience

40% used, 47% didnt use, 13% unable to use AI

53% of students opted for AI-assisted summarisation, showcasing a keen interest in exploring the capabilities of technology for academic purposes. This choice not only demonstrated a desire for efficiency but also provided students with valuable hands-on experience in harnessing AI tools for practical applications.

However, the practical application of AI tools had its challenges. 25% of students who chose AI encountered difficulties, with the tools unable to produce a summary at all.

tldr this 4 scholarcy 3 chat gpt 4 unknown 1

In their candid feedback, students highlighted both positive and negative aspects of their experiences. While some were impressed by the efficiency of AI tools, all students expressed concerns about gaps and missing details in the generated summaries. Specific instances of errors, omissions, and disjointed reading experiences were noted, revealing the practical limitations of relying solely on AI for complex tasks. The majority of students who opted for AI, eventually opted to manually summarise the articles anyway, indicating a less-than-ideal outcome from the AI tools.

The AI tool also provided a second longer summary. This summarised most sections of the paper individually, which was presented like a smaller version of the paper. There was still important information missing, which was clear from the disjointed reading experience. Even so, I was still quite impressed with how well the AI tool had summarised the vast amount of information in the original paper into something relatively usable. 

Student experience of Scolarcy

No inaccuracies were noted. Good summary of the epidemiology, although it seems that the AI summary has basically just been derived from the abstract of the article. A number of gaps were identified. 

Student experience of TLDR This

The article has been summarised into ten key points, but these are not detailed. For example. only one of the statistics provided in the article have been included in the AI summary.

Student experience of Chat GPT

Final Thoughts

These nuanced results underscore the importance of balancing technological innovation with practical considerations. While the incorporation of AI offered students valuable exposure to emerging technologies, the ultimate outcome indicated that, as of now, AI tools might not be the ultimate solution we were hoping for.

Despite the unexpected challenges encountered in the use of AI, this experiment has provided invaluable insights. Recognising the evolving nature of technology, we remain committed to maintaining the task, observing how AI technology progresses year after year and see if, as the technology advances, the dialogue from students changes.


This post was written with the assistance of AI tool, Chat GPT.

ERDP Project: Exploring AI

As we develop our understanding of AI and its capabilities, we are looking at how advancing technologies such as ChatGPT might assist colleagues and students with day to day tasks. FMS TEL team members Simon Cotterill, Gemma Mitchelson and Michael Hughes succeeded in securing ERDP funding to explore such possibilities.

Project aims:

  1. To enable staff and students to access contextualised and personal data via AI machine learning software
  2. To investigate a process for generating AI responses in a more ethical way 
  3. To improve the University’s understanding of AI machine learning in an HE context.

We are investigating use of contextualised data, formatted​ with natural language, optimized for A.I. For example, using a student’s programme and module information, their timetable data, and MOF information to assist the student in accessing key information more easily.​ Later this could be enhanced with richer information, such as programme/module study guides, VLE course information and other sources. via APIs. Likewise, a chatbot for staff could draw together University, Faculty and School-specific information.

An exciting new feature to be explored is that of agents (aka ‘Assistants’) and their ability to take on different functions and different personas; effectively acting as a small workforce to support, user needs. Up to now, most of us are familiar with having a conversation with a single agent, yet there is growing scope for multi-agent use. In the visual below you can see an Ai Agents overview from Chat Dev. By setting instructions and ’embedding’ information into the system users can encourage each agent to behave differently. For example, “You are a member of the Design team who will come up with simple ways to achieve a set goal”, “You are a CEO who will talk to the CTO on what steps should be taken to achieve X,Y and Z.”…

A picture showing agents positioned in various roles, lead by a virtual CEO.

image source: https://github.com/OpenBMB/ChatDev

The technologies are evolving very rapidly. At an “AI Sprint” in late November, the FMS TEL team were able to work with newly released features from OpenAI; these make the embedding of customised information and personalising Assistants (agents) much more accessible. These and other new features support the integration of AI features within other systems. As such, there is likely to be a proliferation of new AI products and plugins based on these features – and hopefully eventually within the systems supported by FMS TEL. All work in our project will be cross-referenced to our university principles on AI which can be accessed here:  Artificial Intelligence (AI) | Learning and Teaching @ Newcastle | Newcastle University (ncl.ac.uk)

There are challenges to consider; in particular those related to Data Protection, which we continue to review. There are also financial considerations when using external AI services like OpenAI or via Azure API, which are metered (pay according to use), rather than fixed price plans, which need investigating as part of our intended trial/pilot.

We are in the early stages of fact-finding but will be reaching out to FMS schools in the new year with an invite to workshops to share our proof of concept.

FMS AI Project

A drawing of two different halves of a brain left side is connected with electronic circuits representing logic and the right side full of 70s style paint drops representing creativity.

With the rise of Large Language Models (LLM) and their potential this year the FMS TEL Team have been successful in an application for funding. We are in the very early stages of planning out how we can integrate some of our services with a LLM whilst also maintaining security over the data.

We have looked at feedback from a recent survey and are taking on board ideas from colleagues and students, to help guide us through this exploratory work.

This is just the beginning and we will keep you updated on our progress.

Presentations powered by A.I – Gamma.app Review

Gamma.app is an A.I-based tool which generates presentations, documents, and webpages. It’s focus on presentations makes it of potential interest to those involved in teaching and learning. https://gamma.app

Quick Look:

In ‘guided mode’, I gave a title ‘history of Newcastle upon Tyne’, and Gamma provided a choice of templates and then generated a suggested presentation structure within a few seconds. It then generated a deck of 8 slides in about 1 minute. The slide deck included relevant images and could be exported as Powerpoint or PDF. Additionally, Gamma allows for the import of custom text, which it adapts and converts into a slide deck or document. The ‘AI editor’ provides options, such as “Suggest a professional theme”, “Give more detail”, “Give me a more exciting way to say this” etc.

The Gamma app in use showing chat interaction with AI and the 3 slides generated.
Cost:

Gamma currently (October 2023) has a three tier model:

  1. Free limited use – you get a one-off 400 ‘AI Credits’ (credits used each time you generate a document), exported slides and documents are branded
  2. ‘Plus’ – £78/year, 400 ‘AI Credits’ per month
  3. ‘Pro’ – £147/year, unlimted credits and extra features
Thoughts:

Gamma is a powerful tool which can quickly generate slide decks and documents which are ‘usable’ with little modification. With all the focus on the tools of the ‘big players’, such as Microsoft/Chat-GPT and Google, it is refreshing to see a tool from a seemingly independent company (though, like many other A.I. apps, it may well be using the back-end services of Chat-GPT ).

Of course, to use A.I. generated materials, it is important to have grounded subject knowledge and critically review and adapt outputs, to avoid mistakes. It is also important to carefully word the prompts which you provide to the A.I.; for example, a presentation generated for me by Gamma, about “Newcastle University”, included accurate information about the 19th century pre-cursors of the University of Newcastle upon Tyne, but then mentioned a merger with UCL in 2002, and included a photo of Newcastle University in Australia.

There are obvious plagiarism and academic integrity issues to consider. In common with most other A.I. apps, there is no acknowledgement of the source materials used in training of the A.I. As such it may be part-based on copyrighted materials and licenced content such as Wikipedia, which has an Attribution-Share-Alike licence. Likewise, the source of images aren’t acknowledged – though the ‘A.I. Editor’ does give the option of ‘all images’ (even if licencing unknown), ‘Free to use’ (which seem ‘loose’, by including sources which don’t generally display image licence information, such as Facebook and Twitter) and ‘Free to use commercially’ – and you can click through to the source of the image. The pricing model for Gamma is similar to that of other A.I. tools, all of which lead the universal problem of inequality of access, giving advantage to students from more well-off backgrounds. But these tools are widely available now, and this is the new reality that H.E. needs to adapt to.

Talking about ChatGPT

There has been a lot of discussion in the FMS TEL office about ChatGPT over the past couple of weeks. Michael and Eleanor have spent some time putting this post together to help our readers to learn a bit more about ChatGPT, and to catch up on the discourse around it in HE in general.

We used ChatGPT to generate an about page for the blog last week, which you may have seen. It provides a good example of an introduction for a website. It feels somewhat generic, it has gotten a few things wrong, and expresses itself a little strangely in places. At the same time, it’s a decent starting point to work from.

What is ChatGPT?

ChatGPT is one of many advanced Artificial Intelligence models which has been trained to understand inputs (e.g. questions) in natural language (hence ‘chat’) and answer them. Interacting with ChatGPT feels similar to interacting with a chatbot or virtual assistant on the surface, but the technologies are completely different.

How does Chat GPT work?

The data that ChatGPT draws from is an offline dataset which was created in 2021. The exact content of the dataset is not clear, however ChatGPT is able to formulate responses to a massive range of topics, so it is safe to assume the dataset is enormous, and most likely taken from public internet sites (i.e. not password-protected sites). Part of the training process involved feeding computer code into the model, which helped it learn logic and syntax – something that is present in natural languages in the form of grammar. Feedback was also given to the model to help it improve – this is called ‘supervised learning’.

Q: How were you trained?
A: I was trained by OpenAI using a deep learning technique called Transformer-based neural network architecture on a massive amount of text data sourced from the internet. The training process involved inputting large amounts of text data into the model and updating the model's parameters to minimize the difference between the model's predicted output and the actual text in the data. This process was repeated multiple times until the model learned to generate text that is similar to the input data. The end result is a model that can generate human-like text based on the input it receives.
Some of the language is somewhat technical, but the summary is quite clear!

While ChatGPT can produce extended responses drawing on its huge dataset, it doesn’t understand what it is producing – similar to a parrot repeating after you. It can mimic the sounds but has no true understanding of the meaning.

What are people saying about ChatGPT and University Education?

Assessment Security

With its ability to generate text that looks like a human wrote it, it is natural to be worried that students may use ChatGPT for assessed work. Many automated text editors and translators are already in this market, though they work in a different way. Tools like Word’s spellchecker and Grammarly can both be employed to boost writing skills – though these do not generate text. ChatGPT is different in this respect, and it is free, quick, and easy for anyone to use.

“…The big change is that this technology is wrapped up in a very nice interface where you can interact with it, almost like speaking to another human. So it makes it available to a lot of people.”

Dr Thomas Lancaster, Imperial College London in The Guardian

Assessment security has always been a concern, and as with any new technology, there will be ways we can adapt to its introduction. Some people are already writing apps to detect AI text, and OpenAI themselves are exploring watermarking their AI’s creations. Google Translate has long been a challenge for language teachers with its ability to generate translations, but a practiced eye can spot the deviations from a student’s usual style, or expected skill level.

Within HE, clear principles are already in place around plagiarism, essay mills and other academic misconduct, and institutions are updating their policies all the time. One area in which ChatGPT does not perform well is in the inclusion of references and citations – a cornerstone of academic integrity for many years.

Authentic assessment may be another element of the solution in some disciplines, and many institutions have been doing work in this area for some time, our own included. On the other hand for some disciplines, the ability to write structured text is a key learning outcome and is inherently an authentic way to assess.

Consider ChatGPT’s ability to summarise and change the style of the language it uses.

  • Could ChatGPT be used to generate lay summaries of research for participants in clinical trials?
  • Would it do as good a job as a clinician?
  • How much time could be saved by generating these automatically and then simply tweaking the text to comply with good practice?

The good practice would still need to be taught and assessed, but perhaps this is a process that will become standard practice in the workplace.

Digital skills, critical thinking and accessibility

Prompting AI is certainly a skill in itself – just as knowing what to ask Google to get your required answer. ChatGPT reacts well to certain prompts such as ‘let’s go step by step’ which tells it you’d like a list of steps. You can use that prompt to get a broad view of the task ahead. A clear example of this would be to get a structure for a business plan, or outline what to learn in a given subject. As a tool, ChatGPT can be helpful in collating information and combining it into an easily readable format. This means that time spent searching can instead be spent reading. At the same time, it is important to be conscious of the fact that ChatGPT does not understand what it is writing and does not check its sources for bias, or factual correctness.

ChatGPT can offer help to students with disabilities, or neurodivergent students who may find traditional learning settings more challenging. It can also parse spelling errors and non-standard English to produce its response, and tailor its response to a reading level if prompted correctly.

Conclusions

ChatGPT in its current free-to-use form prompts us to change how we think about many elements of HE. While naturally it creates concerns around assessment security, we have always been able to meet these challenges in the past by applying technical solutions, monitoring grades, and teaching academic integrity. Discussions are already ongoing in every institution on how to continue this work, with authentic assessment coming to the fore as a way of breaking our heavy reliance on the traditional essay.

As a source of student assistance, ChatGPT offers a wealth of tools to help students gather information and access it more easily. It is also a challenge for students’ critical thinking skills, just like the advent the internet or Wikipedia. It is well worth taking the time to familiarise oneself with the technology, and to explore how it may be applied in education, and in students’ future workplaces.

Resources

  • Try ChatGPT – you will need to make an account with OpenAI and possibly wait quite a while as the service is very busy.
  • Try DALL-E – this AI generates images based on your inputs.

Sources and Further Reading