ANALYSIS AI & EXAM FRAUD
Use of ChatGPT is exam fraud
The popular AI tool ChatGPT is also stirring up KU Leuven. It is forbidden to use the tool for graded assignments, unless a course's Toledo page explicitly allows it.
What is ChatGPT?
ChatGPT is currently one of the most popular examples of a Large Language Model (LLM).
'LLMs are a form of machine learning, where a model is learned to automatically compose text,' says Tim Van de Cruys, Professor of Computational Linguistics at KU Leuven.
'The model was trained with an enormous chunk of the internet. Based on that, it has learned to predict every next word in a text,' says Van de Cruys. This allows the model to answer questions in a fairly natural and realistic-looking way, as if you were chatting with someone. However, the model has only been trained with data until 2021, which means that ChatGPT cannot tell us anything about the past two years.
ChatGPT adds another layer of training with what it calls Reinforcement Learning with Human Feedback (RLHF). The model is encouraged to give specific answers through feedback it has received from human individuals. 'This way, they also try to avoid the model giving racist or sexist answers to questions,' says Van de Cruys.
OpenAI, the company behind ChatGPT, has made the programme available for free since its release in November 2022. There is also a paid version where, for 20 dollars a month, you get a number of extra benefits, such as unlimited access during peak hours and faster responses.
This article was written on ChatGPT3, and there have since been further developments with a new version of the program.
In recent months, the popularity and accessibility of artificial intelligence have suddenly skyrocketed. The American company OpenAI launched the generative model ChatGPT late last year. This language model allows free text generation based on a large corpus of existing texts. It does so by predicting the next word as best it can.
The technology was not universally well received. A school in New York even decided to ban the website on its campus. At KU Leuven, such a ban is not yet in place, although its use for papers and exams is still illegal. 'It's of the same order as letting someone else write a text for you,' said Vice Rector of Education Policy Tine Baelmans.
Exam fraud difficult to detect
There is already a framework around exam fraud that students should be aware of. That framework can be found in the Education and Examination Regulations (OER). According to the OER, students can be penalised if they are not transparent about their use of resources or commit irregularities.
Baelmans says that each semester a number of students per faculty are sanctioned for violating the rules of that article. 'In spite of that, these are only a few,' she clarifies. According to her, there may be more cases of fraud that go undetected.
'ChatGPT is even able to completely invent references'
Tim Van de Cruys, Professor of Computational Linguistics
According to academic integrity expert Gert Storms, research on dishonest students at theses has been done, and it states that most students caught did something dishonest during their research. 'But only a few have falsified data heavily,' he continued.
'The control mechanisms we have now do not provide certainty,' believes Professor of Computational Linguistics Tim Van de Cruys. According to him, they will have to be adapted to accommodate new forms of fraud. Tine Baelmans mentions that the use of a tool like ChatGPT is easier to detect than, say, involving a family member.
Baelmans assures us that the OER already covers the use of new technology such as ChatGPT. In fact, Article 84 states that 'any conduct is punishable where the student cannot be evaluated'. This is deliberately written so that it can include any kind of fraud. Van de Cruys confirms this.
In addition, the use of ChatGPT may cause a student to implicitly commit plagiarism. Indeed, the tool does not always manage to correctly reference sources or sometimes paraphrases passages of text without any reference. 'The tool is even capable of completely inventing references,' thus Van de Cruys. By blindly copying such pieces of text into a paper or thesis, the student commits plagiarism, which is a criminal offence.
'Using such a tool is always a conscious choice'
Tine Baelmans, Vice Rector of Educational Policy
This does not mean that every use is necessarily unethical. According to Storms, it depends on how and when in the process you use the tool. 'If you use it responsibly as you would deal with existing literature, you can hardly be against it,' he adds. Baelmans sees it differently: 'If your intention is to write something yourself, you mustn't use ChatGPT for that purpose.'
Special framework follows
Since this is a new technology, there were no special rules into force for the previous exam period. However, several faculties are in the process of developing a framework to provide professors with the knowledge and tools on how to handle ChatGPT.
At the Faculty of Arts, for example, discussions are already in full swing. The framework that will be decided will be presented to the Programme Committees, where it can be weighed up per course whether ChatGPT should be allowed or not. 'If it is explicitly mentioned on Toledo, then it falls within the rules of the OER,' Van de Cruys assures. On top of those faculty frameworks, great importance is also attached to this issue university-wide.
The possibility of establishing a clear limit from which use is problematic is still under discussion. After all, a tool like ChatGPT has several applications and can also be used to rewrite a self-written text. 'We now accept that students use a spelling corrector, thirty years ago they probably didn't think that was kosher either,' Storms said.
Since the use of ChatGPT falls under already existing rules around exam fraud, the same punitive measures also apply. A total of seven different types of punishments are possible, ranging from a zero on the exam in question to a ban on university enrolment.
A number of factors are taken into account when pronouncing these punitive measures. For example, whether the student knowingly committed exam fraud plays an important role. On that, Baelmans is clear: 'The use of such a tool is always a conscious choice.'
'A number of courses will have to adapt their evaluation technique'
Tine Baelmans, Vice Rector of Education Policy
This means that students who used ChatGPT during the previous exam period— if caught – could face severe penalties. By April, all cases will be collected for evaluation afterwards. Baelmans admits that a number of cases have already come to her attention.
Adjustment of evaluation needed
Now that ChatGPT can be used by students, which will in all likelihood be allowed for some subjects, the question arises as to how professors are going to adapt. According to Van de Cruys, professors will have to adjust their assignments based on how a generative model would adapt to it.
Baelmans also thinks that a number of courses will have to adjust their evaluation technique. 'Although there will certainly also be courses for which nothing will have to change,' she continues. There are several possibilities: 'An oral test, for example, but also having a text written on the spot during an exam can be a solution.'