Our model
Smart Revise AI marking uses an adapted large language model provided by OpenAI that is fine tuned to the Smart Revise context.
Data protection
No personal data is sent to OpenAI. We only send the question, meta data about the question to assist the marking process, the anonymous answer provided by the student and a numeric key generated by the system to identfy the response when it is returned. There are no additional data protection implications when using AI marking.
Limitations
As OpenAI provides the automatic marking functionality and not a human, there are limitations that you should be aware of. While we continually evaluate the meta data that informs the marking process, we cannot guarantee that marking is accurate. Indeed, teachers often disagree, apply benefit of doubt and the examiner marking itself is not 100% accurate. Our aim is for AI marking to be as good as a teacher, but this is not always achievable because it is impossible to check the accuracy of every possible response a student might make.
Response time
Most tasks will be marked within a few minutes to hours, but we are limited by the throttling and reliability of the OpenAI servers. We cannot guarantee the time frame in which a task will be returned from OpenAI, so please do not expect instant responses.
A teacher can cancel automatic marking at any time and instead choose teacher, self or peer marking instead.
Feedback to students
AI provides a short justification for the mark awarded for each question. This has been extensively tested to check that it does not return inappropriate responses or can be tricked into providing inappropriate material/advice. However, we make no guarantees with this as because AI technology is still very new. Automatic marking should be used with teacher discretion. We advise moderating at least a sample of the responses returned from the active tasks. Please inform us of any issues you discover.
Helping us to improve the model
Teachers can disagree about 1 or 2 marks with exam marking having a degree of discretion and tolerance factor. Therefore, expect to disagree with AI over a mark here and there. However, if you discover marking is very inaccurate for a question/answer, please click/tap the thumbs down icon in the AI feedback panel. This will send a silent alert to us to look more closely at the meta data sent to AI for that question.
Don't forget the self and peer marking options
Automatic marking should not be used exclusively or seen as a replacement for self and peer marking. The latest research from John Hattie, Professor of education and Director of the Melbourne Education Research Institute at the University of Melbourne, Australia indicates that peer marking can make a significant difference to the learning outcomes for students.