AI marking in Smart Revise

AI marking in Smart Revise

Our model

Smart Revise AI marking uses an adapted large language model provided by OpenAI that is fine tuned to the Smart Revise context.

Data protection

No personal data is sent to OpenAI. We only send the question, meta data about the question to assist the marking process, the anonymous answer provided by the student and a numeric key generated by the system to identfy the response when it is returned. There are no additional data protection implications when using AI marking.

Limitations

As OpenAI provides the automatic marking functionality and not a human, there are limitations that you should be aware of. While we continually evaluate the meta data that informs the marking process, we cannot guarantee that marking is accurate. Indeed, teachers often disagree, apply benefit of doubt and the examiner marking itself is not 100% accurate. Our aim is for AI marking to be as good as a teacher, but this is not always achievable because it is impossible to check the accuracy of every possible response a student might make.

Response time

Most tasks will be marked within a few minutes to hours, but we are limited by the throttling and reliability of the OpenAI servers. We cannot guarantee the time frame in which a task will be returned from OpenAI, so please do not expect instant responses.

A teacher can cancel automatic marking at any time and instead choose teacher, self or peer marking instead.

Feedback to students

AI provides a short justification for the mark awarded for each question. This has been extensively tested to check that it does not return inappropriate responses or can be tricked into providing inappropriate material/advice. However, we make no guarantees with this as because AI technology is still very new. Automatic marking should be used with teacher discretion. We advise moderating at least a sample of the responses returned from the active tasks. Please inform us of any issues you discover.

Helping us to improve the model

Teachers can disagree about 1 or 2 marks with exam marking having a degree of discretion and tolerance factor. Therefore, expect to disagree with AI over a mark here and there. However, if you discover marking is very inaccurate for a question/answer, please click/tap the thumbs down icon in the AI feedback panel. This will send a silent alert to us to look more closely at the meta data sent to AI for that question.

Don't forget the self and peer marking options

Automatic marking should not be used exclusively or seen as a replacement for self and peer marking. The latest research from John Hattie, Professor of education and Director of the Melbourne Education Research Institute at the University of Melbourne, Australia indicates that peer marking can make a significant difference to the learning outcomes for students.

    • Related Articles

    • Why doesn't Smart Revise use AI to mark Advance questions?

      Smart Revise uses AI to mark longer answer questions that are set in a Task by the teacher, but it does not use the AI marking function for Advance mode. There are three main reasons for this. Reliability With Tasks, teachers have an opportunity to ...
    • Whitelisting Smart Revise

      If you need to update your firewall to allow access to the Smart Revise website, please add the following domains. smartrevise.online *.smartrevise.online (currently the following subdomains in use are "www" and "cdn1" but this may vary over time.) ...
    • Try Smart Revise for free

      Smart Revise teacher accounts are free. Student accounts have access to some free topics in each course. So you don't need to contact us to arrange a trial, you can set up accounts and start using Smart Revise yourself now. We recommend teachers and ...
    • Is Smart Revise included with a Craig 'n' Dave membership?

      No, Smart Revise is not included with a Craig 'n' Dave membership. Smart Revise has significantly more overheads in terms of hosting and development. We cannot afford to include it in a membership unfortunately.
    • What devices can I access Smart Revise on?

      Smart Revise works on any internet connected device: desktop, laptop, tablet or smart phone. It is ideal for students to engage with anytime, anywhere. At home, at school, or whilst waiting for a bus!