Smart Revise uses AI to mark longer answer questions that are set in a Task by the teacher, but it does not use the AI marking function for Advance mode. There are three main reasons for this.
Reliability
With Tasks, teachers have an opportunity to moderate the marking and consider the AI feedback before releasing it to students. Allowing Advance questions to be marked by AI would remove this human filter.
AI is still in its infancy and the way examiners expect candidates to answer questions is not necessarily something the large language model understands. Typical examples include:
- Some courses where "justify" does not mean considering both sides of an argument, it means giving reasons for one side of the argument.
- Some courses where "evaluate" does not mean considering factors that are outside the case study given but referring to the context provided.
- Some courses where "chained line of reasoning" following assessment objectives (AOs) outlined in the specification means that candidates must structure their answer in that way.
AI does not appreciate these nuances and sometimes tries to be helpful in explaining how an answer could be improved (even though it should only be justifying its mark awarded), and as a result could misdirect a student.
Pedagogy
The best preparation for examinations is to engage with the mark schemes yourself. You will be able to see typical answers that examiners would expect, together with how the mark schemes are structured. If you used AI to mark answers you would bypass what is probably the most important part of answering the question, that is evaluating the accuracy of your answer and understanding what additional knowledge or structure you could have applied. While AI can help with this it is not as good as a student.
Cost
We want Smart Revise to be as affordable as possible. Using AI for commercial applications is not free, there is a charge per "token" which you can think of as a syllable in a word. The longer the answer a student gives, and the more often they use Advance mode (which we want to encourage) the more tokens are used and the more it will cost to provide the service.