Every challenge hides a deliberate mistake. Find it, explain it, fix it, and turn AI from a shortcut into a sparring partner.
Ask a model for a solution and you get one instantly, written with total assurance, correct most of the time. The habit that forms is acceptance. The skill that decays is the one school was supposed to build: noticing when something does not add up.
“Generative AI challenges traditional assessments by allowing students to produce correct answers without demonstrating understanding or reasoning.”A. S. Wazan, 2026
If certainty is the problem, take it away. Every passage here is written to look authoritative and contain one planted fault. You cannot tell which by skimming, only by working it through.
A model produces a passage with exactly one deliberate fault, drawn from a taxonomy of 23 error types across five subjects.
Highlight the span that is wrong, name the type of fault, and write what it should have said instead.
A separate call with a rubric prompt scores your answer, then reveals the ground truth so you can see what you missed.
Step three deserves detail, because a single score would hide the useful part. Splitting the mark into three axes means partial understanding still earns credit, and it shows you precisely which part of the skill is letting you down.
Did you find the right span?
Highlighting the text that actually carries the fault, not the part that merely looks complicated.
Do you know why it is wrong?
Naming the underlying fault. This is the axis most people lose marks on.
Can you state the correction?
Saying what it should have been. Spotting an error is not the same as repairing it.
Points shown for an advanced challenge. Intermediate pays 0.66 times that and beginner 0.33 times, so harder content is worth more and costs more.
| Average score | Verdict | Points | Why |
|---|---|---|---|
| 60 to 100% | Correct | +60 to +100 | You located it, explained it and fixed it. |
| 40 to 59% | Partial | +10 to +15 | Real understanding, incompletely expressed. |
| 30 to 39% | Weak | 0 to -2 | A genuine attempt that mostly missed. |
| Under 30% | Missed | down to -60 | Wrong span, or no diagnosis at all. |
There is a floor. One bad answer can never cost more than 70 points, so a single rough session cannot wipe out a month of work.
Once you know a mistake is coming, the obvious strategy is to flag anything unusual and hope. Three things are built in to make that strategy fail.
Roughly one in six contains no error at all. Claiming a fault that is not there costs you, so you have to actually read rather than assume.
An Elo style rating per error type tracks what beats you, then weights the next challenge toward exactly those faults.
A duel score is your average multiplied by how much you completed, so answering two questions perfectly still loses to eight answered well.
Critical reading improves with repetition, not with one good session. The same loop is wrapped three ways so it survives contact with a normal week.
One shared puzzle a day. Everyone gets the same one, so you can compare notes.
Race an opponent through identical questions on a synchronised clock, with points at stake.
Pick any subject, topic and difficulty, or type your own topic and let the engine build it.
The marking only means something if the marker is independent. The model that plants the mistake never grades your answer. Generation and grading are separate calls with separate prompts, so the grader cannot simply agree with itself.
You pick a subject
Five subjects, three difficulties, or any topic you type in.
Model A writes the flaw
One deliberate mistake, or none at all about 18% of the time.
Ground truth is sealed
The answer is held server side. The browser never receives it.
You answer
Span, error type and explanation are submitted together.
Model B marks it
A separate call with a rubric prompt, scoring all three axes independently.
The engine adapts
Your rating for that error type moves, changing what you see next.
FRONTEND
Next.js 16, React 19
App Router, KaTeX for mathematical notation, token based sessions.
BACKEND
FastAPI, SQLAlchemy
Multi provider LLM client, row level locking on duels, background expiry job.
DATA
PostgreSQL
Users, challenges, attempts, skill ratings and duel state.
Teaching with deliberately flawed AI output is an active research direction, not a gimmick we invented. Four papers shaped how this works, and every claim below links to its source so you can check it yourself.
A. S. Wazan, 2026
“uncertainty is a central pedagogical concept for stimulating students critical thinking”
Wazan argues for deliberately withholding certainty, including having AI generate plausible but flawed responses. That is exactly what a challenge here is. The answer looks authoritative, and you cannot tell whether it is sound without reasoning it through.
Read the paper on arXivH. Hosseini, 2026
“frequent errors and hallucinations, often seen as limitations, offer a unique pedagogical opportunity”
Hosseini reframes AI error as the teaching material rather than a defect to engineer away. We take that literally. The mistake is authored on purpose, and analysing it is the entire exercise.
Read the paper on arXivW. F. Lamberti et al., 2025
“students critically evaluate the accuracy and appropriateness of GAI-generated responses”
Lamberti and colleagues built classroom activities requiring students to analyse, critique and revise AI generated solutions. Our three part answer, locate then diagnose then fix, is that same loop turned into something you can repeat daily.
Read the paper on arXivS. Sonkar et al., 2025
“how can we verify if an AI truly understands how students think”
Sonkar and colleagues generate distractors conditioned on a student’s own misconceptions. Our adaptive engine runs the same idea in reverse. It tracks which error types beat you, then weights future challenges toward them.
Read the paper on arXivQuotations are taken verbatim from each paper’s abstract and are the authors’ own words. The commentary beneath each one is ours, describing how we applied the work. It is not a claim made by the authors, and no endorsement is implied.