AI That Predicts
Every Answer.
Agix partnered with Riiid Labs to build a deep knowledge tracing engine that predicts learner responses with 91% accuracy, delivering +224 TOEIC score points in 10 weeks, 58% less study time, and adaptive assessment intelligence that scales to millions of test-takers worldwide.
The No.1 AI-based EdTech company, built on a single belief that AI should know every learner better than they know themselves.
Riiid Labs is the research and AI division behind Riiid, the world's leading AI-powered test preparation platform. Known for their breakthrough performance on TOEIC preparation, where learners achieved an average score improvement of +224 points in just 10 weeks, Riiid Labs exists to push the boundaries of what AI can know about a learner's current knowledge state, and use that intelligence to optimize every next learning action.
Their core research focuses on Knowledge Tracing, modeling a learner's understanding of every concept across time, and Assessment Intelligence: predicting not just what a learner knows, but the precise probability they'll answer any given question correctly. Riiid approached Agix to build the production AI system that could deploy these models at scale, across millions of learner-question interactions, in real time.
Breakthrough research. No production system. And millions of learners who needed it yesterday.
Riiid Labs had developed some of the world's most sophisticated Knowledge Tracing models, deep learning architectures that tracked a learner's mastery across hundreds of concept nodes simultaneously. But these models lived in research environments. Deploying them at scale, handling millions of real-time learner interactions, across diverse test formats, with sub-second response times, required a production AI platform that didn't yet exist.
A five-stage AI pipeline, from raw learner data to personalized mastery.
Data Ingestion → Processing → AI Modules → Real-Time Delivery → Continuous Improvement, running end-to-end for every learner interaction, every session.
Six AI systems that power a platform capable of predicting any learner's next answer, before they give it.
Each component was engineered to deploy Riiid's research models in production, handling real-world data complexity, sub-second latency requirements, and the full diversity of learner behavior at scale.
Deep Knowledge Tracing Engine
The core deep learning model that tracks a learner's knowledge state across 300+ concept nodes simultaneously, updating after every single question response. Built on a transformer-based architecture trained on over 100 million learner-question interactions, the model maintains a real-time probabilistic estimate of mastery for every concept in the knowledge graph. Unlike classical IRT models that treat each question independently, the Deep KT engine models the sequential dependencies between learning events, understanding that a correct answer on concept B after struggling with concept A carries different signal than the same correct answer in isolation. This temporal modeling is what produces the 91% response prediction accuracy that drives the platform's efficiency gains.
Assessment Intelligence Model
A dedicated Item Response Theory layer that calibrates every question in the content library across three parameters: difficulty (the probability a median learner answers correctly), discrimination power (how well the question differentiates learners at the concept boundary), and guessing parameter (the probability of a correct response independent of knowledge). By separating question parameters from learner ability estimates, the model eliminates the systematic bias that occurs when hard questions are interpreted as evidence of low ability. The Assessment Profile module feeds directly into Response Correctness Prediction, producing per-question, per-learner probability scores that are the foundation of the Maximum Information Selection system.
Maximum Information Question Selection
The real-time selection engine that chooses the single most informative next question for each learner at each moment, using the combined output of the Knowledge Tracing model and the Assessment Intelligence Model to compute Fisher Information for every candidate question in the pool. The selected question is the one that would maximally reduce uncertainty about the learner's true ability, neither too easy (provides no information about the upper ability boundary) nor too hard (provides no information about the lower boundary). This Maximum Information Selection approach is what compresses study time by 58%: every question is chosen for maximum diagnostic value, eliminating the time wasted on questions that wouldn't change the model's estimate of the learner's state.
Response Correctness Prediction
The prediction layer that outputs, for every (learner, question) pair, a probability score representing the likelihood of a correct response, along with a confidence score and the top features driving that prediction. This layer is the convergence point of the Knowledge Tracing model and the Assessment Profile: it takes the learner's current concept mastery estimates and the question's calibrated parameters and produces a single actionable probability. At 91% accuracy, this prediction is precise enough to select questions at exactly the right difficulty level, and to detect, in advance, which questions a learner is likely to answer incorrectly before they attempt them, enabling pre-emptive review rather than post-failure remediation.
Real-Time Learning Analytics & Feedback
A live analytics layer that surfaces the learner's knowledge state in an interpretable format, concept mastery bars, predicted score range, weakest concept clusters, and time-to-target projections, updating after every question. Unlike aggregate score reporting, this layer exposes the underlying knowledge model directly to the learner: rather than a single number, learners see which concepts are secure, which are fragile, and which are completely unmapped. This transparency transforms engagement, learners understand what they're practicing and why, which increases session frequency and reduces abandonment when encountering difficult material. Feedback explanations are generated per error type: conceptual, structural, and vocabulary errors receive differentiated responses mapped to the specific knowledge component the question was testing.
Continuous Model Improvement Pipeline
A human-in-the-loop retraining infrastructure that continuously improves every model component as new learner-question interactions accumulate. The pipeline runs automated A/B experiments across question selection strategies, feedback approaches, and knowledge graph structures, measuring impact against downstream outcome metrics (score improvement, study time efficiency) rather than model accuracy metrics alone. Expert review gates surface edge cases, unusual learner trajectories, newly introduced question types, domain drift in test content, for human evaluation before they propagate into model updates. The Feature Store maintains processed behavioral signals that can be reused across model versions, reducing retraining time from weeks to hours as the content library and learner population grow.
Three stages from knowledge tracing to response prediction.
Knowledge Tracing Model → Assessment Profile → Response Correctness Prediction. Each stage builds on the last to produce the probability that any learner answers any question correctly.
What happened when every question was chosen by AI.
Measured across learners completing TOEIC preparation with the full AI Assessment Intelligence system deployed.
Average score gain in 10 weeks vs. +70 points with traditional prep, a 3.2× improvement in score velocity
42 hours to target score vs. 100+ hours with conventional test prep, driven by Maximum Information question selection
The Knowledge Tracing + Assessment Intelligence model correctly predicts whether a learner will answer a question correctly
Learners using the adaptive system exceeded their target TOEIC score at 40% higher rates than the control cohort
Concept-level mastery modeled simultaneously per learner, the most granular knowledge state in commercial EdTech
From answer submission to next question delivered, including knowledge state update and question selection
What Agix built wasn't just a deployment of our research, it was a transformation of it. The production system they delivered hardened our Knowledge Tracing models against every edge case we'd never seen in the lab, and added the Maximum Information Selection layer we'd theorized but never had the engineering bandwidth to build. The result is a platform that genuinely knows what a learner needs next before they do.
Three engineering decisions that separated this system from every other adaptive platform.
Separating question parameters from learner ability
Most platforms interpret a wrong answer as evidence of low ability. The Riiid system separates the two, modeling each question's difficulty and discrimination power independently through the Assessment Intelligence Model, so a learner's knowledge estimate is never contaminated by a poorly-calibrated question. This architectural decision is what makes the 91% prediction accuracy possible: without clean separation, the knowledge state model systematically under- or over-estimates ability depending on the question pool composition.
Maximum information, not maximum difficulty
Traditional adaptive platforms increase difficulty as a learner gets questions right, a simple heuristic that wastes enormous amounts of study time. The Riiid engine selects questions based on Fisher Information: which question would most reduce uncertainty about the learner's true ability across all concept nodes simultaneously. A question that's 60% likely to be answered correctly carries more information than one that's 95% or 20% likely. This single selection principle is responsible for the 58% reduction in study time, every practice session contains only high-information questions, with zero wasted repetition.
Production hardening before performance optimization
Riiid's research models achieved high accuracy on benchmark datasets, but the first engineering priority was production hardening, not performance optimization. We built robust handling for cold-start learners, data sparsity, session interruptions, and concept graph gaps before touching the model architecture. This sequencing is counterintuitive, most teams optimize first, but it meant that when we did apply performance improvements, they compounded on a stable foundation rather than masking instability. The result: production accuracy that exceeded lab benchmarks rather than falling short of them.
Is AI Assessment Intelligence the right build for your test prep platform?
What powers this system.
Deep Knowledge Tracing Models
Transformer-based Knowledge Tracing architectures that model learner mastery across hundreds of concept nodes, tracking knowledge state as a function of the full interaction history, not just the last answer.
Item Response Theory & Assessment Calibration
IRT modeling that calibrates every question across difficulty, discrimination power, and guessing parameters, separating question quality from learner ability to produce unbiased knowledge state estimates.
Maximum Information Question Selection
Fisher Information-based selection engines that choose the single most informative next question for each learner at each moment, the algorithmic core behind 58% study time reduction.
Response Correctness Prediction
Per-(learner, question) probability scoring systems that output correctness likelihood, confidence scores, and top prediction features, enabling pre-emptive difficulty adjustment and targeted review before failure.
Differentiated Feedback Systems
Error-type classification engines that distinguish conceptual, structural, and computational mistakes, delivering targeted explanations mapped to specific knowledge components rather than generic "incorrect" responses.
Test Prep AI Systems
End-to-end AI for standardized test preparation, knowledge tracing, adaptive question selection, score prediction, and study plan optimization, built to maximize score improvement per hour of practice.
Common questions about building AI Assessment Intelligence systems.
A mastery score aggregates a learner's history into a single number, typically percentage correct or a weighted average of recent performance. Knowledge Tracing maintains a separate probability distribution for every concept node, updated after every interaction, tracking not just whether the learner got questions right but the temporal pattern of their responses, how quickly they're improving, whether performance is stable or variable, which concepts are prerequisites for others. This granularity is what enables Maximum Information Selection: you can't select the most informative next question without knowing the learner's current state at concept resolution, and a single mastery score provides no concept-level signal.
Useful models can be trained with as few as 50,000 learner-question interactions, enough to establish reasonable difficulty calibration and concept relationships. Models trained on 1–10 million interactions achieve the accuracy thresholds required for production Maximum Information Selection. For platforms without historical data, we use transfer learning from similar-domain datasets and run an initial calibration phase where the platform collects diagnostic interactions before personalizing, typically 10–20 questions per learner to establish a baseline knowledge state. Cold-start performance is managed by defaulting to Information-Theoretic selection based on population-level statistics until sufficient individual data accumulates, usually within the first 2–3 sessions.
Yes, the AI Assessment Intelligence system is domain-agnostic. The same Knowledge Tracing and Maximum Information Selection architecture has been applied to USMLE Step 1 preparation, CFA exam prep, bar exam review, and AWS/GCP certification practice. The domain-specific work is knowledge graph construction (identifying concepts, prerequisite relationships, and concept-question mappings) and content calibration (running IRT parameter estimation on the question library). These are one-time investments per domain that we complete during the discovery and design phases. The mathematical machinery, transformer KT model, Fisher Information selection, response prediction, transfers directly across exam types and content domains.
Retaken questions are handled explicitly in the Knowledge Tracing model, a second-attempt correct response on a previously failed question is weighted differently than a first-attempt correct response, because it signals consolidation rather than initial acquisition. The model tracks the recency and spacing of exposures to each concept, incorporating spaced repetition signals into the knowledge state estimate. For learners preparing for multiple exams with overlapping content (TOEIC and TOEFL, for example, share vocabulary and grammar concept nodes), the system maintains a shared knowledge graph that transfers mastery estimates across exam contexts, so a learner who has demonstrated strong vocabulary mastery in one exam preparation context doesn't re-learn those concepts from scratch when starting preparation for a related exam.
The 91% accuracy is measured as AUC (Area Under the ROC Curve) on held-out learner-question pairs, the same metric used in the Riiid AI for Education Kaggle competition, which Riiid Labs won with a score of 0.817 on a public benchmark. Our production system was validated on 3 months of live interaction data, held out from training, using the same AUC metric with a threshold of ≥0.75 for production deployment. The 91% figure reflects the final model trained on the full production dataset, the model averaged 0.88 during the initial calibration period and improved to 0.91 as more learner-question interaction history accumulated. We publish per-concept accuracy breakdowns as part of the model monitoring dashboard delivered to the Riiid Labs team.
Ready to build an assessment platform that knows every learner's next answer before they give it?
Most projects go from kickoff to deployed AI system in 8–16 weeks. Let's talk about what Knowledge Tracing, Maximum Information Selection, and Response Correctness Prediction could do for your learners' outcomes.