DOI: 10.3390/app162110561 ISSN: 2076-3417

SocraGate: Inference-Time Admission Control for Reducing Answer Leakage in Scaffolded LLM Tutoring

Wanfang Li, Keyun Zhao

Large language models can solve mathematics problems accurately yet compromise tutoring by revealing answers before learners complete the reasoning. Prompt- and reward-based alignment changes the response distribution but does not screen the particular utterance delivered to a learner. We therefore formulate answer leakage as an output-admission problem and introduce SocraGate, a two-stage framework that combines scaffold-oriented policy optimization with inference-time screening. A five-level scaffold-spectrum reward shapes candidate generation. Deterministic answer-equivalence rules and a reference-conditioned semantic classifier then screen complete candidates against a fixed leakage threshold. An empty admitted set triggers re-screened regeneration and, if necessary, a fixed probe. Across five MathDial training seeds, SocraGate reduces utterance-level telling rate (TR) from 0.168 for the fixed-simulator GRPO baseline to 0.032, while simulator-based Success@10 rises from 0.482 to 0.546. Under the same simulator-based protocol and without parameter updates, TR also falls on QATD-2k (0.159 to 0.047) and CoMTA (0.150 to 0.053). In a blind evaluation of 160 dialogues, SocraGate receives the highest mean rating on all four dimensions, including 4.78 out of 5 for non-leakage. The cumulative analysis shows the largest single-stage reduction in TR after deterministic screening, whereas guarded regeneration raises Success@10 from 0.521 to 0.546. These results support output admission as a practical complement to policy alignment for scaffolded LLM tutoring.

More from our Archive