One question.
A careful journey.

Gather and prepare the evidence

The study assembled authentic teaching and institutional material alongside expert-reviewed synthetic variants. The source corpus contained 1,227 sentences: 811 authentic and 416 synthetic. One insufficient-context reference was excluded from the original binary custom-model experiments. Source sentences, annotations and datasets remain private.

Compare offline and online language models

The research compared locally hosted Llama, Gemma, Mistral and DeepSeek with hosted GPT and Gemini. Models received a common classification task. Expert-0 supplied the reference judgments. Saved outputs enabled comparisons without repeating completed requests.

Investigate decisions with SHAP and LIME

Word-deletion and phrase-grouping pilots preceded the Main explanation experiment on 20 selected sentences. The first five-model Main saved 8,675 unique responses; a later local DeepSeek extension added 1,735. The original sentences remained unchanged; temporary perturbations tested response behavior. Exact group Shapley calculations, sampled approximations and LIME diagnostics were distinguished. These explain observed response changes, not internal reasoning.

Build smaller classification baselines

The custom-model study compared TF–IDF, SetFit–MiniLM and frozen transformer encoders, including BERT, RoBERTa, DeBERTa and DistilBERT. Five category holdouts with inner development folds supported exploratory comparisons while keeping related sentence boundaries in view. Authentic and synthetic results needed separate interpretation.

Adapt selected pretrained models

SetFit adapted MiniLM through its contrastive training procedure. BERT encoder adaptation and a Gemma adapter were also studied locally. Gemma was compared with a matched unadapted baseline because its prompt and quantization differed from the original LLM study. These were adaptations of pretrained models, not foundation models built from scratch.

Combine model judgments

BERT + Gemma and SetFit–MiniLM + Gemma were examined as decision-level combinations. Their weights were not merged. Agreement produced a provisional proposal; disagreement or invalid output required review. Agreement alone could still be wrong, which motivated expert inspection.

Bring experts into the loop

A 101-item diagnostic booklet and paired annotation clarification materials supported human review. Expert-03’s labels and wording suggestions were transcribed and clarified, with original annotations preserved. Expert-02’s review remains pending. This selected review sample is not an independent representative test.

Create a working, updateable prototype

For this provisional release, Expert-03’s labels replace the original reference only for her reviewed cases. Fourteen insufficient-context cases are excluded, and a new balanced classification head is fitted on 1,212 unchanged binary examples using an existing adapted BERT encoder. The public app offers BERT + Gemma and SetFit–MiniLM + Gemma as two live choices. Fixed earlier SetFit and Gemma snapshots are reused; only BERT’s head incorporates the provisional Expert-03 labels. Both components assess a visitor’s sentence, agreement yields a proposal, and disagreement requests priority expert review. Surface wording cues support review; no trained rewriting or automatic expert adjudication is provided. The website and on-demand GPU inference operate independently of the researcher’s Mac.

What this journey does—and does not—establish

Earlier comparisons informed the design. They do not supply an independent performance estimate for the newly fitted public classifier. Expert-label revisions, inspected examples, limited authentic positive cases, unresolved synthetic lineage and source-transfer limits must be considered. A useful demonstration is a starting point for evaluation, not proof of readiness for automatic decisions.

The research datasets, individual sentences, model-response archives and expert annotations are not published on this site.