Synthetic answers and frozen review scores

200 fixed synthetic answer records: 190 correct, 10 incorrect. Incorrect answers are the positive class (failure). Flag a record when its frozen review score ≥ threshold. Scores are constructed signals, not probabilities. Only the threshold changes; answers, labels, and scores stay fixed. No model training or API calls.

• Original deterministic teaching cases: every twentieth source case has an answer one above the correct sum, then cases are permuted by (index × 73) modulo 200. No real LLM outputs or human-labeling study are represented.
• Scores 90, 70, 40, 10 occur on 2, 4, 2, 2 failures and 2, 8, 20, 160 passes respectively. Scores intentionally overlap across labels.
• The reference labels are established by comparing each answer with its known arithmetic result. They are used for evaluation, not for the threshold decision.
• Threshold changes do not regenerate records, scores, or labels. The explorer shows reference inputs; run reports preserve the inputs and every prediction.
• This is a development fixture for exploring metrics. A threshold selected here needs independent evaluation on representative held-out data before a deployment claim.

Counts exclude the row identifier. Duplicate counts are extra occurrences after the first. Feature duplicates ignore labels and experimental conditions. — means a statistic is undefined or not applicable.

Row

Stable record identifier, not a model feature.

Request

Request

Answer

Answer

Expected answer

Known arithmetic result, used only to establish the reference label.

Reference label

Failure means the answer is incorrect; this is the positive class.

Review score

Constructed fixed ranking signal. Not a probability and not fitted from these records.