Quiz reveals the answer too early? Inspect labels and hints

Give a text AI synthetic questions and answer rules to draft an answer-leakage and reset checklist for human review. This is a drafting exercise; the source used Devin.

Execution environment and usage

AI / software

General text chat (example: Claude)

Where to operate

A general text-AI conversation; for example, a new chat on the Claude website where available. This is a text exercise, not an agent run.

Getting started

  1. Open an authorized text-chat tool; Claude’s web entry is claude.ai.
  2. Paste the relevant text from synthetic questions and answer rules and the prompt below; request an answer-leakage and reset checklist.

Setup and plan requirements

Prepare shareable, de-identified synthetic questions and answer rules; paste only relevant excerpts. An available chat account and allowance are needed; Claude also has location and age requirements. This exercise needs no connector or execution permission from the original source workflow. The chat can produce the text draft. Any later checks of actual files, screens, audio or test results require a person, suitable software and existing outputs. Mark checks unverified when those outputs are absent.

Prompt portability limits

Questions and supplied text can be adapted to another capable chat tool. This produces a draft, not file access, code execution, account connections or the original operational outcome.

Usage limitations

The source reports Devin. This page adapts it into an answer-leakage and reset checklist, not a completed automated workflow in that product. Follow the checks below; source outcomes do not establish this exercise’s results.

Environment sources

適用情境 / When to use

Give a text AI synthetic questions and answer rules to draft an answer-leakage and reset checklist for human review. This is a drafting exercise; the source used Devin. The event host’s transcript supports a vendor demonstration, not a learning-outcome study.

步驟 / Steps

  1. Prepare synthetic object questions, answer keys and label rules.
  2. Inspect labels, alternative text and hints before and after answering.
  3. Ensure one score update per answer and reset prior state for a new round.

提示詞 / Prompt

Using [synthetic object questions/answer keys/hint rules/scoring rules/accessibility needs], design quiz acceptance checks. Inspect unanswered, answered and restarted states for answer leakage through labels or alternative text and repeated scoring. Plan tests only; do not identify people or claim learning benefits.

留意事項 / Cautions

  • Keep useful accessible descriptions; if they necessarily reveal the answer, provide an equivalent accessible task instead of withholding needed information.
  • Do not collect real portraits or assess learning ability.

來源限制 / Evidence limits

This exercise uses fictional objects rather than identifying people or building a face database.

流程說明 / Workflow note

These steps and prompts are original, untested exercises by this site, not the author’s verbatim workflow. Reported results do not guarantee exercise outcomes.

原始工具組合 / Original tool stack

Devin; React; tabular quiz data

Source checked: 2026-10-20

The Making of Devin: Scott Wu talk