ADR 0014 promises that a failed Check leaks only a scalar. The Student is told how many Challenges are wrong and never which, so a Sentence Selection cannot be solved by diffing feedback instead of reading the passage.
That promise works right up until it doesn’t. A Student who has Checked three times and been told a number three times has nothing left to reason with. So they start permuting — which is exactly the guessing until it passes the Card exists to stop, and it is a worse outcome for the answer key than saying more would be, because brute force recovers the key anyway, just slowly and without teaching anybody anything.
So from the 3rd failed Check of a visit, Sentence Selection and Word Match say more. That is a Hint, and it is deliberate enough to have cost an ADR.
The rung that told the Student nothing
The ladder was first drafted with three rungs above the baseline. Fails 1–2: today’s mistake count. Fail 3: the set has Y Sentences. Fail 4: you got X of them. Fail 5: the marks.
That middle rung does not exist, because it is free.
Look at what is already on the screen in the shot above. The Student is told N = 3 mistakes. The counter under the passage says S = 2 Sentences selected — and S is not a Hint, it is just a count of clicks they made themselves. The mistake count is missed correct Sentences plus false positives, so with X the number of their selections that are actually correct:
N = (Y − X) + (S − X) = Y + S − 2X
Solve it for X:
X = (Y + S − N) / 2
Every term on the right except Y is already on screen. Revealing the size of the correct set therefore reveals the score. A rung that announced there are 3 Sentences to find would have handed over you got 1 of them in the same breath, and then charged the Student another Check to be told it out loud.
So the two rungs merged. The screen does the arithmetic and says the useful half.
The equation runs backwards too, which settled a smaller question. Once the Student knows X and Y, N is derivable the same way — so the score line replaces the mistake count rather than joining it. Printing both would be printing one number twice, in two grammars, one of which is a scolding.
Merging also lands the marks at Check 4 instead of Check 5, which matters more than it sounds: every rung is a round trip the Student spends while stuck.
The oracle you get for free if marks re-evaluate
The marks themselves arrive one Check later.
Two rules bound what those rings give away, and both are load-bearing.
The first is that a mark decays rather than re-evaluates. Deselect a Sentence and its ring disappears; select a new one and it stays bare until the next Check. The obvious alternative — recompute the marks on every toggle, since the server knows the answer anyway — is a catastrophe dressed as a convenience. Toggle one Sentence on, read its ring, toggle it off. Repeat. Five toggles later you have the entire correct set, having spent zero Checks and left zero Challenge Attempts behind.
Decay closes that. The Student’s information grows only by spending a Check, every Check costs a round trip, and every Check writes to the log — so the cost of extraction is visible in Analytics rather than silent.
flowchart TD
CK(["a failed Check"]) --> N{"failed Checks
this visit"}
N -->|"1 – 2"| R0["You made N mistakes.
a scalar — ADR 0014 as written"]
N -->|"3"| R1["You got X of the Y Sentences.
replaces the count; still no marks"]
N -->|"4 and after"| R2["the same line, plus a ring on
every Sentence the Student selected"]
R2 --> M[["marks: Sentence id → verdict
the component's assigns, nothing persisted"]]
TG(["Student toggles a Sentence"]) -->|"deletes that one entry —
and recomputes nothing else"| M
NAV(["navigates away and back"]) -->|"counter and marks
both reset to zero"| N
The second rule is that an unselected Sentence is never marked, at any rung. In the screenshot above, three Sentences carry nothing — and two of those three are in the correct set. That ambiguity is the whole point. If every Sentence were marked, the Student would invert the wrong ones and read the answer off the passage in a single Check. Because only their own picks are judged, unmarked stays ambiguous between correctly left out and you missed this one, and nothing can be inverted.
A red ring on a selected Sentence, by contrast, leaks nothing new: among the Sentences they picked, not green already meant wrong.
That asymmetry has a small, precise expression in the code. A mark is not a boolean:
defp mark_class(marks, id) do
case Map.fetch(marks, id) do
{:ok, true} -> "correct"
{:ok, false} -> "incorrect"
:error -> nil
end
end
:error is not false. Three states, not two — judged right, judged wrong, and not judged, which is where every unselected Sentence and every Sentence the Student has touched since the last Check lives. Collapsing those last two into false would paint a rust ring on a Sentence nobody had ever judged.
Word Match needs one rung and no asymmetry
The same threshold, a shorter ladder. Below it, the screen says exactly what it always said:
And at it, every Blank is judged at once.
There is no score rung, because the fill bar already reads 4 / 4. Telling a Student how many Blanks they have filled, when they can see how many Blanks they have filled, is the same informationless rung the arithmetic deleted upstairs — caught by looking at the screen rather than by algebra.
There is no never mark the unselected rule either, because Word Match has no equivalent. Every Blank is answered by definition — the Check is gated on a full board — so there is no third state to protect. And an ✗ says the word sitting here is wrong; it never says what belongs there. The Bank still holds six chips for four Blanks, and the Blank→answer mapping still never reaches the browser.
The one deliberate cruelty: the wrong word stays in its Blank. Bouncing it back to the Bank would be the platform doing the correcting, and it would wipe the evidence of what the Student actually tried — which is the mistake the whole Card exists to make them look at. So it sits there, ringed rust, until they clear it themselves. And when they do, the mark goes with it and the Check button greys out until the Blank is refilled.
Grouping was left out on purpose, and not for scoping reasons. Its ✓/✗ looks like the same feature and is not: keep_correct re-deals only the wrong Cards, so those marks are load-bearing mechanics rather than help. Gating them behind three failures would break the Round. Quiz was left out for the opposite reason — it Checks each pick as it is made, and its answer key is already on the review screen.
Where the ladder lives
Nowhere, is the short answer. The counter and the marks are two assigns on the LiveComponent, beside selected and placements, and they die with the same page.
Navigating away and back returns the Student to zero Hints. That looks like an exploit surface and is the reverse of one: leaving loses help. It also cost no schema, no persistence, no expiry policy, and it matches what CONTEXT.md already says about both Lesson types — navigating away and back starts over.
The threshold is a module attribute. Making it an Instructor setting would have meant a migration plus an authoring control on two Lesson types, handing every Instructor a decision on every Lesson, to tune a number nobody has evidence about yet. One constant moves after watching real Students; thirty Instructors’ guesses do not.
Nothing announces any of this. No hint unlocked banner, no countdown of Checks remaining. The feedback line rewords and the rings appear — least chrome, least to translate, and no incentive to burn Checks farming Hints.
What the tests could reach
The ladder is a pure function of an integer, so when each rung unlocks needs no database at all:
assert Hint.rung(:sentence_selection, 2) == :none
assert Hint.rung(:sentence_selection, 3) == :score
assert Hint.rung(:sentence_selection, 4) == :marks
assert Hint.rung(:word_match, 3) == :marks
Thirteen async: true assertions, no DataCase. What each rung is allowed to compute is pure too — score/2 and selection_marks/2 sit in the domain, and the test that matters most is the one asserting a negative:
test "never mentions a Sentence the Student left unselected", %{document: document} do
marks = SentenceSelection.selection_marks(document, MapSet.new(["s2"]))
assert Map.keys(marks) == ["s2"]
end
That leaves the LiveView tests to cover only what a component can show and a pure function cannot: which elements carry which class, that a toggle drops exactly one mark and recomputes nothing, and that a second live/2 starts from zero. Twenty-one of those, plus a browser run of 29 checks — including a grep of the rendered page at the loudest rung, proving that "correct" and "answer" appear nowhere in it. ADR 0014’s mechanism is untouched; only its behavioural promise moved.
One pre-existing test had to change. "retrying is unlimited after an incorrect Check" loops three failed Checks, so its third now reads the score line instead of the mistake count. Its actual subject — that a Check is never refused — is unchanged, but it is a nice illustration of a test that had quietly encoded a policy it was not written to be about.
Colour is not a text alternative
Every mark carries a word. A clipped, out-of-flow span rides inside each ringed Sentence and each marked Blank, so a screen reader hears — a correct pick or — the wrong word and never has to see the difference between moss and rust.
Getting it inside the element was the fiddly part. An aria-label on the Sentence would have been simpler and wrong — the Sentence is a role="button", so a label there replaces the sentence text rather than appending to it, and the Student would have heard the verdict instead of the passage. And the passage renders under white-space: pre-line, where a stray newline in the template is a line break on screen, so the span had to be packed flush against the text with no whitespace between them.
The Blanks were kinder: .wm-blank is an inline-flex container, and flex layout discards whitespace-only text nodes between its children, so the ✓/✗ chip could be formatted normally.
The translation loss that didn’t happen
The refinement predicted one, and was wrong in a useful way.
mix.exs pins --no-fuzzy, so rewording an existing msgid loses its translation rather than inheriting a guess. The Confirmation.md therefore flagged “You made %{count} mistakes.” and “You have %{count} errors.” as strings that would need retranslating by hand.
They didn’t, because the ladder never rewords them. At rungs 1–2 the msgid is byte-identical; at rung 3 a different msgid is rendered instead. Replacing a line is not the same operation as rewriting a string, and only the second one is destructive.
Wrote priv/gettext/pt/LC_MESSAGES/default.po
(5 new messages, 0 removed, 664 unchanged, 0 reworded (fuzzy), 0 marked as obsolete)
Five new strings, five pt translations written by hand, and every existing translation left alone.
An exact match still completes the Lesson at any rung, and the Challenge Attempts are written identically whether a Hint was showing or not — a Student who needed six Checks still shows six. Only the wrong rate dilutes, and only on the Challenges hard enough to trigger a Hint, which are already the ones Analytics ranks hardest. Making Hint-assisted work discriminable, the way Retake work is for Quiz, is a migration plus a query change for a second-order effect. It is deferred, not rejected.