Estimating Vocabulary Size: A Stratified Beta–Binomial First Pass

A learning-in-public first pass at estimating receptive vocabulary from stratified responses with a Beta–binomial model.
Author
Published

July 12, 2026

Keywords

vocabulary-estimation, beta-binomial, posterior-predictive, stratified-sampling

Reading controlsOpen or close help, code, and equations. Choices for code and equations apply across this article namespace.Help is hidden.

Code is hidden.
Equations are shown.

A test result is meaningful only after its target is defined. Measurement is the disciplined act of connecting recorded observations to that target. It is not the same as attaching a number to a person.

I am building Lexibench, but the model in this article is a historical first-pass target scorer. It is not a description of the scorer currently deployed at Lexibench. Correct probability calculations cannot rescue a vague target or an unsuitable item pool.

  1. Define
    What exactly would the reported count mean?
  2. Sample
    How can eight balanced strata represent a fixed pool?
  3. Update
    How do responses change uncertainty within each stratum?
  4. Predict
    How do tested and untested pairs combine into a finite total?
  5. Decide
    When is the estimate precise enough to stop?
  6. Reproduce
    How do fixtures, seeds, tests, and publication gates protect the result?

1. Define: what is being measured?

The casual question “How many words do you know?” hides several choices. Does ran count separately from run? Does one meaning of bank count separately from another? Which language inventory supplies the denominator? This first pass cannot answer every version of that question. It answers one narrower, reproducible version.

The estimand here is:

Receptive knowledge of lemma–surface-form pairs in a fixed, versioned pool.

A lemma such as run may have several surface forms. The NKJP and SUBTLEX frequency data used to rank a pair does not distinguish senses. The item curation process mitigates this by rejecting ambiguous or overly context-dependent pairs and placing each remaining surface form in a short, commonly heard sentence showing a normal use. This makes unusual senses less likely to add difficulty, but it is a design safeguard, not evidence that sense has no effect. The context and translation select an intended meaning for the canonical item, but they remain measurement metadata; this model does not pretend it has estimated sense-specific knowledge.

Receptive knowledge

Recognising and understanding a form when it is encountered, rather than being able to produce it unaided.

Estimand

The precisely defined quantity a measurement procedure aims to estimate. Here it is a count of known lemma–surface-form pairs in one fixed pool.

Lemma

A dictionary headword used to group related inflected forms. For example, the English lemma run groups forms such as run, runs, ran, and running.

Surface form

The exact written form that appears in text or in a question, such as ran or running.

Lemma–surface-form pair

One lemma ID linked to one surface-form ID—the countable unit in this model. For example: lemma run + surface form ran.

Sense

One distinct meaning of a lemma, such as run meaning “move quickly” versus a machine that “runs.” The frequency table has no sense IDs, so the model cannot estimate sense-specific knowledge.

Canonical item

One fixed, versioned quiz question for a pair. Its context selects the intended meaning for measurement, but does not create a sense-specific frequency count.

The estimand counts the four lemma–form pairs separately. It does not claim four independent lemmas or sense-specific frequencies.

This target can support a qualified statement such as “estimated known pairs in pool P, version V, under algorithm A.” It cannot by itself report a universal count of “words known,” productive vocabulary, sense-specific knowledge, or overall language proficiency. The narrowing is useful because it supplies a denominator, a pool version, and repeatable items.

Verified mathematicsOnce the outcome, prior, and likelihood are fixed, the Beta posterior and posterior-predictive calculation below follow exactly.
Provisional modelling choiceProvisional means chosen as a testable starting assumption, not established by the data. The pair inventory, canonical contexts, frequency strata, response mapping, prior, and stopping threshold all need empirical validation.

2. Sample: balanced rounds from a fixed pool

For exposition, imagine a fixed, versioned pool split by frequency rank into eight strata with the same number of pairs. A round samples one item uniformly at random from each stratum. Later answers do not change which item is selected: selection remains non-adaptive.

The full pool is the population about which this test makes its claim. The much smaller set of administered items is the sample. Sampling is needed because asking all 8,000 questions would defeat the purpose of an estimate.

Pool

The complete fixed, versioned set of lemma–surface-form pairs that this test is allowed to estimate.

Population

Every pair in the fixed pool about which this particular estimate makes a claim.

Sample

The smaller set of pool items actually administered to the learner.

Pair

Shorthand here for one lemma–surface-form pair—not two arbitrary words and not a word sense.

Frequency rank

The position of a pair after ordering the source corpus data from most frequent to least frequent. Rank 1 is most frequent; rank is a proxy, not calibrated learner difficulty.

Stratum

One of eight equal-count bands formed from adjacent frequency ranks.

Item

A versioned quiz question for one pair, including its context, intended answer, and distractors.

Non-adaptive

Earlier answers update the estimate but do not change which later items are selected.

Adaptive

Later items are selected using earlier answers or the learner’s current estimated knowledge.

An adaptive test does change later item selection in response to earlier answers. In principle, it can choose the next question expected to be most informative about the learner’s current knowledge, avoiding many questions that look much too easy or much too hard. That can shorten a test, but only if the item-difficulty model is trustworthy.

I therefore want to keep selection non-adaptive while collecting response data from real learners. Those data should reveal the relative difficulty of the actual questions—including context effects and departures from frequency rank—before I implement an adaptive test. Adapting sooner would risk steering the test with assumptions that have not yet been checked against learners.

flowchart LR P[Fixed versioned pool] --> S1[Stratum 1
most frequent] P --> S2[Stratum 2] P --> S3[Stratum 3] P --> S4[Stratum 4] P --> S5[Stratum 5] P --> S6[Stratum 6] P --> S7[Stratum 7] P --> S8[Stratum 8
least frequent] S1 --> R[One random item from each
8-item round] S2 --> R S3 --> R S4 --> R S5 --> R S6 --> R S7 --> R S8 --> R

Frequency rank partitions the illustrative pool; this diagram does not claim that rank is a calibrated item-difficulty scale.

Frequency rank is only a proxy: a convenient observable used in place of an unobserved quantity. Here it stands in for item difficulty, but it has not been calibrated against learner responses. Equal-count strata balance the sample across that proxy; they do not prove equal difficulty inside a band.

Watch the fixed schedule

Select items one at a time below. Read across the eight cards: each round takes exactly one unseen item from every stratum. The sample responses are shown, but the “next item” labels advance according to the pre-existing queues, never according to those responses.

Loading the balanced-round demonstration…

Build / Check / Decide

Build: version a finite population and queue unseen items in eight frequency bands. Check: every completed round contains one item from each band, with no repeats. Decide: keep selection non-adaptive until a learner-calibrated difficulty model exists.

3. Record first, collapse only for v1 inference

The interface keeps three raw response values: :correct, :wrong, and :dont-know. Stage one maps a correct response to “known” and both other responses to “not known.” Keeping the raw outcomes distinct lets a later model treat guessing, slips, and explicit uncertainty differently without corrupting the original data.

The mapping keeps :wrong and :dont-know as different raw keys even though their stage-one likelihood contribution is the same.

A Bernoulli trial has two outcomes for this model: 1 for correct and 0 for not correct. That binary value is derived for inference; it does not overwrite the original event. This separation is what permits a later model to distinguish a wrong answer from an explicit “don’t know.”

Code detail: Mapping raw responses without discarding the events

The pure function returns the v1 binary outcome. Unknown values fail loudly instead of silently entering the model.

(defn collapse-response [response]
  (case response
    :correct 1
    :wrong 0
    :dont-know 0
    (throw (ex-info "Unknown raw response"
                    {:response response}))))

Storage retains the original keyword; only the input to this first inference model is collapsed.

View the executable article source

4. Update: Beta in, Beta out within one stratum

A sequence of Bernoulli trials can be summarized by a binomial count: \(k\) correct responses among \(n\) attempts. A Beta distribution describes uncertainty about a probability between 0 and 1. It has two positive shape parameters, \(\alpha\) and \(\beta\). Choosing a Beta prior makes the update conjugate: after binomial data, the posterior is another Beta distribution, so no numerical approximation is needed for this step.

Let \(p_s\) be the knowing rate in stratum \(s\). This first pass gives every stratum the same prior:

\[p_s \sim \operatorname{Beta}(1,1).\]

p_s

The learner’s unknown knowing rate in stratum s: the proportion of that stratum’s fixed pairs the learner knows.

s

The stratum index. This first model has eight separate frequency-rank strata.

“Is distributed as.” It describes uncertainty about p_s, not an equality to one number.

Beta(1,1)

A Beta distribution with shape parameters α = 1 and β = 1. Its density is uniform from 0 to 1.

α, β

The Beta distribution’s two positive shape parameters. In this Bernoulli model they update like prior correct and not-correct counts.

About this equation

Beta(1,1) is the deliberately simple v1 prior; calling it uniform does not make it universally uninformative.

Code detail: Representing the Beta(1,1) prior

Source: beta_binomial_first_pass.clj — posterior-parameters default arity

alpha

The code name for the prior shape α; its default is 1.0.

beta

The code name for the prior shape β; its default is 1.0.

k

The observed correct count, omitted conceptually before data.

n

The observed attempt count, omitted conceptually before data.

(defn posterior-parameters
  ([k n] (posterior-parameters 1.0 1.0 k n))
  ([alpha beta k n]
   {:alpha (+ alpha k)
    :beta (+ beta (- n k))}))

The default arity supplies alpha = 1.0 and beta = 1.0, the two parameters in the displayed prior.

That density is uniform over rates from zero to one. I am not claiming that it is universally “uninformative”; parameterization and context matter. If \(k\) of \(n\) sampled pairs are correct, the Bernoulli/binomial likelihood is proportional to \(p_s^k(1-p_s)^{n-k}\), so:

\[p_s \mid k,n \sim \operatorname{Beta}(1+k,1+n-k).\]

Read this as: “start with one prior count on each side; add the \(k\) correct responses to \(\alpha\) and the \(n-k\) not-correct responses to \(\beta\).” For example, three correct among four attempts changes Beta(1,1) into Beta(4,2).

p_s | k,n

The knowing rate in stratum s after conditioning on the observed response counts k and n.

|

“Given” or “conditional on.” Everything to its right is treated as observed information.

k

The number of correct responses in this stratum.

n

The total number of tested pairs in this stratum.

n − k

The number of tested responses treated as not known by the v1 inference model.

1 + k

The posterior α parameter: prior α = 1 plus correct responses.

1 + n − k

The posterior β parameter: prior β = 1 plus not-correct responses.

Beta(…)

The posterior stays in the Beta family because the Beta prior is conjugate to the Bernoulli/binomial likelihood.

About this equation

The complete response history reduces to k and n for this update, although the original correct, wrong, and don’t-know events remain stored separately.

Code detail: Updating the two Beta shape parameters

Source: beta_binomial_first_pass.clj — posterior-parameters

alpha

The code name for the prior α shape; 1.0 in v1.

beta

The code name for the prior β shape; 1.0 in v1.

k

The correct count k added to alpha.

n

The attempt count n; n - k is added to beta.

This function is pure: the same prior and counts always return the same posterior parameters, and no external state changes.

(defn posterior-parameters
  ([k n] (posterior-parameters 1.0 1.0 k n))
  ([alpha beta k n]
   {:pre [(<= 0 k n) (pos? alpha) (pos? beta)]}
   {:alpha (+ alpha k)
    :beta (+ beta (- n k))}))

The precondition rejects impossible counts and non-positive Beta parameters before they can contaminate a result.

View the complete posterior update

{:alpha 4.0, :beta 2.0} is the entire update for three correct among four. “Posterior” names the updated density for \(p_s\). The “likelihood” is the information supplied by the last response as a function of \(p_s\); it is not itself a posterior.

Try the update

Press Correct, Wrong, or Don’t know. The accessible SVG compares the uniform prior, the previous posterior, the last-response likelihood, and the current posterior. Curve height is density, not the probability of one exact value of \(p\).

How to read the chart

  • Horizontal position is a candidate knowing rate from 0 to 1.
  • Curve height is probability density: where the distribution concentrates, not the probability of one exact decimal.
  • The solid current curve is the result after the latest response; the dashed curves show what was multiplied to obtain it.

Loading the Bayesian update simulator…

5. Predict: from knowing rates to an untested finite pool

For a stratum containing \(N_s\) pairs, of which \(n_s\) were tested and \(k_s\) were correct:

  1. draw \(p_s\) from the posterior;
  2. draw the number known among the untested pairs from \(\operatorname{Binomial}(N_s-n_s,p_s)\);
  3. add the \(k_s\) observed correct pairs;
  4. sum the eight stratum totals.

\[U_s \mid p_s \sim \operatorname{Binomial}(N_s-n_s,p_s), \qquad T=\sum_{s=1}^{8}(k_s+U_s).\]

Read this as: “predict \(U_s\) known pairs only among the \(N_s-n_s\) untested pairs; add the already observed correct pairs \(k_s\) once; then sum all eight strata.” Tested-not-correct pairs contribute zero and are never predicted again. This avoids both double counting and pretending an observed response is still unknown.

U_s

The predicted number known among the untested pairs in stratum s.

N_s

The fixed number of pairs in stratum s.

n_s

The number of pairs already tested in stratum s.

p_s

One knowing rate drawn from stratum s's Beta posterior.

k_s

The observed correct count, added directly rather than predicted again.

T

One complete posterior-predictive draw of total known pairs across all eight strata.

Σ

Sum the parenthesized stratum total for s = 1 through 8.

About this equation

The binomial draw concerns exactly the untested finite remainder, not a new infinite population and not the tested items.

The following functions are the executable version. Fastmath supplies the Beta and binomial distributions; a seeded Mersenne Twister makes the rendered result reproducible.

Code detail: Making one finite-pool posterior-predictive draw

Source: beta_binomial_first_pass.clj — posterior-predictive-stratum and posterior-predictive-total

pool-size

The code name for N_s, the fixed number of pairs in one stratum.

n

The code name for n_s, the tested count.

k

The code name for k_s, the observed correct count added once.

p

One sampled p_s from the stratum's Beta posterior.

untested

The code name for N_s - n_s.

predicted

One sampled U_s for the untested remainder.

The draw keeps the three groups separate: observed correct, observed not-correct, and untested. Only the untested group is simulated.

(defn posterior-predictive-stratum
  [rng {:keys [pool-size k n]}]
  (let [{:keys [alpha beta]} (posterior-parameters k n)
        p (random/sample
           (random/distribution :beta
             {:alpha alpha :beta beta :rng rng}))
        untested (- pool-size n)
        predicted (random/sample
                   (random/distribution :binomial
                     {:trials untested :p p :rng rng}))]
    (+ k predicted)))

A random seed initializes the pseudo-random generator. Supplying the same fixture and seed reproduces the same draw sequence.

View the complete seeded prediction functions

Code detail: Selecting equal-tail interval endpoints

Sort the finite totals, convert a probability to an index, and select that ordered value. A 95% equal-tail interval uses probabilities 0.025 and 0.975.

(defn quantile [xs probability]
  (let [ordered (vec (sort xs))
        i (long (Math/floor
                 (* probability (dec (count ordered)))))]
    (nth ordered i)))

(equal-tail-quantiles draws 0.95)

This function selects quantiles from Monte Carlo draws. The exact reference interval below instead selects where the cumulative exact discrete probability crosses the same two cut points.

View the quantile implementation

6. Combine: a pedagogical 8,000-pair pool

8,000 is an illustration, not a current CEFR or Lexibench pool size. It consists of eight 1,000-pair strata. After four complete rounds, suppose the correct counts are [4 4 3 3 2 1 1 0].

Code detail: Calculating a stratum's posterior-predictive mean

The Beta posterior mean is alpha divided by alpha plus beta. Multiply it by the untested count, then add observed correct pairs exactly once.

(defn analytic-stratum-mean
  [{:keys [pool-size k n]}]
  (let [{:keys [alpha beta]}
        (posterior-parameters k n)]
    (+ k
       (* (- pool-size n)
          (/ alpha (+ alpha beta))))))

This is analytic: it follows directly from the distribution's expectation and has no simulation noise.

View the worked calculation

The mean is the probability-weighted average of all possible totals under the model. It is 4,334 pairs here. Row by row:

Worked example by frequency-rank stratum
Stratum Pool Correct / tested Posterior Mean known
1 1000 4 / 4 Beta(5, 1) 834
2 1000 4 / 4 Beta(5, 1) 834
3 1000 3 / 4 Beta(4, 2) 667
4 1000 3 / 4 Beta(4, 2) 667
5 1000 2 / 4 Beta(3, 3) 500
6 1000 1 / 4 Beta(2, 4) 333
7 1000 1 / 4 Beta(2, 4) 333
8 1000 0 / 4 Beta(1, 5) 166
Total 8,000 18 / 32 4,334

A point mean hides uncertainty: the model still supports a range of totals. Here an exact calculation is possible because every stratum has a finite number of outcomes. For each possible untested count \(x\), the Beta–binomial probability is

\[P(U_s=x\mid k_s,n_s)= {m_s \choose x} \frac{B(x+\alpha_s,m_s-x+\beta_s)}{B(\alpha_s,\beta_s)}, \qquad m_s=N_s-n_s.\]

Read this as: “assign an exact probability to every possible known count among the untested pairs, using the updated Beta uncertainty.” Convolving—the discrete equivalent of adding—all eight probability lists produces an exact distribution for the total. The 95% equal-tail credible interval removes 2.5% probability from each tail. Its endpoints are 3,404–5,249.

P(U_s = x | k_s,n_s)

The posterior-predictive probability that exactly x untested pairs are known in stratum s.

m_s

The untested remainder N_s − n_s; 996 in every worked-example stratum.

choose(m_s,x)

The number of ways x known outcomes can occur among m_s untested pairs.

B(·,·)

The Beta function; the ratio integrates over uncertainty in p_s rather than plugging in one rate.

α_s, β_s

The two posterior Beta parameters for stratum s.

About this equation

Calculating all x values and adding the eight independent stratum distributions yields the exact finite-pool total distribution.

Code detail: Translating the exact Beta–binomial mass

Code explanation: Direct Clojure translation using the article implementation's pool-size, k, n, alpha, and beta names.

pool-size

The code name for N_s, the complete stratum size.

m

The local code name for m_s = N_s - n_s, the untested remainder.

x

One possible value of U_s, the known count among untested pairs.

alpha

The code name for posterior shape α_s.

beta

The code name for posterior shape β_s.

beta-function

A direct code name for the mathematical Beta function B(a,b); this explanatory snippet does not add a new production dependency.

(defn beta-binomial-mass
  [{:keys [pool-size k n]} x]
  (let [{:keys [alpha beta]} (posterior-parameters k n)
        m (- pool-size n)]
    (* (binomial-coefficient m x)
       (/ (beta-function (+ x alpha)
                         (+ (- m x) beta))
          (beta-function alpha beta)))))

This is explanatory code, not copied implementation: the article's reference interval is preserved evidence, while the snippet names each factor in the equation.

Reader-facing, I would report:

About 4,330 known pairs, with a 95% credible interval of roughly 3,400–5,250, conditional on this model and fixed pool.

“95% credible interval” is not “95% certainty that every modelling choice is right.” It describes posterior uncertainty conditional on the specified model, prior, observed responses, and pool.

Worked-example posterior mean and 95 percent credible intervalAn interval from 3,404 to 5,249 known pairs, with posterior mean 4,334, on a scale from zero to 8,000.02,0004,0006,0008,0003,404 — 4,334 — 5,249
Blue: central 95% credible interval. Red: posterior-predictive mean. Scale: pairs known in the fixed 8,000-pair pool.

Watch complete posterior-predictive draws accumulate

One complete draw below passes through all eight strata. In each stratum it draws a knowing rate from that stratum’s Beta posterior, predicts the known count among its 996 untested pairs, adds the observed correct pairs, and finally sums the eight counts.

The first view exposes the newest complete draw rather than showing only its total. The second retains every complete draw as one dot. Its live mean and interval therefore use exactly the dots you can see—none are hidden in an aggregate. Choose 10, 20, or 50 draws per second; changing speed does not change the seeded sequence.

How to read the posterior-predictive simulator

  • In the newest draw, each row separates tested correct, tested not-correct, and predicted untested pairs.
  • The tested groups are fixed by observed events. Only the untested count is drawn.
  • In the lower chart, one dot is one complete eight-stratum total; the line and marker summarize only the visible dots.

Loading the posterior-predictive sampling simulator…

The 500 browser draws are for seeing the mechanism, not for replacing the larger verification run. Early live intervals are especially noisy.

A seeded numerical check

The Monte Carlo calculation below is an independent numerical check on the analytic mean and exact reference interval, not the source of those values.

Small differences are Monte Carlo error. The verification tolerance used for this draft is ±20 pairs for the mean and ±40 pairs for each endpoint.

7. Decide: when should the test stop?

The stopping rule is also provisional:

  • ask at least 32 items (four complete rounds);
  • reassess only after another complete eight-item round;
  • target a 95% interval half-width near 10% of the pool;
  • use 96 items as a soft maximum;
  • always allow the learner to stop voluntarily.

The minimum length prevents an unstable early estimate from triggering a precision rule. Round boundaries preserve the same number of sampled items per stratum. The precision target asks whether the interval’s half-width is at most 10% of the pool. The soft maximum recommends stopping at 96 without making continued participation impossible. A voluntary stop overrides all statistical criteria because the learner remains in control.

Code detail: Applying the v1 stopping decision

Assessment occurs only at complete eight-item rounds and only after the 32-item minimum. The precision target and soft maximum are separate reasons to recommend stopping.

(let [complete-round? (zero? (mod items-tested 8))
      assess? (and complete-round?
                   (>= items-tested 32))
      half-width (/ (- upper lower) 2.0)
      target? (and assess?
                   (<= half-width (* 0.10 pool-size)))
      soft-max? (and assess?
                     (>= items-tested 96))]
  {:stop? (or voluntary? target? soft-max?)})

The complete function also returns each intermediate condition, making the final recommendation explainable and testable.

View the stopping predicate and examples

Examples of the provisional stopping rule
Items Complete round? Assess? Half-width Stop?
24 Yes No 1,100 No
32 Yes Yes 923 No
36 No No 800 No
40 Yes Yes 750 Yes
96 Yes Yes 900 Yes

At 36 items the interval happens to be narrow enough, but the function does not assess it: 36 is not the end of an eight-item round. At 96, the soft maximum recommends stopping even when the width target is missed.

Explore the rule without rewriting history

The controls below begin at the authoritative v1 defaults: minimum 32, interval half-width target 10% of the pool, and soft cap 96. Changing those three values creates a counterfactual teaching scenario only. It does not alter this article’s recorded model, examples, assertions, or scorer.

Loading the teaching-only stopping-rule explorer…

Build / Check / Decide

Build: compute the exact finite-pool distribution and a qualified estimate. Check: reproduce it with a seeded simulation and inspect stopping only at round boundaries. Decide: stop for voluntary choice, adequate precision after the minimum, or the soft maximum—without changing the rule after seeing one learner's result.

8. Reproduce: protect this model’s evidence

A trustworthy calculation needs a boundary between the mathematical model and software that reads files, draws charts, or responds to clicks. A pure function returns an output determined only by its inputs. A side effect changes or observes something outside that return value, such as a database, clock, random generator, file, or browser state. The scorer’s mathematics should remain pure; storage, UI, clocks, and unseeded randomness belong at the boundary. Clojure encourages this separation without pretending all useful programs are side-effect-free.

The worked example is also a fixture: a small, named input with expected outputs used repeatedly in checks. An immutable version is never edited after events refer to it; a changed pool or item receives a new identifier. The original correct, wrong, and dont-know events remain unchanged so a later algorithm can replay them. A seed turns pseudo-random prediction into a repeatable function of versioned inputs.

The wider workflow lives in Article 0

This section keeps the evidence specific to v1: pure scoring, immutable fixtures and response events, seeded replay, and CLJ/CLJS parity. Article 0 explains the complete theory-to-algorithm cycle and its separate model, software, and publication gates.

Code detail: Keeping a pure scoring boundary and deterministic fixture

All changing inputs are explicit data. The function creates its seeded generator internally, returns a value, and neither reads nor writes application storage.

(def worked-fixture
  {:fixture-version "beta-binomial-v1-worked-2026-07-12"
   :algorithm-version "beta-binomial-v1"
   :pool-id "pedagogical-8000-v1"
   :seed 20260712
   :strata worked-strata})

(defn score-worked-fixture [{:keys [seed strata]}]
  {:analytic-mean
   (long (reduce + (map analytic-stratum-mean strata)))
   :one-seeded-draw
   (posterior-predictive-total seed strata)})

Calling this function twice with the same fixture must return equal maps; the final regression checks enforce that deterministic replay.

View the fixture, scorer, and assertions

Clay evaluates this article’s Clojure source, writes QMD, and hands it to Quarto; Scittle/Reagent supplies the bounded browser interactions. Browser checks cover labels, both colour themes, responsive widths, control behaviour, and console errors. Those checks protect this explanation without changing the model or turning a provisional measurement assumption into a fact.

What this model postpones

A small coherent model is a useful place to begin, not a useful place to end. The next steps are:

  • use continuous pair frequency as a difficulty proxy rather than treating eight bins as calibrated difficulty — that is the next post;

  • define and version the conversion among CEFR descriptors, lemmas, and pair inventories;

  • aggregate pair probabilities hierarchically into latent lemma knowledge;

  • use all three response outcomes in a guessing/slip model;

  • investigate multiple contexts, item calibration, item-response theory, and adaptive selection.

In particular, I am not using a naive independent-form formula as a final lemma estimator. Forms of the same lemma are related, contexts vary in informativeness, and a latent-variable model should express that structure.

Checkpoint

The v1 model now has a precise estimand, balanced non-adaptive sampling, lossless raw events, eight conjugate Beta updates, finite-pool posterior prediction, an exact qualified interval, seeded numerical checks, and an explicit stopping rule. Every modelling choice remains provisional. The next article tests one refinement—continuous pair frequency—without silently rewriting this checkpoint.

Sources and further reading

This is a learning-in-public checkpoint. Follow-up posts will replace the roughest assumptions one at a time, beginning with continuous frequency.

Regression check All executable assertions passed, including the preserved 4,334 mean, Monte Carlo tolerance around the exact 3,404–5,249 interval, round-boundary behavior, and deterministic fixture replay.