Bayes’ Theorem, Revisited: Three Interactive Simulations

Rebuilding three earlier Bayesian simulations with Kindly, Scittle, Reagent, and accessible SVG.

Keywords

bayes-theorem, grid-approximation, posterior-sampling, normal-distribution, data-visualisation

I thought I’d reproduce my previous learning about Bayes’ theorem simulations (source code) as a Civitas article. I previously produced these simulations when I was working with Daniel’s JointProb group and used visualisation tools that were easily available at the time. I’m pretty confident that the current visualisation tools are going to do a lot better job with this simulation, judging by how wonderful a job they did with my first Civitas article.

Series contents

Revisiting basic Bayes' theorem and applying it to a real word problem: estimating vocabulary size from Lexibench.com quiz responses.

1. Bayes' Theorem, Revisited: Three Interactive Simulationspublished
2. Estimating Vocabulary Size with a Simple Bayesian Modelpublished
3. Does Pair Frequency Predict Learner Responses?planned
4. From Self-Reported CEFR to a Versioned Lemma–Form-Pair Poolplanned
5. From Correlated Form Pairs to Latent Lemma Knowledgeplanned
6. Modelling Correct, Wrong, and Don't-Know Separatelyplanned
7. Calibrating Items Before IRT and Adaptive Selectionplanned
8. When Contexts and Senses Become Identifiableplanned

What I changed in this recreation

I kept the old application's complete three-part teaching sequence and its numerical grids, but rebuilt how it is delivered:

• The three routed Shadow-CLJS pages are now one executable Civitas article, so the globe update, posterior-sampling decision, and Gaussian update can be read as one argument.
• The compiled Shadow-CLJS application is replaced by the same Kindly–Scittle–Reagent arrangement used in my first article. The mathematical core lives in this .clj article; browser state and controls live in one adjacent .cljs file.
• Vega, Hanami, Semantic UI React, and MathJax React are removed. Kindly and Quarto render the prose and equations; semantic HTML supplies the controls; Reagent renders the visualisations directly as responsive SVG.
• The original 201-point probability grid, four selectable priors, 200-item globe simulation, 2,000 animated posterior draws, 10,000-draw comparison, 41 × 41 Gaussian parameter grid, and complete adult-height dataset are preserved.
• Previously ambient calls to rand are replaced by explicit seeds. Reset now replays the same globe data and posterior draws, making screenshots, explanations, and later regression checks reproducible.
• The old long quotations from Statistical Rethinking are paraphrased while retaining their examples, section references, and attribution.
• Every chart now has an SVG title and description; controls have associated labels and explicit keyboard behavior; status text is announced selectively; and the layout collapses without horizontal overflow on small screens.
• The Gaussian heat maps use within-panel logarithmic opacity. This keeps low but non-zero plausibilities visible as the posterior concentrates. Their colour is therefore for comparing location and shape within a panel, not absolute density across panels.
• I added direct Previous and Next controls to the height simulation and bounded its final update, avoiding the old end-of-data indexing edge case.

The original project was a three-page Shadow-CLJS application using Vega, Hanami, Semantic UI React, and MathJax React. This version keeps the three lessons together in one executable Civitas article. Kindly renders the article, while Scittle and Reagent run the controls directly in the browser. The charts are responsive SVG with text alternatives, so there is no JavaScript build step and no charting wrapper between the data and the marks.

All simulated randomness below is seeded. Resetting a simulation replays the same sequence, which makes the explanations and any later checks repeatable.

What is being recreatedThe globe-toss grid approximation, sampling from its posterior to make a decision under absolute loss, and sequential Gaussian updating for adult heights. The long quotations in the old application are paraphrased here and still attributed to their source.

1. Updating a probability with globe tosses

The first example follows the globe thought experiment in section 2.2 of Richard McElreath’s Statistical Rethinking. Toss a small globe, catch it, and note whether the point beneath your index finger is water or land. The unknown parameter \(p\) is the proportion of the globe covered by water.

I approximate the continuous range of \(p\) using 201 candidates:

\[p \in \{0, 0.005, 0.010, \ldots, 0.995, 1\}.\]

(def probability-grid
  (mapv #(/ % 200.0) (range 201)))

(defn binomial-coefficient
  "Number of orderings containing k successes among n observations."
  [n k]
  {:pre [(<= 0 k n)]}
  (let [k (min k (- n k))]
    (reduce (fn [acc i]
              (* acc (/ (- (inc n) i) i)))
            1.0
            (range 1 (inc k)))))

(defn normalize-mean-one
  "Scale non-negative grid weights so their arithmetic mean is one."
  [weights]
  (let [mean-weight (/ (reduce + weights) (count weights))]
    (if (pos? mean-weight)
      (mapv #(/ % mean-weight) weights)
      (vec weights))))

(defn binomial-likelihood
  "Likelihood of water and land counts for one candidate p."
  [p water land]
  (* (binomial-coefficient (+ water land) water)
     (Math/pow p water)
     (Math/pow (- 1.0 p) land)))

(defn grid-posterior
  "Posterior grid weights for a prior and water/land observations."
  [prior water land]
  (->> probability-grid
       (mapv #(binomial-likelihood % water land))
       (mapv * prior)
       normalize-mean-one))

(def uniform-prior (vec (repeat 201 1.0)))

(def example-posterior
  (grid-posterior uniform-prior 6 3))

{:grid-points (count probability-grid)
 :example :six-water-three-land
 :posterior-mode (first (apply max-key second
                               (map vector probability-grid example-posterior)))}

{:grid-points 201,
 :example :six-water-three-land,
 :posterior-mode 0.665}

For \(W\) water observations and \(L\) land observations, the probability of one particular ordered sequence is

\[p^W(1-p)^L.\]

There are

\[\binom{W+L}{W}=\frac{(W+L)!}{W!L!}\]

orderings with the same counts, giving the binomial likelihood

\[\Pr(W,L\mid p)=\binom{W+L}{W}p^W(1-p)^L.\]

Bayes’ theorem combines that likelihood with the prior:

\[\Pr(p\mid W,L)=\frac{\Pr(W,L\mid p)\Pr(p)}{\Pr(W,L)},\]

where the denominator is the average probability of the data across the prior. On a grid, normalising the products performs the same job.

The simulator restores all four priors from the old application: uniform, step up, step down, and a symmetric ramp. Add water or land deliberately, or let a seeded process whose true water probability is 0.6 generate up to 200 observations. The panels separate the previous posterior, the latest observation’s likelihood, the full likelihood, and the new posterior.

Loading the globe-toss Bayesian update simulator…

The ordered-sequence likelihood and the likelihood of any ordering have the same shape as functions of \(p\): the binomial coefficient is constant for fixed \(W\) and \(L\). Their raw vertical scales differ, but normalising either curve produces the same visual shape. Keeping both panels makes that fact visible rather than leaving it buried in the algebra.

2. Sampling from the posterior and choosing a decision

Section 3.2 of Statistical Rethinking turns the globe problem into a bet. Choose a value \(d\) for the proportion of water. A perfect answer earns \(100\), but the payoff falls in proportion to the absolute error \(|d-p|\). As in the previous application, I make that concrete as a loss of $1 for every 0.01 of error.

The browser first generates a reproducible dataset of 100 globe observations with true \(p=0.6\), then calculates its grid posterior. For every candidate decision \(d\), the expected absolute loss is

\[E[|d-p|\mid W,L] = \sum_p |d-p|\Pr(p\mid W,L).\]

(defn expected-absolute-loss
  "Expected absolute error at decision d for grid weights."
  [posterior d]
  (let [weight-total (reduce + posterior)]
    (/ (reduce + (map (fn [p weight]
                        (* (Math/abs (- d p)) weight))
                      probability-grid
                      posterior))
       weight-total)))

(def example-losses
  (mapv #(expected-absolute-loss example-posterior %)
        probability-grid))

(def example-minimum-loss
  (apply min-key second (map vector probability-grid example-losses)))

{:example :six-water-three-land
 :minimum-loss-decision (first example-minimum-loss)
 :expected-absolute-loss (second example-minimum-loss)}

{:example :six-water-three-land,
 :minimum-loss-decision 0.645,
 :expected-absolute-loss 0.11264775961967216}

The direct calculation iterates over decisions from 0 to 1 in steps of 0.005. For each decision it multiplies the loss at every possible \(p\) by that \(p\)’s posterior weight and averages over the curve. The lowest point of the resulting loss curve is the decision that minimises expected loss.

We can reach the same answer by drawing possible values of \(p\) from the posterior. Under absolute loss, the posterior median is the optimal decision. The interactive view animates up to 2,000 draws, then compares their trace, 200-bin counts, and standardised density with a fixed 10,000-draw simulation. Every visible summary uses the draws shown by that simulation.

Loading the posterior sampling and decision simulator…

3. Updating a Gaussian model for adult heights

The final simulation follows section 4.3 of Statistical Rethinking. A Gaussian distribution has two parameters: its mean \(\mu\) and standard deviation \(\sigma\). There are infinitely many possible pairs, so the original exercise considers a finite grid and ranks each candidate Gaussian by how compatible it is with the observed adult heights.

For an observed height \(h\), a candidate \((\mu,\sigma)\) has density

\[f(h\mid\mu,\sigma)= \frac{1}{\sigma\sqrt{2\pi}} \exp\left(-\frac{(h-\mu)^2}{2\sigma^2}\right).\]

(defn normal-density
  "Gaussian probability density at x."
  [x mean sd]
  (/ (Math/exp (/ (* -0.5 (Math/pow (- x mean) 2.0))
                  (Math/pow sd 2.0)))
     (* (Math/sqrt (* 2.0 Math/PI)) sd)))

(normal-density 151.765 155.0 8.0)

0.0459528391076949

The grid matches the old simulation: 41 values of \(\mu\) from 150 to 160 and 41 values of \(\sigma\) from 7 to 9, or 1,681 candidate Gaussians. The prior is uniform over \(\sigma\) and gives \(\mu\) a Normal(178, 20) density. For each height, the display shows three heat maps:

1. the posterior before considering the next height;
2. the relative likelihood supplied by that height;
3. the posterior after multiplying and normalising.

Move one height at a time or play through the complete dataset. A logarithmic colour scale keeps low but non-zero plausibilities visible after the posterior becomes concentrated.

Loading the sequential Gaussian updating simulator…

What this recreation is for

This is deliberately a record of my earlier learning. It does not silently turn these examples into the vocabulary estimator described in my first Civitas article; it exposes the probability tools that helped me get there.

Sources

• Jamie Pratt and the JointProb group, original interactive simulations and ClojureScript source.
• Richard McElreath, Statistical Rethinking, examples from sections 2.2, 3.2, and 4.3. The prompts above are paraphrases.
• My first Civitas article: Estimating Vocabulary Size with a Simple Bayesian Model.

:verified

source: src/language_learning/vocabulary_estimation/bayes_theorem_simulations.clj