Galton’s Bayesian Machine

by on March 5, 2011 at 7:25 am in History, Science | Permalink

Stephen Stigler has a cool piece on a machine that Francis Galton built in 1877 that calculated a posterior distribution from a prior and a likelihood function. Galton's originality continues to astound.

Here is Stigler:

StigFigure1 The machine is reproduced in Figure 1 from the original publication. It depicts the fundamental calculation of Bayesian inference: the determination of a posterior distribution from a prior distribution and a likelihood function. Look carefully at the picture–notice it shows the upper portion as three-dimensional, with a glass front and a depth of about four inches. There are cardboard dividers to keep the beads from settling into a flat pattern, and the drawing exaggerates the smoothness of the heap from left to right, something like a normal curve. We could think of the top layer as showing the prior distribution p(θ) as a population of beads representing, say, potential values for θ, from low (left) to high (right)….

…the beads fall to the next lower level. On that second level, you can see what is intended to be a vertical screen, or wall, that is close to the glass front at both the left and the right, but recedes to the rear in the middle.

…The way the machine works its magic is that those beads to the front of the screen are retained as shown; those falling behind are rejected and discarded. (You might think of this stage as doing rejection sampling from the upper stage.)

…The final stage turns this into a standard histogram: The second support platform is removed by pulling to the right on its knob, and the beads fall to a slanted platform immediately below, rolling then to the lowest level, where the depth is again uniform–about one inch deep from the glass in front. This simply rescales the retained beads… the magic of the machine is that this lowest level is proportional to the posterior distribution!

Hat tip: The Endeavour.

Bill March 5, 2011 at 5:32 am

If the financial analysts on Wall Street just had had this machine we could have avoided the financial crisis!

Or, is it because they had models like this that we had the financial crisis?

Or, is it because we trust and marvel at the machine too much?

Six Ounces March 5, 2011 at 9:59 am

We'd have to build a machine with fatter tails, Bill.

Or better yet, just observe the amount and quality of food served by a Chinese restaurant, the operations and cleanliness in its kitchen, the contents of its dumpster, the number of fliers they distribute, the zoning laws, the size of the SBA loan they got, the number of phony and genuine Yelp reviews, and the size of the bribe they paid to the corrupt health inspector.

There are millions of tiny asset, business, financial, and budget crises going on every day.

Bill March 5, 2011 at 10:13 am

Six Ounces,

Agree with the fatter tails. Maybe we should add a government backstop too in case the beads fall out of the machine and onto the floor in case we reach a systemic six sigma event.

Andrew March 5, 2011 at 11:43 am

The government punishes safety margin.

Nice machine. Would make a good school project.

k March 5, 2011 at 12:10 pm

whatever. this pushes the concept of bayesian sampling in a simple yet profound way.

if this gets used more generally maybe bayesian econometrics can be regarded as equal to the frequentist approach. and future generations will wonder what the fuss was all about. it will appear "obvious".

Ricardo March 7, 2011 at 2:19 am

The more Darwin gets sanctified to stick it to the Red Staters, the more Galton gets demonized to take the blame for all the bad things that flowed from the Darwinian revolution.

Eugenics did not "flow from the Darwinian revolution." If it flowed from anything scientific, it flowed applied statistics (another of Galton's fields) and from the least original part of Darwin's "On the Origin of Species": Chapter 1, Variation Under Domestication. But as Steve Sailer almost certainly knows, people had been musing about inherited differences in intelligence between different populations for at least 100 years before through the work of botanist Carl Linnaeus.

Neither can be called an "amateur" but Galton's wish, among other things, to sterilize "bohemians" ranks with one of the dumber things he ever said.

Comments on this entry are closed.

Previous post:

Next post: