The latest rumour is that both ATLAS and CMS have evidence that the Higgs mass is about 125 GeV/C2 at confidence levels of 3.5σ and 2.5σ respectively. At 3.5σ, the measurement could be the result of a random fluke just 0.1% of the time whereas at 2.5σ the fluke factor is about 1%.

If you are really optimistic, I believe you can add these two results together in quadrature to get an overall result with a significance of 4.3σ.

While these might sound like fantastic odds to you and me, particle physicists normally wait until they have a confidence of 5σ or greater before they call it a “discovery”. Anything over 3σ is described as “evidence”.

…Crease wisely cites past experience as the number-one reason for caution. Indeed he quotes University of Oxford physicist and data-analysis guru Louis Lyons as saying “We have all too often seen interesting effects at the 3σ or 4σ level go away as more data are collected.”

As Crease points out, nearly everyone he spoke to in writing his article “had tales – many well known – of signals that went away, some at 3σ: proton decay, monopoles, the pentaquark, an excess at Fermilab of high-transverse-momentum jets”.

Here is more, and for the pointer I thank Ken Regan.  Here are the InTrade markets, which are pricing observation before 2014 at 88.

We now return to your regularly scheduled programming.


Not sure this prediction market holds much weight. Take a look at this:

Higgs Boson Particle to be observed on/before 31 Dec 2013
Event: Observation of the Higgs Boson Particle

Higgs Boson Particle to be observed on/before 31 Dec 2014
Event: Observation of the Higgs Boson Particle

Irrespective of whether the Boson gets detected or not; shouldn't there be an arbitrage strategy to make easy money from this apparent inconsistency? I can't figure out how though.

Exactly, so one of the two must be must be mis-priced. If you look at the volume, there doesn't appear to have been any trades recently so I think this just won't be an accurate prediction until more people bet.

It's not an arbitrage opportunity, but it is an opportunity for a middle. Sell contracts at 88% and buy at 60%. Then you win one, lose one if it is discovered before 3 years or after 4 years, but win both if it is discovered between 3 and 4. Of course the long wait for the result reduces your return, which is probably why the liquidity is thin. And thin liquidity means this might not work, since you will need a match for each contract.

Although, you can get out of your position at any time by buying/selling your contracts if you can find a match, which might allow you to do the buy/sell outlined above and then when the prices normalize, get out and take your "arbitrage" profits- but it all depends on the liquidity.

The 2014 bid-ask prices are 25-90% and for 2013 they are 30-80%, so there is no arbitrage opportunity. The last trade price for 2013 isn't even in range! The last 2014 trade is more than a month old.

Even for the more popular bets on politics, prediction markets are usually much too thinly-traded to be taken seriously... yet nevertheless often are.

Nice. Our standards are rather different !

Justin, is that an artifact of low market volume?

That's really the only reasonable explanation. The biggest flaws in prediction markets are low probability events and inadequate volume. I think Robin Hanson has some research showing the volume doesn't need to be very high to get accurate predictions, but what other reason could there be for such an obvious arbitrage opportunity?

I just checked and the volume recently on these trades is 0. Maybe there doesn't have to be much volume for an accurate prediction, but surely there must be some.

Are you (and mostly everybody since the first post) really arguing that if there were more people on the market they would get the right answer?!

Yes. This isn't about reading the mind of God; merely about objectively (and anonymously) evaluating claims, evidence and insider information that the scientists themselves might have incentive to publicly lie about.

You certainly cannot just add the significances together.

And the reason physicists require 5 sigma significance is (simply put) that the number of different particles which could possibly be found is so large, that at 3 sigma significance (99.7% rejection of background) you would expect quite a few particles show up.

Yes, another way to put it is in a data set with a thousand observables you expect some three sigma anomalies. The flip side is: if you have a theoretical reason to look for a particular signal, your statistics return to their normal interpretation.

For example, if I have a large data set including exposure to a large set of substances and cancer rates, I should not be surprised to find some three sigma correlations and correspondingly they shouldn't be treated as evidence that a given substance causes cancer. On the other hand, if I have a theoretical reason to believe a certain substance causes cancer (due to microbiology or chemistry or whatever), then if after the fact I look at the data set and find three sigma evidence in favor, that's good evidence.

This Higgs is a bit more tricky because we have theoretical reason to think it's there but we don't have strong theoretical reason to sharply constrain its mass.

You can't add those measurements in quadrature.

What's the right way to combine confidence intervals?

You can't do it without knowing what the underlying data looks like. For instance, it is possible that the CMS data is centered around 123 GeV and the ATLAS data is around 127 GeV, then the combined bell-curve (compared to combined background) would actually look quite weird - it will have some significance, but it might not be significantly greater than the significance of the better experiment.

But from the report it seems both are centered at 125 GeV just with different confidence levels. That's what I am talking about.

If Expt A and Expt B both indicate 125 GeV but with 3.5 sigma and 2.5 sigma; what is the net expectation of sigma.

See "meta-analysis" for legitimate ways to combine these. The statement about adding them together is mathematically ignorant, and undercuts the idea that the author has any idea what they are discussing.

That article would suck a great deal less if the author bothered to explain the relevance/implications of the mass's proving to be 125 GeV/C^2 rather than some other number.

Ahh, the terribly non-objective reality of science. I suspect the person who first used the term "green shoots" was also the first to distinguish between "discovery" and "evidence"

Comments for this post are closed