Adjusting My Confidence as Evidence Arrives

Why it attracts me

I like that this idea turns uncertainty into something you can work with. Instead of arguing about whether a claim is true, you can ask how confident to be, and what would change that.

The idea

Start with a rough degree of confidence in an idea. When evidence arrives, compare two chances: how likely you were to see it if the idea is true, and how likely if it is false. If the evidence is much more likely under the idea, your confidence should rise a lot. If the two chances are similar, the evidence barely matters. This is called Bayesian updating, after Thomas Bayes, and his rule makes the arithmetic exact.

An example

The Process Stability Lab is not a Bayesian tool. It uses a control chart, a different statistical method. But it trains the same instinct. Each dot is the tightness of one bolt, and the lab warns that its alarm rules go off on about 1 in 100 normal parts. So one odd dot is weak evidence that anything has changed. A pattern that persists is much stronger evidence, and that is when the lab says to act. Learning to tell those apart is learning how far each reading should move you.

Where it connects

This gives Probability, Risk & Uncertainty a precise language for uncertain choices. It sits next to quantum probability in The Rule Behind Quantum Probabilities, but it is not the same thing: in quantum physics the theory sets the odds for each experiment, while here the odds are my own estimates.

Questions I'm still exploring

• Where do my starting guesses come from, and how much do they shape where I end up?
• Which of my beliefs have I never put even a rough number on, and why?
• When is it reasonable to act on a belief I hold at only 60 percent?

Sources and further reading

• James Joyce, "Bayes' Theorem", Stanford Encyclopedia of Philosophy