What Does a 95% Confidence Interval Really Mean?
Why a Range Tells You More Than a Single Number
What does “95% confidence” actually promise you?
Say a study reports a treatment effect with a 95% confidence interval of [2.1, 8.4]. Most people read that and think: “there’s a 95% chance the true effect is somewhere in that range.” That’s the single most common misunderstanding in all of statistics, and it’s not quite right. Let’s unpack what a confidence interval actually tells you.
What a Confidence Interval Really Is
In our last post, we talked about the p-value: a single number that answers a yes-or-no question — is this result surprising if there’s truly no effect? A confidence interval asks a more useful question: what’s a plausible range for the true effect, based on the data you collected?
It’s built from the same foundation as everything else in this series: the normal distribution and the Central Limit Theorem. Because sample averages tend to follow a predictable, bell-shaped pattern, we can use that pattern to say: “given what we observed, here’s the range where the true value most likely falls.”
What “95%” Actually Means
Here’s the part that trips almost everyone up: a 95% confidence interval does not mean there’s a 95% chance the true effect is inside this particular range. The true effect is a fixed number — it either is or isn’t in the interval; there’s no probability left to talk about once the data is collected.
What “95%” really describes is the method: if you repeated the same study 100 times and built a new confidence interval each time, about 95 of those 100 intervals would contain the true value.
A 95% confidence interval is a statement about the reliability of the method, not a probability about this one result.
Why Width Matters
The width of a confidence interval tells you how precise your estimate is.
A narrow interval (like [3.9, 4.3]) means your estimate is precise — the data strongly points to a specific range.
A wide interval (like [0.5, 15.2]) means there’s a lot of uncertainty — the true effect could be small, large, or almost anything in between.
Width is driven mostly by sample size. Small studies produce wide, uncertain intervals; larger studies narrow them down. We’ll get into exactly how much sample size matters in the next post.
Confidence Intervals vs. P-Values
The two are closely related. If a 95% confidence interval for a difference excludes zero (no effect), that lines up with a p-value below 0.05. But the confidence interval gives you something the p-value can’t: a sense of how big the effect might be, and how precisely you’ve measured it.
A p-value can only tell you “significant” or “not significant.” A confidence interval can tell you “significant, and probably somewhere between a small and moderate effect” — which is almost always more useful in medicine.
Key Takeaways
A confidence interval gives a plausible range for the true effect, not just a yes/no answer.
“95% confidence” describes the reliability of the method across repeated studies, not the probability that this specific interval is correct.
A narrower interval means a more precise estimate; a wider interval means more uncertainty, usually from a smaller sample.
When a confidence interval excludes “no effect,” that’s consistent with a significant p-value, but the interval also tells you the likely size of the effect.
Up Next:
Next, we’ll look at statistical power and sample size — how to think about “how many patients do I actually need?” before a study even begins.