Confidence Level 99% vs 95% in Statistical Analysis

Hi to all the experts in Elsmar, my name is Hazwan. I'm struggling to understand why 95% confidence interval is more commonly used compare to 99% confidence interval. I did many search in Google but i still could not understand why. Isn't 99% provides more confidence?. Somebody please explain...

The first best answer is that:
Confidence = 1-alpha
alpha is traditionally taken to be 5% or .05. Alpha is essentially the ‘risk’ or probability of being wrong if you decide that there is a difference when no difference exists. There is no such thing as the statistical probability of being right. (There is no real scientific reason for this specific number other than it’s what has become tradition and so no one really questions it.*)
So a confidence level of 95% is traditionally used. Because that’s what everybody else does.

While you are correct that 99% provides a higher ‘confidence’ you must remember that statistical confidence doesn’t have the same meaning as normal people ascribe to confidence. A 99% confidence interval is LONGER than a 95% confidence interval and that confuses some people As it is somewhat counterintuitive.


*Fisher had a reason for it but that has been misunderstood and repeated with out critical thought for a hundred years. Fisher said that 5% was a reasonable ‘probability‘ of being wrong over many independent trials. He didn’t specify how many independent trials (which is commonly referred to as replication) were enough. The use of 5% is purely ritual…

I teach that your level of confidence (1-alpha) should be based on the consequences of your being wrong. If the consequences are low and you plan to/can run additional experiments to replicate your findings, use a lower confidence level such as 90% during the exploratory phase. If the consequences are high and you cannot afford or have time to replicate the findings, use a higher confidence level such as 99%.

In addition to that (Miner’s post) I teach that replicated results trumps confidence intervals and p values for a single result.

Thank you @Bev D and @Miner . This somewhat gives me the insight of this. Just one last silly question, really sorry if it's too idiotic. From your experience, what is the highest level of confidence ever used?. Does 99.99% confidence level ever used or generally the highest used is exactly 99%. Because i understand there is no way for us to know the actual population parameter. I'm aware that confidence interval is just an estimate of the population from the sample data which is why 100% confidence level is not realistic. But how about 99.99%?.

hazwan2283 said:

Just one last silly question, really sorry if it's too idiotic. From your experience, what is the highest level of confidence ever used?

Click to expand...

In the world of elementary particle physics, it is traditional to use a "5-sigma confidence" to announce "discovery".

Higher levels have been used for certain kinds of severe effects (.001 alpha). But really in these cases replication is preferred. Single point estimates are notoriously subject the ravages of non-homogeneity and incorrectly specifying the sampling frame. In other words its not the confidence interval that gives you true (in the gut gonna eat a train full of dioxin if I’m wrong) confidence…it’s the structure of the test that gives real people actual confidence.