Confidence Intervals & Hypothesis Testing:
A Quick & Dirty Approach
 

Connfidence intervals and hypothesis tasting are the key elements of "Inferential statistics." A confidence interval answers the question "Based on sample data, what numbers would & would not be a reasonable guess for the mean of the population from which the sample came?"

The concept of a reasonable guess requires some definition.  Often in business we only  need to guess an upper or lower bound on a number: "the mean operating cost is under ˆ500 per day" or "the mean pages printed per ink cartridge is over 5000."  Sometimes we need to guess  both; this is what statisticians call a two sided interval: "the mean productivity is between 7 and 9 widgets per hour."

A guess qualities as "reasonable" if it was calculated from sample data using a procedure which, despite the inevitable uncertainty of generalizing from a sample to its parent population, is nevertheless guaranteed to be correct in a specified proportions of applications, usually 90% or 95%.

For technical reasons statisticians refuse to call the specified proportion the "probabilitly" that the unknown population mean is in the calculated interval, preferring the phrase "confidence level," but for most practical purposes the difference is inconsequential.

The Descriptive Statistics tool in Excel's Data Analysis add-in provides a very useful number with a very misleading label.  What Excel calls "confidence level" is in fact the margin of error for a two sided confidence interval for the mean of the sample's parent population.  If the specified confidence level is 90%,

The reason the confidence is lower for the two sided interval is that there's a 5% possibility of error on the low side and also a 5% possibility of error on the high side so the confidence is only 90%.  If you specify an 95% (two sided) confidence level to the Descriptive Statistics toolthe corresponding one sided confidence intervals will be at 97.5% confidence.

The margin of error given by Excel's Descriptive Statistics tool (under the misnomer "confidence level") is only strictly accurate if the parent population is normally distributed.  Statistics textbooks detail specific ways to construct confidence intervals in many specific circumstances, but using the margin of error from Excel's Descriptive Statistics tool will very often be good enough for business  decision making.



Hypothesis Testing

Hypothesis testing answers the more focused question "Based on sample data, is the hypothetical number Uo a reasonable guess for the mean of the population from which the sample came?"
 




A rough test of whether two populations are the same

 Often we need to know if two populations have the same population mean or different population means.  (These can be naturally occurring populations, but very often one population is some group without a new treatment and the other is the same group with the new treatment, like a control group and an experimental group.)

If we take a sample from each population, the sample means will almost certainly be at least a little different whether the population means are the same or not.  To do a rough test of whether or not we can be 95% confident that the population means are different, set up three columns of data in Excel: one for each sample, and a longer column for the two samples combined, and use the Descriptive Statistics tool with a confidence level of .95.  If the absolute value of the difference between the two sample means is much greater than the margin of error (miscalled "confidence level" by Excel) for the combined sample, there is good evidence that the two population means are different.  If the absolute value of the difference between the two sample means is much smaller than the margin of error for the combined sample, there is no significant evidence that the two population means are different. (Note this is not the same as evidence that they are the same!)  If the absolute value of the difference between the two sample means is fairly close to the margin of error for the combined sample, collect more data or use an exact method from a statistics textbook.