Hypothesis testing answers the more focused question "Based on sample data, is the hypothetical number Uo a reasonable guess for the mean of the population from which the sample came?"
If the sample mean is so high that if forces the 95%
one-sided lower
bound (found by subtracting the 90% two-sided
margin of error from the sample mean) to be above U0,
we can
be 95% confidence that the mean of the sample's parent population is
also above
Uo. Statisticians call this "rejecting the hypothesis U<Uo at
the
5% significance level." Significance is just 1 minus confidence; the
lower
the significance, the higher the confidence.
If thesample mean is so low that it forces the 95% one-sided
upper
bound (found by adding the 90% two-sided margin
of error from the sample mean) to be below Uo, we can be 95%
confidence that the mean of the sample's parent population is also below
Uo. Statisticians call this "rejecting the hypothesis U>Uo at
the
5% significance level." Significance is just 1 minus confidence; the
lower
the significance, the higher the confidence.
If Uo is outside the 90% "between" confidence interval (either above
the 90% two-sided upper bound or below the 90% two-sided
lower
bound), we can be 90% confidence that the mean of the sample's parent
population
is different from Uo. Statisticians call this "rejecting the
hypothesis
U=Uo at the 10% significance level."
Often we need to know whether two populations have the same population mean or whether they each have different population means.
If we take a sample from each population, the
sample means will almost certainly be at least a little different
whether the population
means are the same or not. To do a rough test of whether or not
we can be
95% confident that the population means are different, set up three
columns
of data in Excel: one for each sample, and a longer column for the two
samples
combined. Then use the Descriptive Statistics tool with a
confidence level
of .95. If the absolute value of the difference between the two
sample means
is much greater than the margin of error (miscalled "confidence level"
by
Excel) for the combined sample, there is good evidence that the two
population
means are different. If the absolute value of the difference
between the
two sample means is much smaller than the margin of error for the
combined
sample, there is no significant evidence that the two population means
are
different. (Note this is not the same as evidence that they are the
same!)
If the absolute value of the difference between the two sample means is
fairly
close to the margin of error for the combined sample, collect more data
or
use an exact method from a statistics textbook.
(The two populations can be naturally occurring populations, but very often one population is the future values of some large collection of people or things if none of them receive a new treatment and the other population is the future values of the same collection if they all receive the new treatment. This is the hypothesis we are testing when we use a control group to represent the first "population" and an experimental group to represent the second.)