Just as we did for proportions, we may examine a confidence interval to decide whether a proposed value of the population mean is plausible.

Suppose we want to test \( H_0: \mu = \mu_0 \) vs. \( H_a: \mu \neq \mu_0 \) using a significance level of \( \alpha = 0.05 \). An alternative way to perform this test is to find a 95% confidence interval for μ and make the following conclusions:

Example We'll use example 2, in which the alternative was two-sided.

Beyond using the confidence interval as a quick way to carry out the two-sided test, the confidence interval can provide insight into the actual value of the population mean if Ho is rejected.

In the concentration level example, Ho was rejected, and all we could conclude about the mean concentration level of the entire shipment, μ, was that it was not 250.

The 95% confidence interval for μ (244.6, 249.4) gives us an idea of what plausible values for μ would be.

In particular, we can conclude that since the confidence interval lies below 250, at least a large portion of the shipment contains medication that is ineffective.

We are done with the case where the population standard deviation, σ, is known. We now move on to the more common case where σ is unknown.

As we mentioned earlier, only in a few cases is it reasonable to assume that the population standard deviation, σ, is known. The case where σ is unknown is much more common in practice. What can we use to replace σ? If you don't know the population standard deviation, the best you can do is find the sample standard deviation, S, and use it instead of σ. (Note that this is exactly what we did when we discussed confidence intervals).

Is that it? Can we just use S instead of σ, and the rest is the same as the previous case? Unfortunately, it's not that simple, but not very complicated either.

We will first go through the four steps of the t-test for the population mean and explain in what way this test is different from the z-test in the previous case. For comparison purposes, we will then apply the t-test to a variation of the two examples we used in the previous case, and end with an activity where you'll get to carry out the t-test yourself.

Let's start by describing the four steps for the t-test:

Technically, this step only changes slightly compared to what we do in the z-test. However, as you'll see, this small change has important implications. The conditions under which the t-test can be safely carried out are exactly the same as those for the z-test:

Assuming that the conditions are met, we calculate the sample mean \( \overline{x} \) and the sample standard deviation, S (which replaces σ), and summarize the data with a test statistic. As in the z-test, our test statistic will be the standardized score of assuming that \( \mu = \mu_0 \) (Ho is true). The difference here is that we don't know σ, so we use S instead. The test statistic for the t-test for the population mean is therefore:

\( t = \frac{\overline{x} - \mu_0}{\frac{s}{\sqrt{n}}} \)

The change is in the denominator: while in the z-test we divided by the standard deviation of \( \overline{X} \), namely \( \frac{\sigma}{\sqrt{n}} \), here we divide by the standard error of \( \overline{X} \), namely \( \frac{s}{\sqrt{n}} \). Does this have an effect on the rest of the test? Yes. The t-test statistic in the test for the mean does not follow a standard normal distribution. Rather, it follows another bell-shaped distribution called the t distribution. So we first need to introduce you to this new distribution as a general object. Then, we’ll come back to our discussion of the t-test for the mean and how the t-distribution arises in that context.

The t Distribution : We have seen that variables can be visually modeled by many different sorts of shapes, and we call these shapes distributions. Several distributions arise so frequently that they have been given special names, and they have been studied mathematically. So far in the course, the only one we’ve named is the normal distribution, but there are others. One of them is called the t distribution.

The t distribution is another bell-shaped (unimodal and symmetric) distribution, like the normal distribution; and the center of the t distribution is standardized at zero, like the center of the normal distribution.

Like all distributions that are used as probability models, the normal and the t distribution are both scaled, so the total area under each of them is 1.

So how is the t distribution fundamentally different from the normal distribution?

The spread.

Recall that we were discussing the situation of testing for a mean, in the case when σ is unknown. We’ve seen previously that when σ is known, the test statistic is \( z = \frac{\overline{x} - \mu_0}{\frac{\sigma}{\sqrt{n}}} \) (note the σ (σ) in the formula), which follows a normal distribution. But when σ is unknown, the test statistic in the test for a mean becomes \( t = \frac{\overline{x} - \mu_0}{\frac{s}{\sqrt{n}}} \) (note the use of "s" in the formula, in place of the unknown σ). Here is where the t-distribution arises in the context of a test for a mean, because \( t = \frac{\overline{x} - \mu_0}{\frac{s}{\sqrt{n}}} \) (with "s" in the formula in place of the unknown σ) follows a t distribution.

Notice the only difference between the formula for the Z statistic and the formula for the t statistic: In the formula for the Z statistic, σ (the standard deviation of the population) must be known; whereas, when σ isn’t known, then "s" (the standard deviation of the sample data) is used in place of the unknown σ. That’s the change that causes the statistic to be a t statistic.

Why would this single change (using "s" in place of "σ") result in a sampling distribution that is the t distribution instead of the standard normal (Z) distribution? Remember that the t distribution is more appropriate in cases where there is more variability. So why is there more variability when s is used in place of the unknown σ?

Well, remember that σ (σ) is a parameter (it’s the standard deviation of the population), whose value therefore never changes. Whereas, s (the standard deviation of the sample data) varies from sample to sample, and therefore it’s another source of variation. So, using s in place of σ causes the sampling distribution to be the t distribution because of that extra source of variation:

In the formula \( z = \frac{\overline{x} - \mu_0}{\frac{s}{\sqrt{n}}} \), the only source of variation is the sampling variability of the sample mean \( \overline{X} \) (none of the other terms in that formula vary randomly in a given study);

Whereas in the formula \( t = \frac{\overline{x} - \mu_0}{\frac{\sigma}{\sqrt{n}}} \), there are two sources of variation: One source is the sampling variability of the sample mean \( \overline{X} \); The other source is the sampling variability of sample standard deviation s.

So, in a test for a mean, if σ isn’t known, then s is used in place of the unknown σ and that results in the test statistic being a t score.

The t score, in the context of a test for a mean, is summarized by the following figure:

In fact, the t score that arises in the context of a test for a mean is a t score with (n – 1) degrees of freedom. Recall that each t distribution is indexed according to "degrees of freedom." Notice that, in the context of a test for a mean, the degrees of freedom depend on the sample size in the study. Remember that we said that higher degrees of freedom indicate that the t distribution is closer to normal. So in the context of a test for the mean, the larger the sample size, the higher the degrees of freedom, and the closer the t distribution is to a normal z distribution. This is summarized with the notation near the bottom on the following image:

As a result, in the context of a test for a mean, the effect of the t distribution is most important for a study with a relatively small sample size.

We are now done introducing the t distribution. What are implications of all of this?

The p-value of the t-test is found exactly the same way as it is found for the z-test, except that the t distribution is used instead of the Z distribution, as the figures below illustrate.

Comment : Even though tables exist for the different t distributions, we will only use software to do the calculation for us.

Comment : Note that due to the symmetry of the t distribution, for a given value of the test statistic t, the p-value for the two-sided test is twice as large as the p-value of either of the one-sided tests. The same thing happens when p-values are calculated under the t distribution as when they are calculated under the Z distribution.

4. Drawing Conclusions As usual, based on the p-value (and some significance level of choice) we assess the significance of results, and draw our conclusions in context.

To summarize:

For comparison purposes, we use a modified version of the two problems we used in the previous case. We first introduce the modified versions and explain the changes.

Example: 1 The SAT is constructed so that scores have a national average of 500. The distribution is close to normal. The dean of students of Ross College suspects that in recent years the college attracts students who are more quantitatively inclined. A random sample of 4 students entering Ross college had an average math SAT (SAT-M) score of 550, and a sample standard deviation of 100. Does this provide enough evidence for the dean to conclude that the mean SAT-M of all Ross College students is higher than the national mean of 500?

Example: 2

Example 1:

Here is a summary:

Example 2:

1. There are no changes in the hypotheses being tested:

\( H_0: \mu = 250 \) \( H_a: \mu \neq 250 \)

2. The conditions that allow us to use the t-test are met:

(i) The sample is random

(ii) The sample size is large enough for the Central Limit Theorem to apply and ensure the normality of . In other words, we are in the following situation:

3. Finding the p-value.

To find the p-value we use statistical software, and we calculate a p-value of 0.014 with a 95% confidence interval of (244.619, 249.381). For comparison purposes, the output we got when we carried out the z-test for the same problem was a p-value of 0.012 with a 95% confidence interval of (244.648, 249.352).

Note that here the difference between the p-values is quite negligible (0.002). This is not surprising, since the sample size is quite large (n = 100) in which case, as we mentioned, the z-test (in which we are treating s as the known \( \sigma \) ) is a very good approximation to the t-test. Note also how the two 95% confidence intervals are similar (for the same reason).

4. Conclusions:

The p-value is small (0.014) indicating that at the 5% significance level, the results are significant. The data therefore provide evidence to conclude that the mean concentration in entire shipment is not the required 250.

Here is a summary:

Comments

1. The 95% confidence interval for \( \mu \) can be used here in the same way it is used when \( \sigma \) is known: either as a way to conduct the two-sided test (checking whether the null value falls inside or outside the confidence interval) or following a t-test where Ho was rejected (in order to get insight into the value of \( \mu \) ).

2. While it is true that when \( \sigma \) is unknown and for large sample sizes the z-test is a good approximation for the t-test, since we are using software to carry out the t-test anyway, there is not much gain in using the z-test as an approximation instead. We might as well use the more exact t-test regardless of the sample size.

1. In hypothesis testing for the population mean \( \mu \), we distinguish between two cases:

2. In the case when \( \sigma \) is known, the test for \( \mu \) is called the z-test, and in case when \( \sigma \) is unknown and s is used instead, the test is called the t-test.

3. In both cases, the null hypothesis is: \( H_0 : \mu = \mu_0 \) and the alternative, depending on the context, is one of the following: \( H_a : \mu < \mu_0 \), or \( H_a : \mu > \mu_0 \), or \( H_a : \mu \neq \mu_0 \)

4. Both tests can be safely used as long as the following two conditions are met:

(ii) Either the sample size is large (n > 30) or, if not, the variable of interest can be assumed to vary normally in the population.

5. In the z-test, the test statistic is: \( z = \frac{\overline{X} - \mu_0}{\frac{\sigma}{\sqrt{n}}} \) whose null distribution is the standard normal distribution (under which the p-values are calculated).

6. In the t-test, the test statistic is: \( t = \frac{\overline{X} - \mu_0}{\frac{s}{\sqrt{n}}} \) whose null distribution is t(n - 1) (under which the p-values are calculated).

7. For large sample sizes, the z-test is a good approximation for the t-test.

8. Confidence intervals can be used to carry out the two-sided test \( H_0: \mu = \mu_0 vs. H_a : \mu \neq \mu_0 \) , and in cases where Ho is rejected, the confidence interval can give insight into the value of the population mean ().

9. Here is a summary of which test to use under which conditions:

Statistical investigations involve making decisions in the face of uncertainty. So there is always some chance of making a wrong decision. In hypothesis testing, the following decisions can occur:

Example: A Courtroom Analogy for Hypothesis Tests Defendants standing trial for a crime are considered innocent until evidence shows they are guilty. It is the job of the prosecution to present evidence that shows the defendant is guilty “beyond a reasonable doubt.” It is the job of the defense to challenge this evidence and establish a reasonable doubt. The jury weighs the evidence and makes a decision.

When a jury makes a decision, only two verdicts are possible:

Guilty: The jury concludes that there is enough evidence to convict the defendant. The evidence is so strong that there is not a reasonable doubt of the defendant’s guilt.

Not guilty: The jury concludes that there is not enough evidence to conclude beyond a reasonable doubt that the person is guilty. Notice that a verdict of “not guilty” is not a conclusion that the defendant is innocent. This verdict says only that there is not enough evidence to return a guilty verdict.

How is this like a hypothesis test?

The null hypothesis, H0, in American courtrooms is “the defendant is innocent.” The alternative hypothesis, Ha, is “the defendant is guilty.” The evidence presented in the case is the data on which the verdict is based. In a courtroom, the defendant is assumed to be innocent until proven guilty. In a hypothesis test, we assume the null hypothesis is true until the dataindicates otherwise.

The two possible verdicts are similar to the two conclusions that are possible in a hypothesis test.

How does the courtroom analogy relate to Type I and Type II errors?

It would be nice to know when each of these errors is happening when we make our decision about the null hypothesis, but statistical decisions are based on evidence gathered through sampling, and our sampling evidence will sometimes fool us. As long as we are making a decision, we will never be able to eliminate the potential for these two types of errors. Thus, we need to learn how to adjust to the consequences of making these types of errors.

If the significance level is 5 percent (α = 0.05), then 5 percent of the time, we will reject the null hypothesis, even if it is true. Of course we will not know whether the null hypothesis is true. But if it is, the natural variability that we expect in random samples will produce “rare” results 5 percent of the time.

This makes sense, because when we create the sampling distribution, we assume the null hypothesis is true. We look at the variability in random samples selected from the population described by the null hypothesis.

Similarly, if the significance level is 1 percent, then we can expect the sample results to lead us to reject the null hypothesis 1 percent of the time. In other words, about one in 100 data sets would show “rare” results that contrast with the other 99 data sets, leading us to reject the null hypothesis. If the null hypothesis is actually true, then by chance alone, we will reject a true null hypothesis 1 percent of the time. So the probability of a type I error in this case is 1 percent.

In general, the probability of a type I error is α.

What is the probability that we will make a Type II error?

As you have just seen, the probability of a type I error is equal to the significance level, α. The probability of a type II error is much more complicated to calculate, but it is inversely related to the probability of making a type I error. Thus, reducing the chance of making a type II error causes an increase in the likelihood of a type I error.

Decreasing the Chance of Type I or Type II Error

How can we decrease the chance of a type I or type II error? Well, decreasing the chance of a type I error increases the chance of a type II error, so we must weigh the consequences of these errors before deciding how to proceed.

Recall that the probability of committing a type I error is α. Why is this? Well, when we choose a level of significance (α), we are choosing a benchmark for rejecting the null hypothesis. If the null hypothesis is true, then the probability that we will reject a true null hypothesis is α. So the smaller α is, the smaller the probability of a type I error there will be.

It is more complicated to calculate the probability of a type II error. The best way to reduce the probability of a type II error is to increase the sample size. But once the sample size is set, larger values of α will decrease the probability of a type II error while increasing the probability of a type I error.

General guidelines for choosing a level of significance:

It is important to remember to take into consideration the possibility of the occurrence of Type I or Type II error, when drawing conclusions from hypothesis tests. Thus:

However, it is not possible to know when either Type I or Type II error has occurred.