This lesson explores the idea of a sampling distribution which is the theoretical distribution of a sample statistic. We discuss the theoretical concepts that allow us to make confident conclusions about an entire population based on a single sample.
By the end of this lesson, you should be able to:
A Population Parameter is a numerical summary (mean, median, standard deviation, proportion, etc.) based on the entire population.
A Sample Statistic is any numerical summary (mean, median, standard deviation, proportion, etc.) based on a sample from a population.
A Sample is a subset of the entire population.
Sample Size is the number of individuals in the sample.
Randomization is used in the collection of a sample to minimize bias and make sure our sample is representative of the population.
Randomization guarantees that if someone else tries to recreate our study, they will get a different sample. Consequently, their sample statistics will not be the same as ours!
While this should not be surprising, it does create a problem. Given sample-to-sample differences, how can we know when our conclusions are valid? We typically only have one sample to go on.
The good news is that we can rely on the theory of sampling distributions to quantify the uncertainty about our sample.
Imagine you and 10 friends want to know about the what percent of BYU-I students whose favorite soda is Dr. Pepper. Each of you could go and survey 50 people each and ask about their favorite soda. All 11 of you calculate the proportion of students who prefer Dr. Pepper.
You’ll all get different results because you’ll be asking different sets of people. Now, instead of looking at your percents individually, summarize the collection of samples from all 11 of you using favstats() and a histogram:
library(mosaic)
library(tidyverse)
library(car)
library(rio)
proportions <- c(0.14, 0.08, 0.02, 0.06, 0.06, 0.02, 0.00, 0.14, 0.00, 0.08, 0.18)
favstats(proportions) %>% knitr::kable()| min | Q1 | median | Q3 | max | mean | sd | n | missing | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 0.02 | 0.06 | 0.11 | 0.18 | 0.0709091 | 0.0609023 | 11 | 0 |
histogram(proportions, main = "Distribution of the % of Students who Prefer Dr. Pepper out of 50")
The above histogram represents 11 different samples of nearly infinite possible samples of 50 BYU-I students.
The Sampling Distribution is the distribution of a sample statistic for all possible samples from a population. It’s completely theoretical, but there are some mathematically provable attributes of this theoretical distribution that make it possible to make inference about an unknown population parameter based on a single sample.
We will discuss these theoretical properties in greater detail below. Before we do, use your intuition to answer the following question.
QUESTION: Given the 11 independent samples of 50 students, what would be your best guess of the True population proportion of students who prefer Dr. Pepper?
NOTE: Because these results are based on a simulation, the true proportion is known.
If you guessed the mean of the 11 samples (0.0709) you’d be VERY close to the truth, which is 7%. Even though none of the individual researchers got 7%, their average was close to the truth! (It’s actually impossible to get exactly 7% when only asking 50 students.)
It turns out that the mean of the sampling distribution of a sample statistic is equal to the population parameter!
The Central Limit Theorem allows us to know not only know about the mean, but the entire shape of the sampling distribution for a mean.
For any population with a mean, \(\mu\), and a standard deviation, \(\sigma\):
The Central Limit Theorem states that with a sufficiently large sample size, the sampling distribution of the sample mean will be approximately normal with a mean equal to the population mean, \(\mu\), and a standard deviation equal to the population standard deviation, \(\frac{\sigma}{\sqrt{n}}\) where \(n\) is the number of individuals in the sample.
While this applies to any sample statistic (even the standard deviation!), we are most often interested in the sampling distribution of a sample mean, \(\bar{x}\).
How many observations are required so that the Central Limit Theorem will assure that the distribution of sample means will be approximately normal? The answer is, “it depends.”
For this course, if the sample size is at least 30, we will conclude that the sampling distribution of \(\bar x\) will be approximately normal.
The following facts are always true. They do not depend on the Central Limit Theorem. They do not depend on the sample size. These facts hold for the sample mean \(\bar x\) of any simple random sample of size \(n\) drawn from a population with mean \(\mu\) and standard deviation \(\sigma\).
Remember, these facts are always true. They do not depend on normality or the sample size. So, even though these facts are often used in conjunction with the Central Limit Theorem, they do not depend on it.
Think about the simulations you have observed. The mean of the sample means was always aligned with \(\mu\). Also, if you took samples larger than \(n = 1\), the standard deviation of sample means was always smaller than \(\sigma\).
\[ \mu = 8.7~\text{minutes} \]
Suppose researchers treated 6 patients with a new cancer treatment. Their estimated 5-year survival rate was estimated to be 50% higher than standard treatment. How confident would you be in my results?
While it is possible that the new treatment is wildly more effective than standard treatments, six patients is not a lot. It is possible to get six patients that might be more resilient than average, just by chance. Six people can hardly be a robust, representative of the whole population.
My sample mean survival rate is not likely to be very close to the true population survival rate.
The Law of Large Numbers states that the sample mean, \(\bar{x}\), will become closer to \(\mu\) as the sample size \(n\) becomes larger.
The result you have discovered in the previous two questions is called the Law of Large Numbers. The Law of Large Numbers states that if the sample size is large, then the sample mean will typically be close to the population mean, \(\mu\). This happens because the standard deviation \(\sigma / \sqrt{n}\) will get smaller as the sample size \(n\) increases.
NOTE: This is very different from the Central Limit Theorem. The Central Limit Theorem is a statement about the DISTRIBUTION of all possible sample means, that if the sample size is large that \(\bar x\) will be approximately normal. The Law of Large numbers states that the sample mean \(\bar x\) will be close to \(\mu\).
Take a moment to study the difference between the Central Limit Theorem and the Law of Large Numbers. They are very different, but it is easy to mix them up when you are first learning about them.
In order to “do statistics” we need to compute probabilities. This requires we know the distribution of the population of interest. The most important distribution in this course is the normal distribution. In this lesson, we are interested in another very important distribution: the distribution of all the sample means, \(\bar{x}\). In order to do statistics with \(\bar{x}\) we need the sampling distribution of \(\bar{x}\) to be normal. There are two situations when the distribution of \(\bar{x}\) is guaranteed to be normal (or at least very close to normal). They are:
The mean and standard deviation of \(\bar{x}\) are:
The distribution of sample means is a distribution of all possible sample means (\(\bar x\)) for a particular sample size.
The Central Limit Theorem states that the sampling distribution of the sample mean will be approximately normal if the sample size \(n\) of a sample is sufficiently large. In this class, \(n\ge 30\) is considered to be sufficiently large.
The mean of the distribution of sample means is the mean \(\mu\) of the population: \(\mu_{\bar{x}} = \mu\).
The standard deviation of the distribution of sample means is the standard deviation \(\sigma\) of the population divided by the square root of \(n\): \(\sigma_{\bar{x}} = \sigma/\sqrt{n}\).
The distribution of sample means is normal in either of two situations: (1) when the data is normally distributed or (2) when, thanks to the Central Limit Theorem (CLT), our sample size (\(n\)) is large.
The Law of Large Numbers states that as the sample size (\(n\)) gets larger, the sample mean (\(\bar x\)) will get closer to the population mean (\(\mu\)). This can be seen in the equation for \(\sigma_{\bar{x}} = \sigma/\sqrt{n}\). Notice as \(n\) increases, then \(\sigma_\bar{x}\) will get smaller.