Performing T-Tests in R

     To begin, I am going to set up the data. The fake variables created will represent the cost of Eggs and Milk at various grocery stores. The set.seed() function will allow the rnorm() functions to return the same values for you as they have for me. The rnorm() functions will each generate 10 values from a normal distribution with each Eggs having a population mean of 5 and Milk having a population mean of 4. The default standard deviation is 1. If you want to use your own data simply use the c() to create 1 or 2 vectors for use below, you can use the example variable as a reference.

# Setting up the data
set.seed(123)

Eggs = rnorm(10, mean = 5)
Milk = rnorm(10, mean = 4)

example = c(1,2,3,4,5,6,7,8,9)

     We first want to know if it is reasonable to believe that the data we have is normally distributed. Many people would use a histogram or statistical test for this, while both can be valid, a preferred method is using what is called a QQ-plot. A QQ-plot compares the density of your variable to the density of an ideal standard normal distribution to determine if they are similar. I am introducing a function that standardizes the distribution (or turns it into z-scores) to match the mean and standard deviation of the comparison distribution. What we actually care about is that the points on the graphs fall near the line uniformly, more tips for interpreting QQ-plots can be found starting on pg.46 of my book here https://www.statswithr.com/neural-networks-with-keras-in-r-a-quickstart-guide. The plots below are representative of what you can expect a normally distributed variable to look like with a small sample size. With a sample size less than 30, points generally falling close to the line is sufficient. If the data is determined to not be normal, you will need to either transform the data or use a non-parametric version of the t-test which comes at a loss of statistical power (ability to find statistical significance).

# Transforms the variable to have mean 0 and sd 1
standardize <- function(x){return((x-mean(x))/sd(x))}

# qqnorm makes the plot, abline draws the line
qqnorm(standardize(Eggs), main = "QQ Plot of Eggs")
abline(0,1)

# qqnorm makes the plot, abline draws the line
qqnorm(standardize(Milk), main = "QQ Plot of Milk")
abline(0,1)

     Lets start with the one sample t-test, where we test the hypothesis that the population mean of Eggs is 4. We know that the population mean is actually 5 (because we set it that way), so we expect to reject the null hypothesis assuming our sample size is sufficiently large. Below we are able to reject the null hypothesis and determine that the population mean is likely not 4. Notice that the degrees of freedom is n-1, and that the confidence interval does not include 4.

# One sample t-test
t.test(Eggs, mu = 4)

## 
##  One Sample t-test
## 
## data:  Eggs
## t = 3.5629, df = 9, p-value = 0.006091
## alternative hypothesis: true mean is not equal to 4
## 95 percent confidence interval:
##  4.392330 5.756922
## sample estimates:
## mean of x 
##  5.074626

     Before we continue with two sample t-tests, we will want to get an idea if the variances are equal. With this generated data, we know that they are, but we should not assume that. var.test() can give us an idea if that assumption is met. If this test is not statistically significant, the equality of variance can be assumed. If the variances are found to likely not be equal, we can still perform t-tests, we will simply need to assume unequal variance in the t.test() function.

var.test(Eggs, Milk)

## 
##  F test to compare two variances
## 
## data:  Eggs and Milk
## F = 0.8442, num df = 9, denom df = 9, p-value = 0.8049
## alternative hypothesis: true ratio of variances is not equal to 1
## 95 percent confidence interval:
##  0.2096867 3.3987337
## sample estimates:
## ratio of variances 
##          0.8441974

     Now, let’s perform a two sample t-test; the default setting assumes unequal variance, which means the degrees of freedom are reduced slightly (17.872 instead of 18). Use this if the prior test is statistically significant with the p-value < 0.05. Note that doing this makes it slightly more difficult to find an effect. This test was not statistically significant with a p-value of 0.06799, which is of course above 0.05. We fail to reject the null hypothesis that the population means between Eggs and Milk are equivalent.

# basic t-test assuming unequal variance
t.test(Eggs,Milk)

## 
##  Welch Two Sample t-test
## 
## data:  Eggs and Milk
## t = 1.9426, df = 17.872, p-value = 0.06799
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.07104883  1.80305620
## sample estimates:
## mean of x mean of y 
##  5.074626  4.208622

     Now, when we assume equal variance our degrees of freedom is exactly 18 or n-2. We once again fail to reject the null hypothesis; however, you will notice that the p-value here is slightly lower than the previous p-value by 0.00012. This means that we were just a little closer to rejecting the null hypothesis, which is related to the test being slightly more powerful in the statistical sense.

# assume equal variance
t.test(Eggs, Milk, var.equal = TRUE)

## 
##  Two Sample t-test
## 
## data:  Eggs and Milk
## t = 1.9426, df = 18, p-value = 0.06787
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.07056945  1.80257682
## sample estimates:
## mean of x mean of y 
##  5.074626  4.208622

     Until now we have been doing what are called two-tailed tests meaning that we are simply testing if the population means of Milk and Eggs are different in some way. If we want to test the hypothesis that the population mean of Eggs is greater than the population mean of Milk we have to change the “alternative” argument of the function to greater. Note that the order of Eggs and Milk matters in this situation; if Milk were first in the function below, we would be testing the opposite hypothesis. The t-test is finally significant with the p-value < 0.05. Directional hypothesis tests have more power to find effects, but it is important that we have a prior hypothesis to justify the use of a directional hypothesis test. Depending on the application, it is more common to use two-tailed hypothesis tests, as one-tailed tests can be looked upon negatively.

# Eggs greater than Milk
t.test(Eggs, Milk, alternative = "greater")

## 
##  Welch Two Sample t-test
## 
## data:  Eggs and Milk
## t = 1.9426, df = 17.872, p-value = 0.03399
## alternative hypothesis: true difference in means is greater than 0
## 95 percent confidence interval:
##  0.0926739       Inf
## sample estimates:
## mean of x mean of y 
##  5.074626  4.208622

Now, we will test the opposite hypothesis. As you may have guessed, this is nowhere near significant, with the p-value being equal to 1 minus the previous p-value. If you get a large p-value like this, it is worth checking if you specified the correct direction in your function.

# Eggs less than Milk
t.test(Eggs, Milk, alternative = "less")

## 
##  Welch Two Sample t-test
## 
## data:  Eggs and Milk
## t = 1.9426, df = 17.872, p-value = 0.966
## alternative hypothesis: true difference in means is less than 0
## 95 percent confidence interval:
##      -Inf 1.639333
## sample estimates:
## mean of x mean of y 
##  5.074626  4.208622

     Finally, we will perform a paired samples t-test. The two variables we have been using are not appropriate for this type of analysis, so I am creating a variable for the milk price next week that depends on the current milk price. Paired samples t-tests are meant to be performed on data that are not independent samples. A common form of this is a test retest situation where the same individuals are tested twice, before and after some manipulation. A regular t-test is inappropriate because the samples are not independent, so we use a paired sample t-test that accounts for the same individuals being tested twice. In this example, the milk price is being observed and then we are waiting a week and re-observing them. The p-value < 0.05, so we can say that there is a statistically significant difference between weeks, and the 95% confidence interval for the increase ranges from 0.39 and 1.76 for the population means.

# New variable milk next week
set.seed(123)
Milk_Next_Week <- Milk + rnorm(10, mean = 1)

# paired t-test
t.test(Milk_Next_Week, Milk, paired = TRUE)

## 
##  Paired t-test
## 
## data:  Milk_Next_Week and Milk
## t = 3.5629, df = 9, p-value = 0.006091
## alternative hypothesis: true mean difference is not equal to 0
## 95 percent confidence interval:
##  0.3923296 1.7569217
## sample estimates:
## mean difference 
##        1.074626