Background

The Space Shuttle Challenger exploded 73 second after liftoff on January 28th, 1986. The disaster claimed the lives of all seven astronauts on board, including school teacher Christa McAuliffe.1 The details surrounding this disaster were very involved. If you are interested in learning more, watch this 18-minute video documentary on PBS.org. For the purposes of this analysis, it is sufficient to point out that engineers that manufactured the large boosters that launched the rocket were aware of the possible failures that could happen during cold temperatures. They tried to prevent the launch, but were ultimately ignored and disaster ensued.

Data from Previous Launches

The main concern of engineers in launching the Challenger was the evidence that the large O-rings sealing the several sections of the boosters could fail in cold temperatures.

The lowest temperature of any of the 23 prior launches (before the Challenger explosion) was 53° F 2. This is evident in the data set shown below. Engineers prior to the Challenger launch suggested that the launch not be attempted below 53°. The “evidence” that the o-rings could fail below 53° was based on a simple conclusion that since the launch at 53° experienced two o-ring failures, it seemed unwise to launch below that temperature. In the following analysis we demonstrate more fully how dangerous it was to launch on this specific day where the outside temperature at the time of the launch was 31°.

The “fail” column in the data set below records how many O-rings experienced failures during that particular launch. The “temp” column lists the outside temperature at the time of launch.

library(alr4)
library(mosaic)
library(DT)
library(pander)
datatable(Challeng)


Logistic Model

The probability of at least one o-ring failing during a shuttle launch based on the known outside temperature at the time of launch is given by the following logistic regression model.

\[ P(Y_i = 1|x_i) = \frac{e^{\beta_0+\beta_1 x_i}}{1+e^{\beta_0 + \beta_1 x_i}} = \pi_i \]

In this model, for each previous shuttle launch \(i\):

If \(\beta_1\) is zero in the above model, then \(x_i\) (temperature) provides no insight about the probability of a failed O-ring. If not zero however, then temperature plays an important role in the probability of o-rings failing. Using a significance level of \(\alpha = 0.05\) we will test the below hypotheses about \(\beta_1\).

\[ H_0: \beta_1 = 0 \\ H_a: \beta_1 \neq 0 \]


Fitting the Model

The estimates of the coefficients \(\beta_0\) and \(\beta_1\) for the above logistic regression model and data are shown below.

chall.glm <- glm(fail>0 ~ temp, data=Challeng, family=binomial)
summary(chall.glm) %>% pander()
  Estimate Std. Error z value Pr(>|z|)
(Intercept) 15.04 7.379 2.039 0.04148
temp -0.2322 0.1082 -2.145 0.03196

(Dispersion parameter for binomial family taken to be 1 )

Null deviance: 28.27 on 22 degrees of freedom
Residual deviance: 20.32 on 21 degrees of freedom


This gives the estimated model for \(\pi_i\) as \[ P(Y_i = 1|x_i) \approx \frac{e^{15.043-0.232 x_i}}{1+e^{15.043 - 0.232 x_i}} = \hat{\pi}_i \] where \(b_0 = 15.043\) is the value of the (Intercept) which estimates \(\beta_0\) and \(b_1 = -0.232\) is the value of temp which estimates \(\beta_1\).

Importantly, the \(p\)-value for the test of temp shows a significant result \((p = 0.0320)\) giving sufficient evidence to conclude that \(\beta_1 \neq 0\). The outside temperature at the time of launch effects the probability of at least one o-ring failure during the launch.


Visualizing the Model

The following plot shows how much colder it was on the day of the Challenger launch (31°F, shown by the vertical dashed gray line) compared to all 23 previous shuttle launches (black dots in the graph).

plot( fail>0 ~ temp, data=Challeng, xlab="Outside Temperature at Time of Launch (Fahrenheit)", ylab='Probability of At least One O-ring Failing', pch=16, main="NASA Shuttle Launch Data from 1981 to 1985", xlim=c(30,85))
curve(exp(15.043-0.232*x)/(1+exp(15.043-0.232*x)), add=TRUE)
abline(v=31, lty=2, col="lightgray")
text(31,0.3,"Outside Temp on Jan. 28, 1986 was 31", pos=4, cex=0.7, col="lightgray")
abline(h=c(0,1), lty=1, col=rgb(.2,.2,.2,.2))
legend("right", bty="n", legend=c("Previous Launches"), pch=16, col="black", cex=0.7)


Diagnosing the Model

To demonstrate that the logistic regression is a good fit to these data we apply the Hosmer-Lemeshow goodness of fit test (since there are only a couple repeated \(x\)-values) from the library(ResourceSelection).

library(ResourceSelection)
hoslem.test(chall.glm$y, chall.glm$fitted, g=6) %>% pander()
Hosmer and Lemeshow goodness of fit (GOF) test: chall.glm$y, chall.glm$fitted
Test statistic df P value
7.412 4 0.1157 
# Note: doesn't give a p-value for g >= 7, default is g=10.
# Larger g is usually better than smaller g.

Since the null hypothesis is that the logistic regression is a good fit for the data, we claim that the logistic regression is appropriate (p-value = 0.1157).


Conclusion

Since the temperature being zero is not really realistic for this model, the value of \(e^{b_0}\) is not interpretable. However, the value of \(e^{b_1} = e^{-0.232} \approx 0.79\) shows that the odds of the o-rings failing for a given launch decreases by a factor of 0.79 for every 1° F increase in temperature. Said differently, the odds of an o-ring failure during launch decrease by 21% (1-0.79) for every 1° F increase in temperature. (Also, from the reverse perspective, every 1° F decrease in temperature increases the odds of a failed o-ring by a factor of \(e^{0.232} \approx 1.26\).) The Challenger shuttle was launched at a temperature of 31° F. By waiting until 53° F, the odds of failure would have been decreased by a factor of \(e^{-0.232(53-31)}\approx 0.006\), a 99.4% reduction in the odds of an o-ring failure!

To state it more clearly, for a temperature of 31° F our model puts the probability of a failure at \[ P(Y_i = 1|x_i) \approx \frac{e^{15.043-0.232\cdot 31}}{1+e^{15.043 - 0.232 \cdot 31}} = \hat{\pi}_i \]

pred <- predict(chall.glm, data.frame(temp=31), type='response')
#The inline code was used below to put this "pred" value into the text:
#$\hat{\pi_i} \approx `r round(pred,5)`$

which, using R to do this calculation we get \(\hat{\pi_i} \approx 0.99961\). This shows that an O-ring failure was sure to happen when the launch temperature was that cold (31°F).

  1. See the article on Britannica.com: https://www.britannica.com/event/Challenger-disaster↩︎

  2. See this article at ics.uci.edu: http://www.ics.uci.edu/~staceyah/111-202/handouts/Dalal_etal_1989-Challenger.pdf↩︎