Background

Delayed flights are not something most people look forward to. In the best case scenario you may only wait a few extra minutes for the plane to be cleaned. However, those few minutes can stretch into hours if a mechanical issue is discovered or a storm develops. Arriving hours late may result in you missing a connecting flight, job interview, or your best friend’s wedding.

In 2003 the Bureau of Transportation Statistics (BTS) began collecting data on the causes of delayed flights. The categories they use are Air Carrier, National Aviation System, Weather, Late-Arriving Aircraft, and Security. You can visit the BTS website to read definitions of these categories.

This json file contains information on delays at 7 airports over 10 years. Your task is to clean the data, search for insights about flight delays, and communicate your results using the provided template. If you have completed the checkpoints for Unit 5, then you are ready to answer the Grand Questions listed below. Refer to the readings for additional help.

Data:

Download: JSON File
Information: Data Description

Readings:

• P4DS: Section 12.1 & 12.2 Tidy data
• P4DS: Chapter 5 Data transformation
• P4DS: Section 7.4 Missing Values
• Python Data Science Handbook: Missing Data
• Wikipedia Missing Data

Optional References

• isin method
• where method
• np.where method
• replace method
• An introduction to JSON (May need to open in ingognito to read.)
• The key word in ‘Data Science’ is not Data…
• How to Handle Missing Data (May need to open in ingognito to read.)

Grand Questions:

1. Which airport has the worst delays? How did you choose to define “worst”? As part of your answer include a table that lists the total number of flights, total number of delayed flights, proportion of delayed flights, and average delay time in hours, for each airport.

2. What is the worst month to fly if you want to avoid delays? Include one chart to help support your answer, with the x-axis ordered by month. You also need to explain and justify how you chose to handle the missing Month data.

3. According to the BTS website the Weather category only accounts for severe weather delays. Other “mild” weather delays are included as part of the NAS category and the Late-Arriving Aircraft category. Calculate the total number of flights delayed by weather (either severe or mild) using these two rules:

   1. 30% of all delayed flights in the Late-Arriving category are due to weather.
   2. From April to August, 40% of delayed flights in the NAS category are due to weather. The rest of the months, the proportion rises to 65%.

4. Create a barplot showing the proportion of all flights that are delayed by weather at each airport. What do you learn from this graph (Careful to handle the missing Late Aircraft data correctly)?

5. Fix all of the varied NA types in the data to be consistent and save the file back out in the same format that was provided (this file shouldn’t have the missing values replaced with a value). Include one record example from your exported JSON file that has a missing value (No imputation in this file).

Deliverables:

Use this template to submit your Client Report. The template has three sections (for additional details please see the instructional template):

1. An “elevator pitch” that summarizes the entire case study.
2. Answers to the grand questions that include text, pictures, and tables.
3. An appendix that provides your commented code and justification for any programming that required you to choose an option.