UFO Sightings

Data

Link to json file

Exercise 1

Read in the json file as a pandas dataframe. After reading in the data, you’ll want to explore it and gain some intuition. Exploring data is a very important step — the more you know about your data the better! Answer the following questions to gain some insight into this dataset.

• How many rows are there?
• How many columns?
• What does a row represent in this dataset?
• What are the different ways missing values are encoded?
• How many np.nan in each column?

Some useful code for exploring data

# Object/Categorical Columns
data.column_name.value_counts(dropna=False)
data.column_name.unique()

# Numeric Columns
data.column_name.describe()

# Counting missing values
data.isna().sum()  # Creates boolean dataframe and sums each column

Exercise 2

After learning different ways our data encodes missing values, now we will neatly manage them. There are many techniques we can use to handle missing values; for example, we can drop all rows that contain a missing value, impute with mean or median, or replace missing values with a new missing category. We will use some of these techniques in this exercise.

• shape_reported - replace missing values with missing string.
• distance_reported - change -999 values to np.nan. (-999 is a typical way of encoding missing values.)
• distance_reported - fill in missing values with the mean (imputation)
• were_you_abducted - replace - string with missing string.

The first 10 rows of your data should look like this after completion of the above steps.

Some useful code for filling in missing data

data.column_name.replace(..., ..., inplace=True)
data.column_name.fillna(..., inplace=True)

Exercise 3

Create a table that contains the following summary statistics.

• median estimated size by shape
• mean distance reported by shape
• count of reports belonging to each shape

Your table should look like this:

Some useful code for grouping and getting summary statistics

(data.groupby(...)
     .agg(...,
          ...,
          ...))

Exercise 4

The cities listed below reported their estimated size in square inches, not square feet. Create a new column named estimated_size_sqft in the dataframe, that has all the estimated sizes reported as sqft. (Hint: divide by 144 to go from sqin -> sqft)

• Holyoke
• Crater Lake
• Los Angeles
• San Diego
• Dallas

The head of your data should look like this.

Some useful code to fix the rows reported in sqin

np.where(...,  # Condition
         ...,  # If condition is true
         ...)  # If condition is false