What’s in a Name?

This is one of the project in datacamp where we can explore the relationship between names & age of the people born in U.S. This is a good exercise for the usage of Pandas in python.

1. Introduction to Baby Names Data

What’s in a name? That which we call a rose, By any other name would smell as sweet.

In this project, we will explore a rich dataset of first names of babies born in the US, that spans a period of more than 100 years! This suprisingly simple dataset can help us uncover so many interesting stories, and that is exactly what we are going to be doing.

Let us start by reading the data.

In [62]:
# Import modules
import pandas as pd

# Read names into a dataframe: bnames
bnames = pd.read_csv('datasets/names.csv.gz')

One of the first things we want to do is to understand naming trends. Let us start by figuring out the top five most popular male and female names for this decade (born 2011 and later). Do you want to make any guesses? Go on, be a sport!!

In [64]:
# bnames_top5: A dataframe with top 5 popular male and female names for the decade
import pandas as pd
bnames = pd.read_csv('datasets/names.csv.gz')
bnames_2010 = bnames.loc[bnames['year']> 2010]
bnames_2010_agg = bnames_2010.groupby(['sex','name'], as_index=False)['births'].sum()
bnames_top5 = bnames_2010_agg.\
    sort_values(['sex','births'], ascending=[True, False]).\
    groupby('sex').head().reset_index(drop=True)
print(bnames_top5)

  sex      name  births
0   F      Emma  121375
1   F    Sophia  117352
2   F    Olivia  111691
3   F  Isabella  103947
4   F       Ava   94507
5   M      Noah  110280
6   M     Mason  105104
7   M     Jacob  104722
8   M      Liam  103250
9   M   William   99144

3. Proportion of Births

While the number of births is a useful metric, making comparisons across years becomes difficult, as one would have to control for population effects. One way around this is to normalize the number of births by the total number of births in that year.

In [66]:
bnames2 = bnames.copy()
# Compute the proportion of births by year and add it as a new column
total_births_by_year = bnames.groupby('year')['births'].transform('sum')
bnames2['prop_births'] = bnames2['births'] / total_births_by_year
bnames2.head()
Out[66]:
name sex births year prop_births
0 Mary F 7065 1880 0.035065
1 Anna F 2604 1880 0.012924
2 Emma F 2003 1880 0.009941
3 Elizabeth F 1939 1880 0.009624
4 Minnie F 1746 1880 0.008666

4. Popularity of Names

Now that we have the proportion of births, let us plot the popularity of a name through the years. How about plotting the popularity of the female names Elizabeth, and Deneen, and inspecting the underlying trends for any interesting patterns!

In [68]:
# Set up matplotlib for plotting in the notebook.
%matplotlib inline
import matplotlib.pyplot as plt

def plot_trends(name, sex):
  data = bnames[(bnames.name == name) & (bnames.sex == sex)]
  ax = data.plot(x="year", y="births")
  ax.set_xlim(1880, 2016)
  return ax

# Plot trends for Elizabeth and Deneen 
for name in ('Elizabeth', 'Deneen'):
    plt.axis = plot_trends(name, 'F')
plt.xlabel('Year')
plt.ylabel('Births')
plt.show()
# How many times did these female names peak?
num_peaks_elizabeth = 3
num_peaks_deneen    = 1

5. Trendy vs. Stable Names

Based on the plots we created earlier, we can see that Elizabeth is a fairly stable name, while Deneen is not. An interesting question to ask would be what are the top 5 stable and top 5 trendiest names. A stable name is one whose proportion across years does not vary drastically, while a trendy name is one whose popularity peaks for a short period and then dies down.

There are many ways to measure trendiness. A simple measure would be to look at the maximum proportion of births for a name, normalized by the sume of proportion of births across years. For example, if the name Joe had the proportions 0.1, 0.2, 0.1, 0.1, then the trendiness measure would be 0.2/(0.1 + 0.2 + 0.1 + 0.1) which equals 0.5.

Let us use this idea to figure out the top 10 trendy names in this data set, with at least a 1000 births.

In [70]:
# top10_trendy_names | A Data Frame of the top 10 most trendy names
names = pd.DataFrame()
name_and_sex_grouped = bnames.groupby(['name', 'sex'])
names['total'] = name_and_sex_grouped['births'].sum()
names['max'] = name_and_sex_grouped['births'].max()
names['trendiness'] = names['max'] / names['total']
top10_trendy_names = names.loc[names['total'] > 999].sort_values('trendiness', ascending=False).\
                    head(10).reset_index()
print(top10_trendy_names)

       name sex  total   max  trendiness
0  Christop   M   1082  1082    1.000000
1   Royalty   F   1057   581    0.549669
2     Kizzy   F   2325  1116    0.480000
3    Aitana   F   1203   564    0.468828
4    Deneen   F   3602  1604    0.445308
5    Moesha   F   1067   426    0.399250
6    Marely   F   2527  1004    0.397309
7     Kanye   M   1304   507    0.388804
8  Tennille   F   2172   769    0.354052
9   Kadijah   F   1411   486    0.344437

6. Bring in Mortality Data

So, what more is in a name? Well, with some further work, it is possible to predict the age of a person based on the name (Whoa! Really????). For this, we will need actuarial data that can tell us the chances that someone is still alive, based on when they were born. Fortunately, the SSA provides detailed actuarial life tables by birth cohorts.

year age qx lx dx Lx Tx ex sex
1910 39 0.00283 78275 222 78164 3129636 39.98 F
1910 40 0.00297 78053 232 77937 3051472 39.09 F
1910 41 0.00318 77821 248 77697 2973535 38.21 F
1910 42 0.00332 77573 257 77444 2895838 37.33 F
1910 43 0.00346 77316 268 77182 2818394 36.45 F
1910 44 0.00351 77048 270 76913 2741212 35.58 F

You can read the documentation for the lifetables to understand what the different columns mean. The key column of interest to us is lx, which provides the number of people born in a year who live upto a given age. The probability of being alive can be derived as lx by 100,000.

Given that 2016 is the latest year in the baby names dataset, we are interested only in a subset of this data, that will help us answer the question, "What percentage of people born in Year X are still alive in 2016?"

Let us use this data and plot it to get a sense of the mortality distribution!

In [72]:
%matplotlib inline
import matplotlib.pyplot as plt


# Read lifetables from datasets/lifetables.csv
lifetables = pd.read_csv('datasets/lifetables.csv')

# Extract subset relevant to those alive in 2016
lifetables_2016 = lifetables[lifetables['year'] + lifetables['age'] ==2016]

# Plot the mortality distribution: year vs. lx
lifetables_2016.plot(x='year', y='lx')
plt.show()
lifetables_2016.head()
Out[72]:
year age qx lx dx Lx Tx ex sex
116 1900 116 0.83363 0.0 0 0 0 0.69 M
236 1900 116 0.83363 0.0 0 0 0 0.69 F
346 1910 106 0.51178 7.0 3 5 9 1.38 M
466 1910 106 0.47062 61.0 29 47 92 1.52 F
576 1920 96 0.30765 1625.0 500 1,375 4,000 2.46 M

7. Smoothen the Curve!

We are almost there. There is just one small glitch. The cohort life tables are provided only for every decade. In order to figure out the distribution of people alive, we need the probabilities for every year. One way to fill up the gaps in the data is to use some kind of interpolation. Let us keep things simple and use linear interpolation to fill out the gaps in values of lx, between the years 1900 and 2016.

In [74]:
# Create smoothened lifetable_2016_s by interpolating values of lx
import numpy as np
year = np.arange(1900, 2016)
mf = {"M": pd.DataFrame(), "F": pd.DataFrame()}
for sex in ["M", "F"]:
  d = lifetables_2016[lifetables_2016['sex'] == sex][["year", "lx"]]
  mf[sex] = d.set_index('year').\
    reindex(year).\
    interpolate().\
    reset_index()
  mf[sex]['sex'] = sex

lifetable_2016_s = pd.concat(mf, ignore_index = True)
lifetable_2016_s.head()
Out[74]:
year lx sex
0 1900 0.0 F
1 1901 6.1 F
2 1902 12.2 F
3 1903 18.3 F
4 1904 24.4 F

8. Distribution of People Alive by Name

Now that we have all the required data, we need a few helper functions to help us with our analysis.

The first function we will write is get_data,which takes name and sex as inputs and returns a data frame with the distribution of number of births and number of people alive by year.

The second function is plot_name which accepts the same arguments as get_data, but returns a line plot of the distribution of number of births, overlaid by an area plot of the number alive by year.

Using these functions, we will plot the distribution of births for boys named Joseph and girls named Brittany.

In [76]:
def get_data(name, sex):
    name_sex = ((bnames['name'] == name) & 
                (bnames['sex'] == sex))
    data = bnames[name_sex].merge(lifetable_2016_s)
    data['n_alive'] = data['lx']/(10**5)*data['births']
    return data

def plot_data(name, sex):
    fig, ax = plt.subplots()
    dat = get_data(name, sex)
    dat.plot(x = 'year', y = 'births', ax = ax, 
               color = 'black')
    dat.plot(x = 'year', y = 'n_alive', 
              kind = 'area', ax = ax, 
              color = 'steelblue', alpha = 0.8)
    ax.set_xlim(1900, 2016)
    return 
    
# Plot the distribution of births and number alive for Joseph and Brittany
#get_data("Brittany", "F")
plot_data('Brittany', 'F')

9. Estimate Age

In this section, we want to figure out the probability that a person with a certain name is alive, as well as the quantiles of their age distribution. In particular, we will estimate the age of a female named Gertrude. Any guesses on how old a person with this name is? How about a male named William?

In [78]:
# Import modules
from wquantiles import quantile

# Function to estimate age quantiles
def estimate_age(name, sex):
    data = get_data(name, sex)
    qs = [0.75, 0.5, 0.25]
    quantiles = [2016 - int(quantile(data.year, data.n_alive, q)) for q in qs]
    result = dict(zip(['q25', 'q50', 'q75'], quantiles))
    result['p_alive'] = round(data.n_alive.sum()/data.births.sum()*100, 2)
    result['sex'] = sex
    result['name'] = name
    return pd.Series(result)
# Estimate the age of Gertrude
estimate_age('Gertrude', 'F')
Out[78]:
name       Gertrude
p_alive       18.73
q25              70
q50              80
q75              89
sex               F
dtype: object

10. Median Age of Top 10 Female Names

In the previous section, we estimated the age of a female named Gertrude. Let's go one step further this time, and compute the 25th, 50th and 75th percentiles of age, and the probability of being alive for the top 10 most common female names of all time. This should give us some interesting insights on how these names stack up in terms of median ages!

In [80]:
# Create median_ages: DataFrame with Top 10 Female names, 
#    age percentiles and probability of being alive


top_10_female_names = bnames.\
  groupby(['name', 'sex'], as_index = False).\
  agg({'births': np.sum}).\
  sort_values('births', ascending = False).\
  query('sex == "F"').\
  head(10).\
  reset_index(drop = True)
estimates = pd.concat([estimate_age(name, 'F') for name in top_10_female_names.name], axis = 1)
median_ages = estimates.T.sort_values('q50').reset_index(drop = True)
median_ages.head()
Out[80]:
name p_alive q25 q50 q75 sex
0 Sarah 86.05 20 30 38 F
1 Elizabeth 74.49 23 38 58 F
2 Jennifer 96.35 31 38 44 F
3 Susan 85.8 52 59 65 F
4 Patricia 76.75 54 63 71 F