The Significance of Changes in the Gender Happiness Gap

  • A
    misunderstanding
    . I suspect that the
    claim that happiness did not significantly change from 1972-2006 comes from the
    fact that we did not include stars when reporting the implied gender gaps in Table
    1 of our paper. Thus, the claim that

the ordered probit analysis found that the "Gender
happiness gap" was not statistically significant, either in 1972 or in
2006, even at the 0.10 level

is simply untrue. Here’s the relevant part of Table 1, which is
an ordered probit regression, of happiness on time trends by gender:

New_picture
The right way to test for whether women
were, on average, happier at the start of the sample is to look at the “Female
dummy”, which is clearly significant. The right way to ask whether this gender gap has changed is to look at
the difference in trends, which is also clearly significant. The last two rows are regression-based predicted
values, so we didn’t think we should put stars next to these numbers.

  •  Statistical
    mischief
    : When you want to make a result go away, throw away enough data,
    and a result will become insignificant. For instance pooling all of the data gives us a useful 46,303
    observations. Analyze any specific year,
    and you are left with only 1,500-3,000 data points. Even so, let’s analyze only data from 1972
    and 2006:

    • %Very happy = 28.7 + 3.1*Female +1.6*(Year2006)
      – 2.4*(Female in 2006)
    • %Not too happy = 18.1 -3.2*Female – 5.5*(Year2006)
      + 4.1*(Female in 2006)

In the first case, no coefficients are
statistically significant, and in the latter, all are. In both cases, the estimates say that women
were once a fair bit happier than men, and this is no longer true. Comparing this regression with those in our
paper, we simply learn that a smaller sample yields similar estimates, but they
are less likely to be statistically significant.

  • Looking for
    a masterpiece, when we are doing collage
    . Sometimes studying social
    phenomena is hard, and one draws on many data sources to put together a collage
    of evidence. Our paper finds declining
    happiness among women relative to men in: the General Social Survey (n=46,303
    from 1972-2006); the Virginia Slims Poll (n=26,701 from 1972-2000); among U.S.
    12th graders (Monitoring the Future; n=433,906 from 1976-2005); in the
    United Kingdom (British Household Panel Study data from 1991-2004; n=121,135);
    in Europe (the Eurobarometer analysis has n=636,400 from 1973-2002, covering 15
    countries), and across developed countries (the International Social Survey
    Program contains surveys 35 countries from 1991-2001 yielding n=97,462). The only dataset that does not yield clear
    results of a decline in women’s happiness relative to men’s is the World Values
    Survey, and even there, the data do not speak clearly.

Let me try to give a particularly transparent description of the data,
simply splitting the GSS data into two periods, 1972-1989 v. 1990-2006. There was a clear gender happiness gap in the
earlier period (34.3% of women were very happy v. 31.8% of men). This difference is clearly statistically
significant (t=4.1). In the later
period, 30.9% of women were very happy, compared with 31.1% of men. This recent gender happiness gap is
insignificant (t=-0.3). The decline in
the share of women who were very happy (34.3% v. 30.9%) is clearly significant
(t=5.9), while the corresponding changes for men were not (t=-1.1). The decline in the share of women who were
very happy relative to men is also significant (t=-3.1). Analyzing the share who are “not too happy”
yields a roughly similar pattern (but in reverse): an insignificant “unhappiness
gap” in the earlier period, but a significant gap emerged in the latter period. Interestingly, the “unhappiness gap” emerged
because as men became less likely to be unhappy, as women’s unhappiness
remained largely stable. The ordered
probit is a regression technique that allows one to make these happiness and
unhappiness comparisons all at the same time; these regressions tell us that
there was a gender happiness gap favoring women in the earlier period, and it
now favors men. For the
regression-heads, if your library subscribes can download the GSS data from the
ICPSR here. I’ll post some stata code in the comments.

This post only deals with whether the effects we
describe in
the paper are statistically significant. The other complaint is that
our results are too small to matter. Later today, I’ll turn to how we
think
about whether these are large or small effects.

[Written jointly with my coauthor Betsey Stevenson]

UPDATE: See discussion of "economic significance" here.

Comments

With apologies for jamming up the comments, I promised to post some Stata code for the regression-heads. (Feel free to ignore.)

cd "C:\Users\jwolfers\Desktop\" /* You will need to change this line to point to your data*/
clear
set mem 400m
set more off
use HAPPY SEX YEAR OVERSAMP using "04697-0001-Data.dta", clear
* Clean the data
replace HAPPY=. if HAPPY==0 | HAPPY==9
gen vhappy=1 if HAPPY==1
replace vhappy=0 if HAPPY==2 | HAPPY==3
gen unhappy=1 if HAPPY==3
replace unhappy=0 if HAPPY==1 | HAPPY==2
gen wt=OVERSAMP /* This is the weighting variable */

* Gender happiness gap at the beginning and end of the sample
oprobit HAPPY SEX [aw=wt] if YEAR==1972
oprobit HAPPY SEX [aw=wt] if YEAR==2006
* Changes in the gender happiness gap using only the first and last years
xi: reg vhappy i.SEX*i.YEAR[aw=wt] if YEAR==1972 | YEAR==2006
xi: reg unhappy i.SEX*i.YEAR [aw=wt] if YEAR==1972 | YEAR==2006
* Using more of the data: Simply split the data into two roughly equal periods
tab YEAR if HAPPY~=.
gen period=YEAR>1989
label define period 0 "1972-1989" 1 "1990-2006"
label values period period
* Analyze %very happy
table period SEX [aw=wt], c(m vhappy)
xi: reg vhappy i.SEX*i.period [aw=wt]
xi: reg vhappy i.SEX [aw=wt] if period==0
xi: reg vhappy i.SEX [aw=wt] if period==1
xi: reg vhappy period [aw=wt] if SEX==1
xi: reg vhappy period [aw=wt] if SEX==2
* Analyze %unhappy
table period SEX [aw=wt], c(m unhappy)
xi: reg unhappy i.SEX*i.period [aw=wt]
xi: reg unhappy i.SEX [aw=wt] if period==0
xi: reg unhappy i.SEX [aw=wt] if period==1
xi: reg unhappy period [aw=wt] if SEX==1
xi: reg unhappy period [aw=wt] if SEX==2
* Analyze whole distribution
xi: oprobit HAPPY i.SEX*i.period [aw=wt]
xi: oprobit HAPPY i.SEX [aw=wt] if period==0
xi: oprobit HAPPY i.SEX [aw=wt] if period==1
xi: oprobit HAPPY period [aw=wt] if SEX==1
xi: oprobit HAPPY period [aw=wt] if SEX==2

This has nothing to do with the content of the post, but the weighting in the regressions made me want to ask this. I have heard different things from econometricians as to when it is and is not appropriate to weight a regression. Say that you have an (to make this concrete) individual-level panel survey dataset that oversamples blacks, and you want to look at mean difference in some outcome measure in which you interact a race dummy with some exogeneous variable. So, in other words, you think Blacks respond differently than Whites to some treatment, called z. All capital letters are parameters to be estimated, and x is a vector of exogeneous variables.

y=xB + z*rA + zC +rD + e

If I am controlling for race in the regression, then do I need to weight the regression with the person weight provided with teh survey? The survey oversamples blacks, for instance, and I am explicitly controlling for that in the regression.

Remind me, briefly, how you'd use the available data to distinguish e.g. "women's happiness levels have changed" from
"women have used different criteria for what mental state counts as very happy, pretty happy, etc."

Who uses Stata? Get with the times, move to R...

Personally, I'm still unconvinced of the economic importance. but I do have a statistical question.

When I look at the graph of these results (first graph here:)

http://itre.cis.upenn.edu/~myl/languagelog/archives/004965.html

(I know that's probably not your favorite URL at the moment.)

Well, in that graph there's an anomaly in the female scores in 1974: (1) Does the significance of the results disappear if this year is omitted? (2) What were "women very happy" about in 1974? Nixon quitting?

A quick follow up -- I just checked the impact of the OVERSAMP vs. WTSSALL weighting variable and the differences are fairly large and make the effect you found seem even larger (at least based on my quick 1972 vs. 2006 comparisons).

Comparing the not too happy category, the OVERSAMP weight yields men declining from 18.1% to 12.6% and women declining from 14.9% to 13.4% (1972 to 2006). Using the WTSSALL weight, the men declined from 19.0 to 10.9 while women declined from 15.2 to 12.4. The net change in Not Too Happy was 5.5% for men and 1.5% for women using your weighting, but I think it should be 8.1% for men and 2.8% for women.

I think you may need to re-do your analysis using the WTSSALL variable instead of OVERSAMP. WTSSALL is the default weight used by UC Berkeley SDA application for querying GSS.

I love Christian Louboutin!I bought 2 pairs of shoes at the www.christianlouboutinhighheels.com.
Christian louboutin pumps feel so much more comfortable and are so much sexier than I would know. The salespeople of christian louboutin pumps on sale are very friendly. I live in CA but call to order often and Scott always helps me pick out the most perfect shoes. They do special orders of shoes from past seasons for a 30% upcharge as long as they still have the materials and free shipping!. These are adorable shoes.I get compliments every time I wear christian louboutin pumps uk. They fit well and are quite comfortable; I walked around downtown DC today at lunchtime w/no problems. Christian louboutin pumps sale is good on the website.I hope that I can share my happiness and good things with you!

Comments for this post are closed