Results for “Tests” 772 found
Model this newsroom estimator
The New York Times’s performance review system has for years given significantly lower ratings to employees of color, an analysis by Times journalists in the NewsGuild shows.
The analysis, which relied on data provided by the company on performance ratings for all Guild-represented employees, found that in 2021, being Hispanic reduced the odds of receiving a high score by about 60 percent, and being Black cut the chances of high scores by nearly 50 percent. Asians were also less likely than white employees to get high scores.
In 2020, zero Black employees received the highest rating, while white employees accounted for more than 90 percent of the roughly 50 people who received the top score.
The disparities have been statistically significant in every year for which the company provided data, according to the journalists’ study, which was reviewed by several leading academic economists and statisticians, as well as performance evaluation experts.
…Management has denied the discrepancies in the performance ratings for nearly two years…
And from the economists:
Multiple outside experts consulted by the reporters consistently said the methodology used in the Guild’s most recent analysis was reasonable and appropriate and that the approach used by the company appeared either flawed or incomplete. Some went further, suggesting the company’s approach seemed tailor-made to avoid detecting any evidence of bias.
Rachael Meager, an economist at the London School of Economics, was blunt: “LMAO, that’s so dumb,” she wrote when Guild journalists described the company’s methodology to her. “That’s what you would do if you want to obliterate signal,” she added, using a word that in economics refers to meaningful information.
“This is so stupid as to border on negligence,” added Dr. Meager, who has published papers on evaluating statistical evidence in leading economics journals.
Peter Hull, a Brown University economist who has studied statistical techniques for detecting racial bias, also questioned the company’s approach and recommended a way to test it: running simulations in which bias was intentionally added. The company’s method repeatedly failed to detect racial disparities in those tests.
Here is the full article, prepared by the NYT Guild Equity Committee, including Ben Casselman. Of course we now live in a world where very few people will be surprised by this. Where exactly does the moral authority lie here for making editorial judgments about content concerning race?
Monday assorted links
1. Fraudulent markets in everything.
2. “We present evidence that discrimination against Asian-American Airbnb users sharply increased at the start of the COVID-19 pandemic. Using a DiD approach, we find that hosts with distinctively Asian names experienced a 12 percent decline in guests relative to hosts with distinctively White names.” Link here.
3. New paper on polygenic scores and economic outcomes.
4. “Higher openness mediated the link between adolescent IQ and late life cognition.”
6. “Ford just raised the price of its electric F-150 by up to $8,500” — coincidence!
7. “…in reality, cynics test lower on cognitive & competency tests.“
The human capital deficit in leadership these days
It is very real, just look around the world. Even Mario Draghi is on the way out. Here is one take from Adrian Wooldridge at Bloomberg:
Leadership is most vital during a period of transition from one order to another. We are certainly in such a period now — not only from the neoliberal order to something much darker but also to a new era of smart machines — yet so far leadership is lacking. We call for leaders who are equal to the times, but nobody answers.
Kissinger offers two explanations for this troubling silence. The first lies in the evolution of meritocracy. (Full disclosure: He mentions a book I have written on this subject). The six leaders were all born outside the pale of the aristocratic elite that had hitherto dominated politics, and particularly foreign policy: Adenauer and Sadat were the sons of clerks, Thatcher and Nixon were the children of storekeepers, Lee’s parents were downwardly mobile. But theirs was a meritocracy with an aristocratic flavor. They went to elite schools and universities that provided an education in human excellence rather than just passing tests. In rubbing shoulders with members of the old elite, they absorbed some of its ethic of noblesse oblige (“For unto whomsoever much is given, of him shall be much required”) as well as its distaste for populism. Hence Lee’s recurring references to “Junzi” (Confucian gentlemen) and de Gaulle’s striving to become a “man of character.” They believed in history, tradition and, in most cases, God.
The world has become much more meritocratic since Kissinger’s six made their careers, not least when it comes to women and ethnic minorities. But the dilution of the aristocratic element in the mix may also have removed some of the grit that produced the pearl of leadership: Schools have given up providing an education in human excellence — the very idea would be triggering! — and ambitious young people speak less of obligation than of self-expression or personal advancement. The bonds of character and duty that once bound leaders to their people are dissolving.
There are further arguments — much more in fact — at the link. And here is an ngram on “leadership.”
Monday assorted links
1. An alternative to the Baumol cost-disease hypothesis (but is it really, isn’t worker allocation across sectors endogenous to, among other things, Baumol-like factors?)
3. Please stop saying that hot drinks cool you down.
4. “…mass shootings are more likely after anniversaries of the most deadly historical mass shootings. Taken together, these results lend support to a behavioral contagion mechanism following the public salience of mass shootings.” Link here.
5. What motivates leaders to invest in nation-building?
Colombia tax facts of the day
According to the OECD, only 5 per cent of Colombians pay income tax. Revenue from personal income tax is worth just 1.2 per cent of gross domestic product compared to an OECD average of 8.1 per cent.
Taxes on businesses, on the other hand, are relatively high. The outgoing rightwing government of Iván Duque initially cut the corporate tax rate from 33 to 31 per cent. But when the coronavirus pandemic hit, followed by prolonged street protests against his rule, he was forced to raise it to 35 per cent, its highest level in 15 years and above the Latin American average.
Here is more from the FT. Note that the president-elect, Petro, actually is planning on cutting taxes on business, though he is proposing a wealth tax on individuals as well.
Quantitative Political Science Research is Greatly Underpowered
We analyze the statistical power of political science research by collating over 16,000 hypothesis tests from about 2,000 articles. Even with generous assumptions, the median analysis has about 10% power, and only about 1 in 10 tests have at least 80% power to detect the consensus effects reported in the literature. There is also substantial heterogeneity in tests across research areas, with some being characterized by high-power but most having very low power. To contextualize our findings, we survey political methodologists to assess their expectations about power levels. Most methodologists greatly overestimate the statistical power of political science research.
Markets in everything those new service sector jobs
A woman who took about 150 theory and practical tests for other drivers has been jailed for eight months.
Police said she was taking tests for people who had difficulty with English.
…Det Ch Insp Steven Maloney said Kaur’s crimes were motivated by greed.
Here is the full story, via Michael B.
What Caused the Murder Spike?
I think there’s clear evidence that the current murder spike was caused primarily by the 2020 BLM protests. The timing matches the protests well, and the pandemic poorly. The spike is concentrated in black communities and not in any of the other communities affected by the pandemic. It matches homicide spikes corresponding to other anti-police protests, most notably in the cities where those protests happened but to a lesser degree around the country. And the spike seems limited to the US, while other countries had basically stable murder rates over the same period.
I agree with Scott Alexander, although I would emphasize a little more the mediating factor of the police pullback.
I would also add that each step in the mechanism–protests lead to police pullback which leads to an increase in murders–is well supported on its own in the academic literature. Step one, for example, is that protests lead to police pullback. In The effect of highly publicized police killings on policing: Evidence from large U.S. cities Cheng and Long document exactly this:
Our regression discontinuity and difference-in-differences estimates provide consistent and strong evidence that those high-profile killings reduced policing activities, including police self-initiated activities and arrests.
That’s step one. Step two is that police on the street reduce crime which you can find from my research using the terror alert level as well as that of many others. Step one plus step two leads to a spike in murders following the 2020 BLM protests.
As Alexander noted, we also have plenty of evidence on a micro level. For example, I showed clear evidence of police pullback–a “blue strike”–and consequent increase in crime in Baltimore after the Freddie Gray protests. Put it all together along with the timing and other evidence and the case is strong that the 2020 BLM protests led to police pullback which led to a spike in murders, especially in black communities.
GiveDirectly to Administer Cash Grants Pilot in Chicago
Long time readers will know that both Tyler and I have been promoting GiveDirectly since its creation in 2011. GiveDirectly was started by four economists and it gives money directly to the very poor in Kenya and Uganda. Unusually for any charity, GiveDirectly has published substantial, high quality, pre-registered RCTs on its work and is a top-rated charity by GiveWell. GiveDirectly, the charity, continues to do great work helping the world’s poorest people but they are now also using their experience to help design and administer programs in the United States.
A nonprofit that originally focused on giving cash to impoverished people in Africa will soon be delivering money to poor residents of Chicago, in one of the largest tests of a guaranteed basic income program in the US.
GiveDirectly is administering a program that will give $500 a month to each of 5,000 households in Chicago as soon as the end of June. The city is using $31.5 million from the federal government’s American Rescue Plan Act for the year-long pilot, and hoping the payments will help it recover faster from the pandemic.
I look forward to seeing the results.
Cognitive Endurance as Human Capital
A favorite topic of mine:
Schooling may build human capital not only by teaching academic skills, but by expanding the capacity for cognition itself. We focus specifically on cognitive endurance: the ability to sustain effortful mental activity over a continuous stretch of time. As motivation, we document that globally and in the US, the poor exhibit cognitive fatigue more quickly than the rich across field settings; they also attend schools that offer fewer opportunities to practice thinking for continuous stretches. Using a field experiment with 1,600 Indian primary school students, we randomly increase the amount of time students spend in sustained cognitive activity during the school day—using either math problems (mimicking good schooling) or non-academic games (providing a pure test of our mechanism). Each approach markedly improves cognitive endurance: students show 22% less decline in performance over time when engaged in intellectual activities—listening comprehension, academic problems, or IQ tests. They also exhibit increased attentiveness in the classroom and score higher on psychological measures of sustained attention. Moreover, each treatment improves students’ school performance by 0.09 standard deviations. This indicates that the experience of effortful thinking itself—even when devoid of any subject content—increases the ability to accumulate traditional human capital. Finally, we complement these results with quasi-experimental variation indicating that an additional year of schooling improves cognitive endurance, but only in higher-quality schools. Our findings suggest that schooling disparities may further disadvantage poor children by hampering the development of a core mental capacity.
Here is the full paper by Christina L. Brown, Supreet Kaur, Geeta Kingdon & Heather Schofield. What are you doing to improve your cognitive durability?
And don’t forget that the Candidates’ Match starts today!
Direct Instruction Produces Large Gains in Learning, Kenya Edition
In an important new paper, Can Education be Standardized? Evidence from Kenya, Guthrie Gray-Lobe, Anthony Keats, Michael Kremer, Isaac Mbiti and Owen Ozier evaluate Bridge International schools using a large randomized experiment. Twenty five thousand Kenyan students applied for 10,000 scholarships to Bridge International and the scholarships were given out by lottery.
Kenyan pupils who won a lottery for two-year scholarships to attend schools employing a highly-structured and standardized approach to pedagogy and school management learned more than students who applied for, but did not win, scholarships.
After being enrolled at these schools for two years, primary-school pupils gained approximately the equivalent of 0.89 extra years of schooling (0.81 standard deviations), while in pre-primary grades, pupils gained the equivalent of 1.48 additional years of schooling (1.35 standard deviations).
These are very large gains. Put simply, children in the Bridge programs learnt approximately three years worth of material in just two years! Now, I know what you are thinking. We have all seen examples of high-quality, expensive educational interventions that don’t scale–that was the point of my post Heroes are Not Replicable and see also my recent discussion of the Perry Preschool project–but it’s important to understand the backstory of the Bridge study. Bridge Academy uses Direct Instruction and Direct Instruction scales! We know this from hundreds of studies. In 2018 I wrote (no indent):
What if I told you that there is a method of education which significantly raises achievement, has been shown to work for students of a wide range of abilities, races, and socio-economic levels and has been shown to be superior to other methods of instruction in hundreds of tests?….I am reminded of this by the just-published, The Effectiveness of Direct Instruction Curricula: A Meta-Analysis of a Half Century of Research which, based on an analysis of 328 studies using 413 study designs examining outcomes in reading, math, language, other academic subjects, and affective measures (such as self-esteem), concludes:
…Our results support earlier reviews of the DI effectiveness literature. The estimated effects were consistently positive. Most estimates would be considered medium to large using the criteria generally used in the psychological literature and substantially larger than the criterion of .25 typically used in education research (Tallmadge, 1977). Using the criteria recently suggested by Lipsey et al. (2012), 6 of the 10 baseline estimates and 8 of the 10 adjusted estimates in the reduced models would be considered huge. All but one of the remaining six estimates would be considered large. Only 1 of the 20 estimates, although positive, might be seen as educationally insignificant.
…The strong positive results were similar across the 50 years of data; in articles, dissertations, and gray literature; across different types of research designs, assessments, outcome measures, and methods of calculating effects; across different types of samples and locales, student poverty status, race-ethnicity, at-risk status, and grade; across subjects and programs; after the intervention ceased; with researchers or teachers delivering the intervention; with experimental or usual comparison programs; and when other analytic methods, a broader sample, or other control variables were used.
Indeed, in 2015 I pointed to Bridge International as an important, large, and growing set of schools that use Direct Instruction to create low-cost, high quality private schools in the developing world. The Bridge schools, which have been backed by Mark Zuckerberg and Bill Gates, have been controversial which is one reason the Kenyan results are important.
One source of controversy is that Bridge teachers have less formal education and training than public school teachers. But Brdige teachers need less formal education because they are following a script and are closely monitored. DI isn’t designed for heroes, it’s designed for ordinary mortals motivated by ordinary incentives.
School heads are trained to observe teachers twice daily, recording information on adherence to the detailed teaching plans and interaction with pupils. School heads are given their own detailed scripts for teacher observation, including guidance for preparing for the observation, what teacher behaviors to watch for while observing, and how to provide feedback. School heads are instructed to additionally conduct a 15 minute follow up on the same day to check whether teachers incorporated the feedback and enter their scores through a digital system. The presence of the scripts thus transforms and simplifies the task of classroom observation and provision of feedback to teachers. Bridge also standardizes a range of other processes from school construction to financial management.
Teachers are observed twice daily! The model is thus education as a factory with extensive quality control–which is why teachers don’t like DI–but standardization, scale, and factory production make civilization possible. How many bespoke products do you buy? The idea that education should be bespoke gets things entirely backward because that means that you can’t apply what you learn about what works at scale–Heroes are Not Replicable–and thus you don’t get the benefits of refinement, evolution, and continuous improvement that the factory model provides. I quoted Ian Ayres in 2007:
“The education establishment is wedded to its pet theories regardless of what the evidence says.” As a result they have fought it tooth and nail so that “Direct Instruction, the oldest and most validated program, has captured only a little more than 1 percent of the grade-school market.”
Direct Instruction is evidence-based instruction that is formalized, codified, and implemented at scale. There is a big opportunity in the developing world to apply the lessons of Direct Instruction and accelerate achievement. Many schools in the developed world would also be improved by DI methods.
Addendum 1: The research brief to the paper, from which I have quoted, is a short but very good introduction to the results of the paper and also to Direct Instruction more generally.
Addendum 2: A surprising number of people over the years have thanked me for recommending DI co-founder Siegfried Engelmann’s Teach Your Child to Read in 100 Easy Lessons.
Difficult-To-Pronounce Names and Labor Market Outcomes
Here is a new paper from Qi Ge and Stephen Wu:
This paper tests for the existence of labor market discrimination based on a previously unstudied characteristic: name fluency. Using data on over 1,500 economics job market candidates from roughly 100 PhD programs during the 2016-2017 and 2017-2018 job market cycles, we find that having a name that takes longer to pronounce is associated with 1) a significantly lower likelihood of being placed into an academic job or obtaining a tenure track position; and 2) an initial placement at an institution with lower research productivity, as measured by the research rankings in the Research Papers in Economics (RePEc) database. We obtain similar results using two alternative ways of measuring pronunciation difficulty, a computer generated algorithm based on commonality of letter and phoneme combinations and a subjective measure based on individual ratings, and they hold after the inclusion of many control variables including fixed effects for PhD institution and home country.
Might the hard-to-pronounce names nonetheless be proxying for lesser job networks in some manner? In any case an interesting result. Via the excellent Kevin Lewis.
Sunday assorted links
2. When do ideas get easier to find? Hard to excerpt, but important piece.
3. The economics of stadium names.
4. Pollution from car tire wear.
5. The Swedish history of not feeding other people’s children.
6. The 1993 ferry sinking off the coast of Jeremie, Haiti had a high death toll — some sources say over one thousand (does anyone know the proper final toll?). Yet the incident doesn’t seem to have its own Wikipedia page.
Testing Freedom
I did a podcast with Brink Lindsey of the Niskanen Center. Here’s one bit on the FDA’s long-history of banning home tests:
Brink Lindsey: …it’s on the rapid testing that we had inexplicable delays. Rapid tests, home tests were ubiquitous in Europe and Asia months before they were in the United States. What was going on?
Alex Tabarrok: So I think it’s not actually inexplicable because the FDA has a long, long history of just hating people testing themselves. So the FDA was against pregnancy tests, they didn’t like that, they said women they need to consult with a doctor, only the physician can do the test because literally women could become hysterical if they were pregnant or if they weren’t pregnant, this was a safety issue. There was no question that the test itself was safe or worked. Instead what the FDA said, “We can regulate this because the user using it, this could create safety issues because they could commit suicide or they could do something crazy.” So they totally expanded the meaning of safety from is the test safe to can somebody be trusted to use a pregnancy test?
Then we had exactly the same thing with AIDS testing. So we delayed personal at-home tests for AIDS for literally 25 years. 25 years these tests were unavailable because the FDA again said, “Well, they’re dangerous.” And why are they dangerous? “Well, we don’t know what people will do with this knowledge about their own bodies.” Now, of course, you can get an HIV test from Amazon and the world hasn’t collapsed. They did the same thing with genetic tests from companies like 23andMe. So I said, “Our bodies ourselves, our DNA ourselves.” That people have a right to know about the functioning of their own bodies. This to me is a very clear violation of the Constitutions on multiple respects. It just stuns me, it just stuns me that anybody could think that you don’t have a right to know, we’re going to prevent you from learning something about the operation of your own body.
Again, the issue here was never does the test work. In fact, the labs which produce these tests, those labs are regulated outside of the FDA. So whether the test actually works, whether yes, it identifies this gene, all issues of that nature, what is the sensitivity and the specificity, are the tests produced in a proper laboratory, I don’t have a lot of problem with that because that’s all something which the consumers themselves would want. What I do have a problem with is then the FDA saying, “No, you can’t have access to this test because we don’t know what you’re going to do about it, what you’re going to think about it.” And that to me is outrageous.
Here’s the full transcript and video.
Let’s eliminate the Covid test entry requirement for the U.S.
That is the topic of my latest Bloomberg column, you ought to be able to guess most of my arguments. Here is the very end:
I am not arguing for passivity in the face of danger. It is distressing that US policymakers do not seem interested in spending big for pandemic preparedness. America needs a new Operation Warp Speed for pan-coronavirus vaccines and nasal spray vaccines. It should be gathering more data on Covid and improving its system of clinical trials for anti-Covid remedies, among other measures.
I am simply saying that removing the Covid test for entry to the US would bring an end to one of the more egregious instances of “hygiene theater.” And it would send a signal that America is welcoming the world once again.
Recommended. And note that the most responsible European countries do not impose such tests.