In the recent affirmative action decision by the U.S. Supreme Court, Justice Jackson made a startling claim:

Dr. Vinay Prasad takes issue with the shoddy study on which Justice Jackson might well have relied upon in good faith. I will assume that neither she nor her law clerks have the necessary expertise for critically analyzing the study she cited for making the claim that Black doctors are twice as good at saving the lives of Black newborns. In this article, Dr. Prasad shows the skepticism one needs to show upon hearing such an extraordinary claim.

The paper in question is catastrophically flawed. First, consider that it is a bold claim that a white doctor is twice as likely to kill a black baby. The effect size (TWICE as likely!) is massive. . . .

Next, in my podcast from Aug 2020 I discuss why this paper is flawed (full podcast is 91 min. but relevant discussion runs from 1:31:00 to 0:52:00 mark). Those notes are also captured here.

• If white doctors have so much worse outcomes, one would expect they are making different decisions in the care of neonates than Black doctors— but this paper cannot show the mechanism of the difference
• The paper assumes doctor-baby pairings are quasi randomized, but that is unfounded assumption. It may not be quasi randomized and well off Blacks may be more likely to have Black doctors
• A baby born is seen by a team of doctors— pediatricians, anesthesiologists, obs— which doctor is ascribed the ‘assigned provider’ per baby. What determines this assignment? (the authors do not provide this information)
• Since, the paper was published it was revealed that some hospitals put a treating doctor on the form and others put the head of the unit. (massive bias)
• A baby born is seen by a team— nurses, staff, doctors, etc— why are the races (and racial concordance) of these people not accounted for.
• If a baby gets sick, and goes to NICU and dies, which doctor is ascribed responsibility. If NICU doctors have different racial make up than other doctors could this not bias results?
• Broader issues of administrative data/ multiple hypothesis testing detailed in the episode.

Prasad also breaks down a second article claiming that Black doctors are substantially better at saving Black lives: "The Supreme Court’s affirmative action decision puts lives at risk."

Prasad sets forth the limitations on this second study, which also makes an extraordinary race-based claim: