The Berman Institute of Bioethics's bulletin called my attention to this New York Times article about this PLoS Biology article, and all I can think is, whatever happened to peer review?
Sure, one interpretation is that Stephen Jay Gould just made it up. I'm not going to say that's impossible.
But let me get this straight: we're testing the hypothesis that a priori biases can affect supposedly objective scientific observation by...making scientific observations and assuming they're objective?
This entire paper hinges on the premise that, sure, racism may have existed way back in the 19th century, but certainly we modern scientists have no a priori biases that could possibly influence our own measurements. The experimental design to test Gould's premise is built entirely around assuming the premise is incorrect.
This paper doesn't prove that the biases don't exist, they're just showing that they're reproducible, which is not at all unreasonable: racism isn't just an individual's opinion, it's a system incorporated structurally into our society.
Fortunately, science has invented a way to get around this problem (or at least try to). It's called blinding your data. Instead of picking a skull out of the bin labeled "African" and then measuring it (which appears to be how this study was done), could you at least have one person take the skull out of the bin, slap an identifying code onto it, and then hand it to a second person, who makes the actual measurement (and doesn't know which bin it came out of)?. This only partially avoids the problem (because you might still be able to guess by looking at the skull what ethnicity it comes from), but the authors didn't even do that much.
PLoS Biology, I'm not impressed.
6 comments:
Measuring the cranial capacity of skulls is hardly rocket science. If accurately doing such a trivial task was beyond the ability of contemporary scientists, modern technology would not be possible.
What happened was that Morton's measurements and those of six contemporary anthropologists turned out to be highly similar, indicating that, contra Gould, Morton did not fudge his measurements. Note that Gould never made any measurements of his own of the Morton collection, he just reinterpreted Morton's reported data, but, as shown in the new study, he manipulated those data to fit his preconceptions. I suggest you carefully read the study, and decide for yourself whether Gould's errors were intentional or not: http://tinyurl.com/692t5kn
Hi JL,
Thank you for your thoughtful comments.
You don't address the substance of my criticism, though: the fact that the measurements are reproducible doesn't necessarily imply that there's no bias: it could alternatively imply that the bias is just reproducible. Hence my suggestion not for more reproduction, but rather for blinding the data.
It's fine to believe that simple measurements aren't rocket science, but that's the very hypothesis being tested here.
Sorry, but you are completely wrong. The method kept the measurer blind to what volumes were being produced, as stated in the methods section of the paper. So even if he wanted to match Morton's volumes, it would be impossible. Plus each skull was measured three separate times.
So what happened to peer review? They read the paper before they judged it.
Gould failed to show any bias in Morton's measurements, so what is your a priori reason for believing that there was bias? The anthropologists who conducted the reanalysis describe their methods thoroughly. They remeasured hundreds of skulls, each several times, and ended up with results that were almost identical to those that Morton arrived at. To suggest that Morton and these six contemporary scientists -- despite being separated by almost two hundred years, and operating under completely different cultural and scientific presuppositions -- were driven by identical biases and ended up replicating the same errors in exactly the same manner thousands of times is an extremely implausible scenario.
Hi Anonymous,
As far as I can tell, while the measurers were blinded to the volumes being produced (which, I must say, even that I'm a little dubious of: even if I don't know exactly what volume I'm producing, I would still know that the more balls I pour in, and the more I shake the skull ("intermittently") to make sure all the little spaces are filled up, the more the final volume is going to be), the measurers were not (once again, as far as I can tell: if I just totally missed that part of the methods, please correct me) blinded to the origin of the skull, which I think is the far more important factor to blind for (to test this particular hypothesis).
1. Your initial post claimed that the authors ignored the entire issue of blinding ("the authors didn't even do that"). Now you are saying you think they should have used a different method of blinding than the one they did. Way to move the goalposts.
2. The method you suggest (blind to ethnicity of skull) is not feasible for the exact reason you point out - you can still infer ethnicity from morphology, not to mention old writing on the skull.
3. Blinding to volume does work, unless you can answer this question: Assume the measurer is consciously trying to match Morton's measurements (so he is completely biased). How can he do that without knowing what volumes he is producing? Just trying to pack in more/less beads on some skulls doesn't work, because then he could be overshooting/undershooting Morton's measurements.
4. The measurements were done three separate times for each skull and the results were virtually the same for each skull every time, as shown in the paper. So in your scenario, the biased measurer would have to blindly guess right three separate times.
You know, it's okay to admit you wrote hastily and are mistaken.
Post a Comment