It’s hard not to get excited about the brave new world of neuroscience, and what we’ve learned about the brain. But is that enthusiasm premature? Researchers at the University of California San Francisco, who discovered that brain development research may be skewed, think so.
One of the basic principles of research is that the sample of people selected for a study should be similar to the larger population. Studies that deviate from that standard run the risk of producing biased results that reflect the characteristic of the sample rather than the population at large. When an entire field of research is based on misrepresentative samples of people, we run the risk of being off the mark in our conclusions.
This misfortune may be plaguing the field of neuroscience according to a paper by researchers at UC San Francisco and published in Nature Communications. In it they reviewed existing brain imaging research and discovered that much of what we know about human brain development comes from samples that don’t resemble the majority of people. This means that our assumptions about how the brain develops, and how the brains of those with illnesses like depression, anxiety and autism differ, is based almost entirely on non-representative samples.
Their conclusions were based on a review of the brain scans of 1162 children aged 3 to 18 years from the Pediatric Imaging, Neurocognition and Genetics (PING) Study. Children in the PING sample, from which many conclusions about brain development have been drawn, tended to come from families that were more affluent and educated than those of typical American children.
To see whether or not using this sample created skewed results, the authors created a “weighted” version of the dataset that more accurately reflected the demographic characteristics (race/ethnicity, sex, income, and parent education) of the US population. They then looked to see whether brain images illustrating typical child and adolescent brain development would differ from the PING dataset.
Biases in Brain Research
Remarkably, their analyses showed that the order in which brain regions develop differed from that of the PING sample. Whereas PING sample data showed that the frontal, temporal and occipital regions of the brain all develop at the same time, results from the “weighted” sample suggested that the brain develops from back to front, beginning with sensory-motor areas and finishing in the development of regions in the front of the brain involved in higher order thinking. The “weighted” data also revealed that certain regions of the brain mature as many as four years earlier than the “unweighted” data indicated. Considering that brain imaging research can inform educational policy and practice, these discrepancies are considerable, and problematic.
Unfortunately, conducting research with non-representative samples isn’t new. In 2010, a group of Canadian scientists at the University of British Columbia published the infamous “WEIRD paper”. Here they reviewed a large database of human behavioral studies only to find that the majority of claims about human behavior made in the world’s leading journals were based on research conducted entirely on Western, educated, industrialized, rich, and democratic (W.E.I.R.D) societies. In other words, most of what science has told us about human behavior comes from samples of college students who don’t resemble the global population.
What these critiques of the research tell us is that we need to pay better attention to who we are studying, why we select certain people as participants, and the implications of those decisions. While not all studies can realistically be conducted on a representative sample of children or adults, it is important that we are mindful of the fact that not all research applies to all people, and refrain from making overgeneralizations in our interpretation of brain and behavior research.
Source
Kaja Z. LeWinn, Margaret A. Sheridan, Katherine M. Keyes, Ava Hamilton, Katie A. McLaughlin. Sample composition alters associations between age and brain structure. Nature Communications, 2017; 8 (1) DOI: 10.1038/s41467-017-00908-7
Comments