Unfortunately, small MRI samples often give strong associations by accident. For example, let’s say you want to know if there is a relationship between eye color and preference for strawberries. If you look at enough groups of 25 random people, you will eventually stumble upon a group in which blue-eyed people like strawberries much more than brown-eyed people. But if this study is done by five independent research groups, and only one of them finds an association between eye color and love for berries, that group is most likely to publish their results, even though the results are the least representative. This is because journals have historically favored unexpected correlations over non-correlation findings, a phenomenon known as publication bias. “The paradoxical effect is that the most incorrect answer is published if you use a small sample,” says Nico Dosenbach, assistant professor of neuroscience at the University of Washington and author of the Nature study.
Scientists across disciplines have known about this dynamic for a long time, but a paper in Nature was able to pinpoint—at least in the case of whole-brain association studies—how many participants are needed to avoid it. Using MRI data from approximately 50,000 people, the authors looked for links between brain structure or activity and complex psychological traits in groups of varying sizes. The subjects had to average in the thousands so that the studies could be reliably replicated.
The fact that so many association studies are not informative enough and often not tested on other subject groups before publication has led to reports of innumerable associations between brain traits and psychiatric disorders that are probably unreliable. They can be intimidating and stigmatizing. “If you see a certain pattern of brain activation in a person with a psychiatric diagnosis, it doesn’t mean that it is causing the disorder or the symptoms,” Jernigan says. “It’s just an association.
But the conclusion of the Nature article only applies to studies that compare MRI scans of multiple people in order to identify differences between them associated with complex mental traits. On the other hand, neuroimaging studies that show brain changes occurring in individuals can be reliable even with very small numbers of participants. For example, the first notable article demonstrating that The brain of most people works in much the same way as it appeared in the journal Science in 2001. and included only six participants, says Russell A. Poldrak, professor of psychology at Stanford University. The researchers in this study recorded each subject’s brain activity while viewing images of cats, faces, artificial objects, and nonsensical images. It didn’t matter that each brain was unique—the changes that occurred in that brain could be related to seeing different types of images. The templates were then tested and found to correctly predict, based on brain activity, what the participant saw. These general patterns, along with other evidence, says Poldrack, established that “when people engage in certain types of mental functions, certain areas of the brain are activated.”
This realization that we tend to share brain patterns raises the tantalizing possibility that somewhere in the variation between the two lies an explanation for why some people have a particular trait or set of symptoms that others don’t. But it is extremely difficult to separate meaningful differences from the myriad random differences that exist between all kinds of brains. One way to try to do this is to compare the MRI scans of thousands of people and look for a variant—say, a certain neural connection pattern—that is more common in people with a certain psychological condition. Recent advances in MRI technology and the ability to analyze vast amounts of data have made this kind of effort possible. For example, tStudy of the cognitive development of the brain of adolescents has enrolled almost 12,000 children in the United States between the ages of 9 and 10 whose brains will be regularly scanned as young adults. The study will also track socioeconomic variables such as parental income and psychological characteristics such as resilience to see how they intertwine with brain development. “Without such a study, you would never have been able to address these questions,” says Jernigan, director of the study’s clearing house.