What explains my lack of openness toward this research claim? Maybe my cortex is just too damn thick and wrinkled

August 9, 2017
By

(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)

Diana Senechal writes:

Yesterday Cari Romm reported that researchers had found a relation between personality traits and cortex shape: “People who scored higher on openness tended to have thinner and smoother cortices, while those who scored high on neuroticism had cortices that were thicker and more wrinkled.”

I [Senechal] looked up the study itself (https://academic.oup.com/scan/article/doi/10.1093/scan/nsw175/2952683/Surface-based-morphometry-reveals-the) and read:

These findings demonstrate that anatomical variability in prefrontal cortices is linked to individual differences in the socio-cognitive dispositions described by the FFM.

At the end of the Statistical Methods section, the authors state:

To control for multiple comparisons in the SBM analysis, cluster correction was completed using Monte Carlo simulation (vertex-wise cluster forming threshold of P < 0.05) at a cluster-wise P (CWP) value of 0.05.

In the discussion, they return to this issue:

In terms of potential shortcomings, it can be surmised that a relatively large number of statistical tests was performed. This could have increased the probability of type I errors, although the use of a large sample size and state-of-art methods to correct for multiple comparisons should have mitigated against this problem.

Are they overly confident that they corrected for multiple comparisons?

My reply: To paraphrase Flava Flav, 0.05 is a joke. If there are multiple potential comparisons to be considered, I think the researchers should study all of them and analyze them using a mutilevel model. That makes a lot more sense than picking just one comparison and trying to correct for things. Why should you care about just one comparison? It strains credulity to think that, when you’re talking about something as multifaceted as “scoring high on openness,” that it would just show up in one particular dimension. I mean, sure, you never know: useful discoveries could come from this approach. But I’m skeptical, as the statistical method seems like such a poor match to the problem being studied.

P.S. Senechal supplied the above picture. I have no idea how thick and wrinkly is the cortex of this cat.

The post What explains my lack of openness toward this research claim? Maybe my cortex is just too damn thick and wrinkled appeared first on Statistical Modeling, Causal Inference, and Social Science.



Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

Tags:


Subscribe

Email:

  Subscribe