A world of Wansinks in medical research: “So I guess what I’m trying to get at is I wonder how common it is for clinicians to rely on med students to do their data analysis for them, and how often this work then gets published”

In the context of a conversation regarding sloppy research practices, Jordan Anaya writes:

It reminds me of my friends in residency. Basically, while they were med students for some reason clinicians decided to get them to analyze data in their spare time. I’m not saying my friends are stupid, but they have no stats or programming experience, so the fact that they are the key data analyst for these data sets concerns me, especially considering they don’t have much free time to devote to the work anyways. So I guess what I’m trying to get at is I wonder how common it is for clinicians to rely on med students to do their data analysis for them, and how often this work then gets published.

I asked Jordan if I could post this observation, and he added:

Here’s some more details. I recently got an email from a friend saying they needed my help. They had previously taken a short introduction to R course and used those skills to analyze some data for a clinician. However, recently that clinician sent them some more data and now their code no longer worked so they asked me for help [“I need to do a proportion and see if it is significant . . .”]

They gave me the file and I confirmed R couldn’t read it (at least with the default read.csv—I’m not an R expert), so I looked at it in Python. There were extra rows, some missing data, and the job variable sometimes had commas in it so you couldn’t use comma as a delimiter. Anyways, I provided them with the information they wanted, and didn’t mention that some of their requests sounded like p-hacking.

My other friend is familiar with my Wansink work and what p-hacking is, and he told me he recently ran a bunch of tests on a data set for a clinician so I told him he p-hacked for that clinician, and he said that yes, he did.

Since I didn’t do my internship years I don’t know how these collaborations come about. I imagine a clinician needs help analyzing data and doesn’t have grad students so they turn to med students, and the med students don’t want to turn down a chance at getting their name on a paper.

As with Wansink, the big problem is not p-hacking but just a lack of understanding of the data, a disaster of the data collection/processing/analysis steps, and a lack of adult responsibility throughout.

P.S. Anaya adds:

I have another update for you. I talked with my friend today and they have refused to do any more work on the project! This was supposed to be a short analysis during their 4th year of med school but I guess there was some data handling problems so the clinician got someone else to get the data out of the database which I guess is what I saw last year (which is when the med student was now a first year resident and didn’t have time to work on the project). Anyways now the med student (now a second year resident) doesn’t trust the data enough to devote any more of their time to the project! So we have people who can’t handle data giving their pu pu platter to a med student (now second year resident) who doesn’t know how to program. I’m really starting to understand what happened with some of Wansink’s papers now.