This study was submitted to Track II for review by the Editorial aboard at PNAS (Proceedings of the National Academy of Sciences of the United States of America) 4 years ago. The study was conducted by psychologists at Yale and editing by a professional at University of Melbourne, AU.

You can find the full article, free at https://www.jstor.org/stable/43278786?mag=when-do-we-have-em.... The article contains a full materials and methods section to aid with replication. The study design was simple enough to allow for relative ease of a repeat study. The authors applied relatively simple and traditional statistical methods (a more complex ANOVA) along with a mechanism to ensure that they were measuring empathy correctly by correlating measure of empathy with a separate measure of personal distress. You can dig into the details in the article, but it all seems to line up well.

In summary, you could probably replicate this yourself using Survey Monkey and Wolfram Alpha.

So am I right in concluding that the design was simple enough to be mostly forking-paths-free, and there weren't any oddly specific choices made in the analysis (to dig a "significant" result out of the data)?

The fact that it could be replicated from details in the paper without excessive expense or difficulty is also reassuring, even before knowing if such a replication has taken place.

Absolutely, this was a straightforward research design with a straightforward statistical analysis. The statistical significance was strong.

To give you a better mental model: they essentially came up with two sets of 6 words where one set was paired to representing empathy and the other to personal distress. The authors then grouped doctors to receive either biological or psychosocial stimuli and asked them to rate themselves on each word after being presented the stimuli. Statistical analysis was simply comparing the results.

Another noteworthy item is that they also included another set of six words to be rated, which they did not use in the statistical analysis, but which they claimed were used as fillers (words like angry, resentful, joyous, etc. - implying that they took care to not make it readily apparent what specific variable they were measuring).

Study integrity aside, they found that schizophrenia vignettes elicited almost the same magnitude of empathy for either bio or psychosocial explanations and depression showed the widest gap.

In all honesty, I’d recommend that you spend an evening perusing the actual research article as it is a good example of academic social science research, in my opinion.

1) Its not some random guy's job to replicate this stuff. Its the job of people getting paid to do this type of research (ie, the peers) to verify each other's work.

2) You're right, its been published for four years, seems to have had some sort of impact (since the news has mentioned it), and doesn't seem that complicated or expensive to do in principle. So, is there a direct replication?

1) I was replying to someone who asked “Given the journal and the time it was published, are we sure this is research that would replicate?”, as I am a person with “more knowledge of statistics and experimental methods”, the kind of person to whom the question was being asked.

2) I can look into your question, later, but you can answer it by searching for citations of the original research.

>"I can look into your question, later, but you can answer it by searching for citations of the original research."

I think we both know that direct replications remain rare for this type of research and it is highly unlikely that one exists.

Yeah, unfortunately, that is a safe assumption these days.

I did a brief search, though, and the paper appears to have made some rounds in the NIH.