“The following needs to be an immutable law of journalism: when someone with no track record comes into a field claiming to be able to do a job many times better for a fraction of the cost, the burden of proof needs to shift quickly and decisively onto the one making the claim. The reporter simply has to assume the claim is false until substantial evidence is presented to the contrary.”

January 17, 2018
By

(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)

Mark Palko writes:

The following needs to be an immutable law of journalism: when someone with no track record comes into a field claiming to be able to do a job many times better for a fraction of the cost, the burden of proof needs to shift quickly and decisively onto the one making the claim. The reporter simply has to assume the claim is false until substantial evidence is presented to the contrary.

Yup. This is related to advice I give to young researchers giving presentations or writing research papers:

1. Describe the problem you have that existing methods can’t solve.

2. Show how your new method solves the problem.

3. Explain how your method works.

4. Explain why, if your idea is so great, how come all the people who came before you were not already doing it.

There are lots of possibilities for step 4. Maybe your new idea is only now possible because of new technology, or network effects, or your idea flowed from earlier ideas that people only recently realized were effective, or maybe it was an idea taken from another area and introduced into your field. Whatever. The point is, if you don’t answer question 4, your responses to question 1, 2, and 3 are suspect.

Palko’s example concerned a newspaper story about “of entrepreneurs and Ivy League grads from the USA rescuing the poor children of Africa from poverty and ignorance.” Here’s Palko:

If you take down a few pages, there is some good, substantial reporting by Peg Tyre on this story. Unfortunately, as is all too often the case with New York Times Magazine articles, the first act is almost entirely credulous and laudatory. For example:

By 2015, Bridge was educating 100,000 students, and the founders claimed that they were providing a “world-class education” at “less than 30 percent” of what “the average developing country spends per child on primary education.” This would represent a remarkable achievement. None of the founders had traditional teaching experience. May had been an unpaid teacher at a school in China; Kimmelman worked with teachers and administrators developing an ed-tech company. How had they pulled it off? In interviews and speeches, they credited cutting-edge education technology and business strategies — the company monitors and stores a wide range of data on subjects including teacher absenteeism, student payment history and academic achievement — along with their concern for the well-being of the world’s poorest children. That potent mixture, they said, had allowed them to begin solving a complex and intractable problem: how to provide cheap, scalable, high-quality schooling for the most vulnerable, disadvantaged children on earth.

So, yes, question #4 is addressed in the news article—but the problem is the intro is entirely presented from the perspective of the boosters, the people who have a stake in selling their idea. It’s ok when giving a talk to sell your idea—there will be others out there to present their own, different views on the topic. But a newspaper should do better.

Palko continues:

Later on in the piece, Tyre actually does start digging into the business model, where the key drivers seem to be using cheap, substandard buildings, hiring undertrained and unqualified instructors, employing strong-arm collection tactics, and doing lots of marketing.

So Palko’a complaint is not with all of the article, just how it starts. Still, I think he’s making a good point. When we hear bold claims, our starting point should be disbelief. Assume the claim is false until substantial evidence is presented to the contrary.

P.S. Palko points here to another example of ridiculously credulous journalism.

The post “The following needs to be an immutable law of journalism: when someone with no track record comes into a field claiming to be able to do a job many times better for a fraction of the cost, the burden of proof needs to shift quickly and decisively onto the one making the claim. The reporter simply has to assume the claim is false until substantial evidence is presented to the contrary.” appeared first on Statistical Modeling, Causal Inference, and Social Science.



Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

Tags: , ,


Subscribe

Email:

  Subscribe