Under the heading, “Latino approval of Donald Trump,” Tyler Cowen writes:
From a recent NPR/PBS poll:
African-American approval: 11%
White approval: 40%
Latino approval: 50%
He gets 136 comments, many of which reveal a stunning ignorance of polling. For example, several commenters seem to think that a poll sponsored by National Public Radio is a poll of NPR listeners.
Should NPR waste its money on commissioning generic polls? I don’t think so. There are a zillion polls out there, and NPR—or just about any news organization—has, I believe, better things to do than create artificial headlines by asking the same damn polling question that everyone else does.
In any case, it’s a poll of American adults, not a poll of NPR listeners.
The other big mistake that many commenters made was to take the poll result at face value. Cowen did that too, by reporting the results as “Latino approval of Donald Trump,” rather than “One poll finds . . .”
A few things are going on here. In no particular order:
1. Margin of error. A national poll of 1000 people will have about 150 Latinos. The standard error of a simple proportion is then 0.5/sqrt(150) = 0.04. So, just to start off, that 50% could easily be anywhere between 42% and 58%. And, of course, given what else we know, including other polls, 42% is much more likely than 58%.
That said, even 42% is somewhat striking in that you might expect a minority group to support the Republican president less than other groups. One explanation here is that presidential approval is highly colored by partisanship, and minorities tend to be less partisan than whites—we see this in many ways in lots of data.
2. Selection. Go to the linked page and you’ll see dozens of numbers. Look at enough numbers and you’ll start to focus on noise. The garden of forking paths—it’s not just about p-values. Further selection is that it’s my impression that Cowen enjoys posting news that will fire up his conservative readers and annoy his liberal readers—and sometimes he’ll mix it up and go the other way.
3. The big picture. Trump’s approval is around 40%. It will be higher for some groups and lower for others. If your goal is to go through a poll and find some good news for Trump, you can do so, but it doesn’t alter the big picture.
I searched a bit on the web and found this disclaimer from the PBS News Hour:
President Trump tweeted about a PBS NewsHour/NPR/Marist poll result on Tuesday, highlighting that his approval rating among Latinos rose to 50 percent. . . .
However, the president overlooked the core finding of the poll, which showed that 57 percent of registered voters said they would definitely vote against Trump in 2020, compared to just 30 percent who said they would back the president. The president’s assertion that the poll shows an increase in support from Latino voters also requires context. . . .
But only 153 Latino Americans were interviewed for the poll. The small sample size of Latino respondents had a “wide” margin of error of 9.9 percentage points . . . [Computing two standard errors using the usual formula, 2*sqrt(0.5*0.5/153) gives 0.081, or 8.1 percentage points, so I assume that the margin of error of 9.9 percentage points includes a correction for the survey’s design effect, adjusting for it not being a simple random sample of the target population. — AG.] . . .
[Also] The interviews in the most recent PBS NewsHour/NPR/Marist poll were conducted only in English. . . .
They also report on other polls:
According to a Pew Research Center’s survey from October, only 22 perfect of Latinos said they approved of Trump’s job as president, while 69 percent said they disapproved. . . . Pew has previously explained how language barriers and cultural differences could affect Latinos’ responses in surveys.
That pretty much covers it. But then the question arises: Why did NPR and PBS commission this poll in the first place? There are only a few zillion polls every month on presidential approval. What’s the point of another poll? Well, for one thing, this gets your news organization talked about. They got a tweet from the president, they’re getting blogged about, etc. But is that really what you want to be doing as a news organization: putting out sloppy numbers, getting lots of publicity, then having to laboriously correct the record? Maybe just report some news instead. There’s enough polling out there already.