Carol Nickerson writes:

I’ve written my fair share of letters to the editor and commentaries over the years, most of them languishing in the file drawer. It used to be impossible to get them published. The situation has improved a bit, but not enough.

In any case, I never think about the sex of the author(s). Or the race, for that matter. I do consider the seniority of the authors. When the lead author is junior — a graduate student, a post doc, a young assistant professor — I much prefer to bring problems to his or her attention using private e-mail. It’s more collegial and constructive. I know that many younger people in psychology have not had good statistical training at their colleges and universities. I like to attach relevant references of which he or she may not be be aware and so on. I’ve also done this with authors who are more senior. Interestingly, I have found that the junior authors are more likely to reply, more likely to initiate a corrigendum, etc. The senior authors are more likely to ignore my e-mail message.

The situation gets sticky when the lead author is a graduate student but the other author(s) are professors, and you know that said professors engage in a lot of questionable research practices or even research misconduct. Not too long ago, I discovered that an article published in Psychological Science re-used data from an article published earlier in another journal without so stating. In psychology, this is considered unethical. Of course, I had to alert the editor to this data re-use, but I worried a lot about the impact on the lead author (then a graduate student, now a post doc) and her career if the editor decided to retract the article. (He decided not to retract it, but to issue a corrigendum, which I think was the wrong decision.) Nick and I have had a lot of discussions about this dilemma. He thinks (a) that the student should know better and (b) that it is better for the student to have something like this happen earlier in his or her career, when recovery from such a setback is easier, than to have it happen later. I understand and have some sympathy with this view, but it still makes me feel terrible to go after a junior person.

I wonder if other people worry about this sort of thing. You could write a blog article about this sometime, Andrew, if you run out of other topics.

My reply: Let me break this up into three parts.

First, the question of contacting the author directly. I understand the appeal of this, but I usually don’t. Why not?

a. I don’t like conflict. It’s been my experience that when I do contact authors directly, they don’t like to admit error, and the conversation becomes awkward. When people send me their papers directly, I have no problem sending back by praise and criticism. But unsolicited criticism—and even unsolicited questions that are open-ended enough that they might imply criticism—that doesn’t always work so well.

b. There’s also a principle here, which is that published articles are . . . public. That’s what publishing means! So even if I can reach the authors directly, not everyone can, or will. I think there’s value in public comment on public papers. If people really don’t want their papers criticized in public, they shouldn’t go around publishing them and publicizing them.

Similarly, if you disagree with something I post here, please say so in a comment: that way others will see your reaction. It won’t just be me you’re talking to.

Second, the question of going easy on younger researchers. I don’t know about this. I published a false theorem when I was 28! I want that sort of thing corrected as soon as possible. One can also flip it around and ask, is it appropriate to be less nice to people, just cos they’re older?

I guess I’d like to think that it’s not mean to point out flaws in published work. Again, I’m happy for people to do this service for me, whether privately or publicly. I was grateful when someone sent me an airmail letter (!) with a counterexample to my false theorem, and I was grateful when someone wrote an angry blog post pointing out implausible estimates that I’d produced. In both cases, I would’ve been even happier if I’d done things right in the first place—but, conditional on the error, I appreciated the corrections.

I know that lots of you disagree with me on this issue, so by posting this I’m setting myself up for some criticism, but that’s fair. It’s good to get others’ perspectives.

The post Consider seniority of authors when criticizing published work? appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post Consider seniority of authors when criticizing published work? appeared first on All About Statistics.

OK, here’s a paper with a true theorem but then some false corollaries.

First the theorem:

The above is actually ok. It’s all true.

But then a few pages later comes the false statement:

This is just wrong, for two reasons. First, the relevant reference distribution is discrete uniform, not continuous uniform, so the normal CDF thing is at best just an approximation. Second, with Markov chain simulation, the draws theta_i^(l) are dependent, so for any finite L, the distribution of q_i won’t even be discrete uniform.

The theorem is correct because it’s in the limit as L approaches infinity, but the later statement (which I guess if true would be a corollary, although it’s not labeled as such) is false.

The error wasn’t noticed in the original paper because the method happened to work out on the examples. But it’s still a false statement, even if it happened to be true in a few particular cases.

Who are these stupid-ass statisticians who keep inflicting their errors on us??? False theorems, garbled data analysis, going beyond scientific argument and counterargument to imply that the entire field is inept and misguided, methodological terrorism . . . where will it all stop? Something should be done.

P.S. For those of you who care about the actual problem of statistical validation of algorithms and software which this is all about: One way to fix the above error is by approximating to a discrete uniform distribution on a binned space, with the number of bins set to something like the effective sample size of the simulations. We’re on it. And thanks to Sean, Kailas, and others for revealing this problem.

The post Bigshot statistician keeps publishing papers with errors; is there anything we can do to get him to stop??? appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post Bigshot statistician keeps publishing papers with errors; is there anything we can do to get him to stop??? appeared first on All About Statistics.

On Jan 27, 2017, at 12:24 PM, ** <**.**@**.com> wrote:

Hi Andrew,

I hope you are well. I work for ** and we are looking to chat to someone who knows about Freud – I read that you used to be an expert in Freud? Is that correct?

Background here.

The post I love when I get these emails! appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post I love when I get these emails! appeared first on All About Statistics.

The riddle this week on The Riddler was about finding the largest sequence of integers between 1 and 100 such that each integer is only used once and always followed by a multiple or a factor. A basic R code searching at random [and programmed during a massive downpour on Skye] led to a solution of 69:

although there is no certainty this is the best p… And the solutions posted the next week showed sequences with length 77! [Interestingly, both posted solutions have a sequence starting with 87. And they seem to exploit the graph of connections between integers in a much more subtle way that my random exploration of subsequences.]

Please comment on the article here: R – Xi'an's Og

The post sequence riddle appeared first on All About Statistics.

Diana Senechal writes:

Yesterday Cari Romm reported that researchers had found a relation between personality traits and cortex shape: “People who scored higher on openness tended to have thinner and smoother cortices, while those who scored high on neuroticism had cortices that were thicker and more wrinkled.”

I [Senechal] looked up the study itself (https://academic.oup.com/scan/article/doi/10.1093/scan/nsw175/2952683/Surface-based-morphometry-reveals-the) and read:

These findings demonstrate that anatomical variability in prefrontal cortices is linked to individual differences in the socio-cognitive dispositions described by the FFM.

At the end of the Statistical Methods section, the authors state:

To control for multiple comparisons in the SBM analysis, cluster correction was completed using Monte Carlo simulation (vertex-wise cluster forming threshold of P < 0.05) at a cluster-wise P (CWP) value of 0.05.

In the discussion, they return to this issue:

In terms of potential shortcomings, it can be surmised that a relatively large number of statistical tests was performed. This could have increased the probability of type I errors, although the use of a large sample size and state-of-art methods to correct for multiple comparisons should have mitigated against this problem.

Are they overly confident that they corrected for multiple comparisons?

My reply: To paraphrase Flava Flav, 0.05 is a joke. If there are multiple potential comparisons to be considered, I think the researchers should study all of them and analyze them using a mutilevel model. That makes a lot more sense than picking just one comparison and trying to correct for things. Why should you care about just one comparison? It strains credulity to think that, when you’re talking about something as multifaceted as “scoring high on openness,” that it would just show up in one particular dimension. I mean, sure, you never know: useful discoveries could come from this approach. But I’m skeptical, as the statistical method seems like such a poor match to the problem being studied.

P.S. Senechal supplied the above picture. I have no idea how thick and wrinkly is the cortex of this cat.

The post What explains my lack of openness toward this research claim? Maybe my cortex is just too damn thick and wrinkled appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post What explains my lack of openness toward this research claim? Maybe my cortex is just too damn thick and wrinkled appeared first on All About Statistics.

Please comment on the article here: Junk Charts

The post Starstruck and doubled over: losing poise over Indian charts appeared first on All About Statistics.

Recently, I was asked whether SAS can perform a principal component analysis (PCA) that is robust to the presence of outliers in the data. A PCA requires a data matrix, an estimate for the center of the data, and an estimate for the variance/covariance of the variables. Classically, these estimates are the mean and the Pearson covariance matrix, respectively, but if you replace the classical estimates by robust estimates, you can obtain a robust PCA.

This article shows how to implement a classical (non-robust) PCA by using the SAS/IML language and how to modify that classical analysis to create a robust PCA.

A classical principal component analysis in SAS

the FACTOR procedure.

Let's use PROC PRINCOMP perform a simple PCA. The PROC PRINCOMP results will be the basis of comparison when we implement the PCA in PROC IML. The following example is taken from

proc princomp data=Crime /* add COV option for covariance analysis */ outstat=PCAStats_CORR out=Components_CORR plots=score(ncomp=4); var Murder Rape Robbery Assault Burglary Larceny Auto_Theft; id State; ods select Eigenvectors ScreePlot '2 by 1' '4 by 3'; run;   proc print data=Components_CORR(obs=5); var Prin:; run;

To save space, the output from PROC PRINCOMP is not shown, but it includes a table of the eigenvalues and principal component vectors (eigenvectors) of the correlation matrix, as well as a plot of the scores of the observations, which are the projection of the observations onto the principal components. The procedure writes two data sets: the eigenvalues and principal components are contained in the PCAStats_CORR data set and the scores are contained in the Components_CORR data set.

A classical principal component analysis in SAS/IML

Assume that the data consists of n observations and p variables and assume all values are nonmissing. If you are comfortable with multivariate analysis, a principal component analysis is straightforward: the principal components are the eigenvectors of a covariance or correlation matrix, and the scores are the projection of the centered data onto the eigenvector basis. However, if it has been a few years since you studied PCA, Abdi and Williams (2010) provides a nice overview of the mathematics. The following matrix computations implement the classical PCA in the SAS/IML language:

proc iml;
reset EIGEN93;       /* use "9.3" algorithm, not vendor BLAS (SAS 9.4m3) */
varNames = {"Murder" "Rape" "Robbery" "Assault" "Burglary" "Larceny" "Auto_Theft"};
use Crime;
  read all var varNames into X;  /* X = data matrix (assume nonmissing) */
close;
 
S = cov(X);                      /* estimate of covariance matrix */
R = cov2corr(S);                 /* = corr(X) */
call eigen(D, V, R);             /* D=eigenvalues; columns of V are PCs */
scale = T( sqrt(vecdiag(S)) );   /* = std(X) */
B = (X - mean(X)) / scale;       /* center and scale data about the mean */
scores = B * V;                  /* project standardized data onto the PCs */
 
print V[r=varNames c=("Prin1":"Prin7") L="Eigenvectors"];
print (scores[1:5,])[c=("Prin1":"Prin7") format=8.5 L="Scores"];

The principal components (eigenvectors) and scores for these data are identical to the same quantities that were produced by PROC PRINCOMP. In the preceding program I could have directly computed R = corr(X) and scale = std(X), but I generated those quantities from the covariance matrix because that is the approach used in the next section, which computes a robust PCA.

If you want to compute the PCA from the covariance matrix, you would use S in the EIGEN call, and define B = X - mean(X) as the centered (but not scaled) data matrix.

A robust principal component analysis

This section is based on a similar robust PCA computation in Wicklin (2010). The main idea behind a robust PCA is that if there are outliers in the data, the covariance matrix will be unduly influenced by those observations. Therefore you should use a robust estimate of the covariance matrix for the eigenvector computation. Also, the arithmetic mean is unduly influenced by the outliers, so you should center the data by using a robust estimate of center before you form the scores.

SAS/IML software provides

/* get robust estimates of location and covariance */ N = nrow(X); p = ncol(X); /* number of obs and variables */ optn = j(8,1,.); /* default options for MCD */ optn[4]= floor(0.75*N); /* override default: use 75% of data for robust estimates */ call MCD(rc,est,dist,optn,x); /* compute robust estimates */ RobCov = est[3:2+p, ]; /* robust estimate of covariance */ RobLoc = est[1, ]; /* robust estimate of location */   /* use robust estimates to perform PCA */ RobCorr = cov2corr(RobCov); /* robust correlation */ call eigen(D, V, RobCorr); /* D=eigenvalues; columns of V are PCs */ RobScale = T( sqrt(vecdiag(RobCov)) ); /* robust estimates of scale */ B = (x-RobLoc) / RobScale; /* center and scale data */ scores = B * V; /* project standardized data onto the PCs */

Notice that the SAS/IML code for the robust PCA is very similar to the classical PCA. The only difference is that the robust estimates of covariance and location (from the MCD call) are used instead of the classical estimates.

If you want to compute the robust PCA from the covariance matrix, you would use RobCov in the EIGEN call, and define B = X - RobLoc as the centered (but not scaled) data matrix.

You can create a score plot to compare the classical scores to the robust scores. The Getting Started example in the PROC PRINCOMP documentation shows the classical scores for the first three components. The documentation suggests that Nevada, Massachusetts, and New York could be outliers for the crime data.

You can write the robust scores to a SAS data set and used the SGPLOT procedure to plot the scores of the classical and robust PCA on the same scale. The first and third component scores are shown to the right, with abbreviated state names serving as labels for the markers. (Click to enlarge.) You can see that the robust first-component scores for California and Nevada are separated from the other states, which makes them outliers in that dimension. (The first principal component measures overall crime rate.) The robust third-component scores for New York and Massachusetts are also well-separated and are outliers for the third component. (The third principal component appears to contrast murder with rape and auto theft with other theft.)

Because the crime data does not have huge outliers, the robust PCA is a perturbation of the classical PCA, which makes it possible to compare the analyses. For data that have extreme outliers, the robust analysis might not resemble its classical counterpart.

If you run an analysis like this on your own data, remember that eigenvectors are not unique and so there is no guarantee that the eigenvectors for the robust correlation matrix will be geometrically aligned with the eigenvectors from the classical correlation matrix. For these data, I multiplied the second and fourth robust components by -1 because that seems to make the score plots easier to compare.

Summary

In summary, you can implement a robust principal component analysis by using robust estimates for the correlation (or covariance) matrix and for the "center" of the data. The MCD subroutine in SAS/IML language is one way to compute a robust estimate of the covariance and location of multivariate data. The SAS/IML language also provides ways to compute eigenvectors (principal components) and project the (centered and scaled) data onto the principal components.

You can download the SAS program that computes the analysis and creates the graphs.

As discussed in Hubert, Rousseeuw, and Branden (2005), the MCD algorithm is most useful for 100 or fewer variables. They describe an alternative computation that can support a robust PCA in higher dimensions.

References

• Abdi and Williams (2010) "Principal component analysis," WIREs Computational Statistics, p 433–459.
• Hubert, Rousseeuw, and Branden (2005) "ROBPCA: A New Approach to Robust PrincipalComponent Analysis," Technometrics.
• Wicklin (2010) "Rediscovering SAS/IML Software: Modern Data Analysis for the Practicing Statistician." Robust PCA on p. 9–11.

The post Robust principal component analysis in SAS appeared first on The DO Loop.

Please comment on the article here: The DO Loop

The post Robust principal component analysis in SAS appeared first on All About Statistics.

Kevin Lewis points us to this article by Ryan Enos, Anthony Fowler, and Christopher Havasy, who write:

This article examines the negative effect fallacy, a flawed statistical argument first utilized by the Warren Court in Elkins v. United States. The Court argued that empirical evidence could not determine whether the exclusionary rule prevents future illegal searches and seizures because “it is never easy to prove a negative,” inappropriately conflating the philosophical and arithmetic definitions of the word negative. Subsequently, the Court has repeated this mistake in other domains, including free speech, voting rights, and campaign finance. The fallacy has also proliferated into the federal circuit and district court levels. Narrowly, our investigation aims to eradicate the use of the negative effect fallacy in federal courts. More broadly, we highlight several challenges and concerns with the increasing use of statistical reasoning in court decisions. As courts continue to evaluate statistical and empirical questions, we recommend that they evaluate the evidence on its own merit rather than relying on convenient arguments embedded in precedent.

Damn. I thought I’d never say this, but Earl Warren knows about as much about statistics as the editors of Perspectives on Psychological Science and the Proceedings of the National Academy of Sciences.

I’d hope for better from the U.S. Supreme Court. Seriously.

Lots of psychology professors, including editors at top journals, don’t know statistics but feel the need to pretend that they do. But nobody expects a judge to know statistics. So one would think this would enable these judges to feel free to contact the nation’s top statistical experts for advice as needed. No need to try to wing it, right?

As Mark Twain might have said, it’s not what you don’t know that kills you, it’s what you know for sure that ain’t true.

The post The Supreme Court can’t do statistics. And, what’s worse, they don’t know what they don’t know. appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post The Supreme Court can’t do statistics. And, what’s worse, they don’t know what they don’t know. appeared first on All About Statistics.

So. I read more by and about Irwin Shaw. I read Shaw’s end-of-career collection of short stories and his most successful novel, The Young Lions, and also the excellent biography by Michael Shnayerson. I also read Adam Begley’s recent biography of John Updike, which was also very good, and it made be sad that probably very few people actually read it. Back in the old days, a major biography of a major writer would’ve had a chance of attracting some readers.

John Updike was a master of the slice of life and also created one very memorable character in Rabbit. Irwin Shaw was known as a “storyteller” but I’m not quite sure what that means, as his stories didn’t have such memorable plots. Kinda like a composer whose music is engrossing but at the same time has no memorable tunes. The guy was no John Le Carre or Stephen King.

In his New York Times obituary, Herbert Mitgang wrote, “Stylistically, Mr. Shaw’s short stories were noted for their directness of language, the quick strokes with which he established his different characters, and a strong sense of plotting.” Well put. Quick strokes. His characters didn’t come to life, but their situations and predicaments did. In that way he had a lot in common with Updike.

One thing Shaw did have was a combination of emotional sympathy, real-world grit, and social observation. Some similarity here with John O’Hara, but O’Hara’s situations always seemed a bit more schematic to me, whereas Shaw’s characters seem to be in real situations (even if they’re not, ultimately, real characters).

Updike and Shaw had different career trajectories. Updike started at the top and stayed here. Shaw started at the top and worked his way down. OK, even at the end he was selling lots of copies, but his books weren’t getting much respect (and, at least according to his biographer, they had some strong moments but they weren’t great; I can’t bring myself to try to read these novels myself). On the other hand, I’ve tried to read a couple of Updike’s later novels and I wasn’t so impressed. From my perspective, Updike redeemed himself by writing a lot of excellent literary journalism. As they got older, both Updike and Shaw reduced their output of short stories, maintaining the high quality in both cases.

Speaking of John Updike, if he were around today I expect he’d’ve had something to say about those rural Pennsylvanians who voted for Donald Trump. Being a rural Pennsylvanian. And John O’Hara, as a Pennsylvanian, and Roman Catholic, and an all-around resentful person: he would’ve had something to say about Trump voters from all those groups. Then we could bring in Lorrie Moore to explain Hillary Clinton voters to us. Hey, here it is—ok, that didn’t work: Moore doesn’t like Clinton. Hmmm, lots of people don’t like Hillary Clinton, but she did get 51% of the two-party vote. We’ll have to find some expert to explain those voters to us.

The post Irwin Shaw, John Updike, and Donald Trump appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post Irwin Shaw, John Updike, and Donald Trump appeared first on All About Statistics.

Please comment on the article here: Big Data, Plainly Spoken (aka Numbers Rule Your World)

The post A call for less automation, more transparency in digital advertising appeared first on All About Statistics.