Alex Gamma writes in with what he describes as “an amusing little story” from two years ago:

When Deaton & Case published their study, and after your re-analysis had uncovered the missing age-correction, I’ve pointed out this issue to several news blogs that reported on the study, but were not aware of the problem (only about 2 in 5 or so responded).

One was the blog of Swiss National Television, titled “Many American men die in their prime”. They brought an interview with their U.S. correspondent about the increasingly dire living condition of these white men and how that might drive them into suicide. I pointed the problem out to them, linking to your Slate piece and adding a relevant quote.

The reply I got was (my translation):

The study was conducted by a Nobel prize winner in economics, published in PNAS [link to paper] and discussed in pretty much all the serious media, e.g. by Paul Krugman in the New York Times.

That’s also described in the Slate article. It’s true that there has now emerged a discussion about the interpretation of the numbers and the right or wrong numbers, respectively. The interview was simply about the content of the study, how it is to be understood and how it is discussed in the USA.

So far, that’s the not all too unexpected, dodging of the issue. But what they added at the end was almost brilliant:

In the last paragraph of the Slate article itself it says fittingly: I’m pretty sure some people will find major problems in my analysis. That’s how it goes, two steps forward and, if we’re lucky, only one step back.

They beat you with your own weapon!

At least that’s what I thought back then. I thought “don’t put a disclaimer that you don’t really mean, it might backfire”. Of course, I don’t know how you meant that disclaimer, probably it was dead serious.

Hmm, good point. I write “I’m pretty sure some people will find major problems in my analysis,” but that was too strong. Here’s what I should’ve said:

I wouldn’t be surprised if people find major problems in my analysis. And in any case I’m sure our understanding will evolve as we learn more. That’s how it goes, two steps forward and, if we’re lucky, only one step back.

Hmmm . . . “Our understanding will evolve as we learn more,” that’s a bit too vague. But it’s an improvement on my original phrasing. People do sometimes find major problems in my work, but saying “I’m sure” that would happen, yeah, that’s not accurate. Most of the time I think my published claims hold up.

The post Two steps forward, one step back appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post Two steps forward, one step back appeared first on All About Statistics.

Bob Carpenter writes:

Here’s what we do and what we recommend everyone else do:

1. code the model as straightforwardly as possible

2. generate fake data

3. make sure the program properly codes the model

4. run the program on real data

5. *If* the model is too slow, optimize *one step at a time* and for each step, go back to (3).

The optimizations can be of either the statistical or the computational variety. Slow iterations can be due to computational statistical problems with parameterization (requiring too many iterations) or due to slow code (each log density and derivative evaluation being too slow).

The post Workflow, baby, workflow appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post Workflow, baby, workflow appeared first on All About Statistics.

Happy holidays to all my readers! My greeting-card to you is an image of a self-similar Christmas tree. The image (click to enlarge) was created in SAS by using two features that I blog about regularly: matrix computations and ODS statistical graphics.

Self-similarity in Kronecker products

I have previously shown that the repeated Kronecker product of a binary matrix leads to self-similar structures.

Specifically, if M is a binary matrix then the Kronecker product M@M is a block matrix in which each 1 in the original matrix is replaced with a copy of M and each 0 is replaced by a zero block. (Here '@' is the Kronecker product (or direct product) operator, which is implemented in SAS/IML software.) If you repeat this process three or more times and create a heat map of the product matrix, the self-similar structure is apparent.

For example, the following 5 x 7 binary matrix has a 1 in positions that approximate the shape of a Christmas tree. The direct product M@M@M is a matrix that has 5³ = 125 rows and 7³ = 343 columns. If you use the HEATMAPDISC subroutine in SAS/IML to create a heat map of the direct product, you obtain the following image in which a green cell represents a 1:

proc iml;
M = {0 0 0 1 0 0 0,
     0 0 1 1 1 0 0,
     0 1 1 1 1 1 0,
     1 1 1 1 1 1 1,
     0 0 0 1 0 0 0};
Tree = M @ M @ M;     /* M is 5 x 7, so Tree is 125 x 343 */
 
ods graphics / width=300px height=450px  ANTIALIASMAX=50000;
call heatmapdisc(Tree) title="Happy Holidays to All my Readers!"
     colorramp={White Green} displayoutlines=0 ShowLegend=0;

That's not a bad result for three SAS/IML statements! You can see that the "tree" has a self-similar structure: the large tree is composed of 17 smaller trees, each of which is composed of 17 mini trees.

Adding a star

My readers are worth extra effort, so I want to put a star on the top of the tree. One way to do this is to use a scatter plot and plot a special observation by using a star symbol. To plot the "tree" by using a scatter plot, it is necessary to "unpack" the 125 x 343 matrix into a list of (x, y) values (this is a wide-to-long conversion of the matrix). You can use the NDX2SUB function to extract the row and column of each 1 in the matrix, as follows:

/* extract (row, column) pairs for matrix elements that are '1' */
idx = loc(Tree = 1);                /* indices for the 1s */
s = ndx2sub(dimension(Tree), idx);  /* convert indices to subscripts */
create Tree from s[c={"RNames" "CNames"}];
append from s;
close;
 
call symputx("XPos", range(s[,2])/2); /* midrange = horiz position of star */
quit;

The previous statements create a SAS data set called TREE that contains the (x, y) positions of the 1s in the direct product matrix. The statements also saved the midrange value to a macro variable. The following SAS procedures create the location of the star, concatenate the two data sets, and create a scatter plot of the result, which is shown at the top of this article.

data star;
x = &XPos; y = 0; 
run;
 
data Tree2;
set Tree star;
run;
 
ods graphics / width=600px height=800px  ANTIALIASMAX=50000;
title "Happy Holidays";
proc sgplot data=Tree2 noborder noautolegend;
scatter x=CNames y=RNames / markerattrs=(color=ForestGreen symbol=SquareFilled size=3);
scatter x=x y=y / markerattrs=(symbol=StarFilled size=20)             /* yellow star */
                  filledoutlinedmarkers markerfillattrs=(color=yellow);
yaxis reverse display=none;
xaxis display=none;
run;

Previous SAS-created Christmas cards

All year long I blog about how to use SAS for serious pursuits like statistical modeling, data analysis, optimization, and simulation. Consequently, I enjoy my occasional forays into a fun and frivolous topic such as how to create a greeting card with SAS. If you like seeing geeky SAS-generated images, here is a list of my efforts from previous years:

• 2016: An animation of falling snow
• 2015: Lo, how a polar rose e'er blooming
• 2014: A Christmas tree from Pascal's triangle
• 2013: A Christmas tree matrix and heat map
• 2012: A fractal Christmas tree
• 2011: An ASCII Christmas tree

Wherever you are and whatever holiday you celebrate, may you enjoy peace and happiness this season and in the New Year.

The post A self-similar Christmas tree appeared first on The DO Loop.

Please comment on the article here: The DO Loop

The post A self-similar Christmas tree appeared first on All About Statistics.

Claude Monnet’s paintings have nothing to do with the topic of this post.

Hi all,

This post deals with a strange phenomenon in R that I have noticed while working on unbiased MCMC. Reducing the problem to a simple form, consider the following code, which iteratively samples a vector ‘x’ and stores it in a row of a large matrix called ‘chain’ (I’ve kept the MCMC terminology).

dimstate = 100
nmcmc = 1e4
chain = matrix(0, nrow = nmcmc, ncol = dimstate)
for (imcmc in 1:nmcmc){
 if (imcmc == nrow(chain)){ #call to nrow
 }
 x = rnorm(dimstate, mean = 0, sd = 1)
 chain[imcmc,] = x #copying of x in chain
}

If you execute this code, you will see that it is surprisingly slow: it takes close to a minute on my computer. Now, consider the next block, which does exactly the same except that the vector ‘x’ is not copied into the matrix ‘chain’.

dimstate = 100
nmcmc = 1e4
chain = matrix(0, nrow = nmcmc, ncol = dimstate)
for (imcmc in 1:nmcmc){
if (imcmc == nrow(chain)){ #call to nrow
}
x = rnorm(dimstate, mean = 0, sd = 1)
# chain[imcmc,] = x #no more copying
}

This code runs nearly instantaneously. Could it be that just copying a vector in a matrix takes a lot of time? Sounds unlikely. Now consider this third block.

dimstate = 100
nmcmc = 1e4
chain = matrix(0, nrow = nmcmc, ncol = dimstate)
for (imcmc in 1:nmcmc){
if (imcmc == nmcmc){ #no call to nrow
}
x = rnorm(dimstate, mean = 0, sd = 1)
chain[imcmc,] = x #copying of x in chain
}

This code runs nearly instantaneously as well; this time ‘x’ is copied into ‘chain’, but the call to the nrow function is removed….?! What is nrow doing? It is meant to simply return dim(chain)[1], the first dimension of chain. So consider this fourth block.

dimstate = 100
nmcmc = 1e4
chain = matrix(0, nrow = nmcmc, ncol = dimstate)
for (imcmc in 1:nmcmc){
 if (imcmc == dim(chain)[1]){ #call to dim instead of nrow
 }
 x = rnorm(dimstate, mean = 0, sd = 1)
 chain[imcmc,] = x #copying of x in chain
}

This one also runs instantaneously! So replacing nrow(chain) by dim(chain)[1] solves the problem. Why?

The answer comes from R guru and terrific statistician Louis Aslett. I directly quote from an exchange of emails, since he brilliantly explains the phenomenon.

You probably know R stores everything by reference, so if I do:

x <- matrix(0, nrow=1e5, ncol=100)
y <- x

I actually only have one copy of the matrix in memory with two references to it. If I then do:

x[1,1] <- 1

R will first make a copy of the whole matrix, update x to point to that and then change the first element to one. This idea is used when you pass a variable to a standard (i.e. non-core, non-primitive) R function, which nrow is: it creates a reference to the variable you pass so that it doesn’t have to copy and the function call is very fast …. as long as you don’t write to it inside the function, no copy need ever happen. But the “bad design” bit is that R makes a decision whether to copy on write based only on a reference count and crucially that reference count stays increased even after a function returns, irrespective of whether or not the function has touched the variable.

So:

x <- matrix(0, nrow=1e5, ncol=100) # matrix has ref count 1
x[1,1] <- 1 # ref count is 1, so write with no copy
nrow(x) # ref count is 2 even though nothing was touched
x[1,1] <- 1 # ref count still 2, so R copies before writing first element. Now the ref count drops to 1 again
x[2,2] <- 1 # this writes without a copy as ref count got reset on last line
nrow(x) # ref count jumps
x[3,3] <- 1 # copy invoked again! Aaaargh!

So by calling nrow in the loop for the first example, the chain matrix is being copied in full on every iteration. In the second example, chain is never written to so there is no negative side effect to the ref count having gone up. In the third example, chain only ever has ref count 1 so there are no copies and each row is written in-place. I did a quick bit of profiling and indeed in the slow example, the R garbage collector allocates and tidies up nearly 9GB of RAM when executing the loop!

The crazy thing is that dim(chain)[1] works full speed even though that is all that nrow is doing under the hood, but the reason is that dim is a so-called “primitive” core R function which is special because it doesn’t affect the reference counter of its arguments. If you want to dig into this yourself, there’s a function refs() in the pryr package which tells you the current reference count to any variable.

Thanks Louis!

Please comment on the article here: R – Statisfaction

The post nrow, references and copies appeared first on All About Statistics.

Please comment on the article here: No Hesitations

The post More on the Problem with Bayesian Model Averaging appeared first on All About Statistics.

My reply: Thanks for sending. I can’t be sure about everything they’re doing but the paper looks reasonable to me. I expect there are various ways that the analysis could be improved, but on a quick look I don’t see anything obviously wrong with it, and the authors seem to know what they are doing. The findings seem important, and the results are mapped clearly enough that once the results are out there, others can comment if they see problems.

The only thing is that I think it would be better if the authors just posted all their graphs online, including but not limited to the graphs in that published paper. I really don’t like the whole embargo thing as it seems to encourage the massive publicity of results before they can be checked by the wider science and policy communities.

The post A reporter sent me a Jama paper and asked me what I thought . . . appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post A reporter sent me a Jama paper and asked me what I thought . . . appeared first on All About Statistics.

StanCon2018 is shaping up nicely as a unique opportunity to immerse oneself in all things Stan, meet Stan developers and fellow users. Registration is still open, but spots are filling up fast. We’re at 130 registrants and counting!

The draft schedule is now up. We have 16 accepted talks and 6 invited talks. Posters are still being accepted for our “Wear your poster” reception.

There are 6 tutorials:

• Intro to Stan (8hrs over 4 sessions)
• Advanced Hierarchical Models
• Gaussian Processes
• How to develop for Stan at the C++ level
• Bayesian Decision Making for Executives and Those who Communicate with them
• Have I Converged Successfully? How to verify fit and diagnose fit problems.

And there’s a 2 hour data/modeling session where you can get feedback on your data/models from Stan developers/modelers.

We have fun after dinner activities like R/C Airplane building/flying, non-snobby blind wine tasting and bonfire s’mores.

Go to http://mc-stan.org/events/stancon2018 and register! StanCon2018 is happening at the beautiful Asilomar conference facility at the beach in Monterey California for three days starting January 10, 2018.

StanCon2018 Organizing Committee

P.S. Thanks to our sponsors: Facebook, Metrum, Generable, and Jumping Rivers.

The post StanCon2018: one month to go, schedule finalized, over 20 talks, 6 tutorials… and flights are cheap appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post StanCon2018: one month to go, schedule finalized, over 20 talks, 6 tutorials… and flights are cheap appeared first on All About Statistics.

You really don’t want to miss Chi Feng‘s absolutely wonderful interactive demos.

(1) Markov chain Monte Carlo sampling

I believe this is exactly what Andrew was asking for a few Stan meetings ago:

• Chi Feng’s Interactive MCMC Sampling Visualizer

This tool lets you explore a range of sampling algorithms including random-walk Metropolis, Hamiltonian Monte Carlo, and NUTS operating over a range of two-dimensional distributions (standard normal, banana, donut, multimodal, and one squiggly one). You can control both the settings of the algorithms and the settings of the visualizations. As you run it, it even collects the draws into a sample which it summarizes as marginal histograms.

Source code

The demo is implemented in Javascript with the source code on Chi Feng’s GitHub organization:

• GitHub source repository for MCMC demo

Wish list

1. 3D (glasses or virtual reality headset)
2. multiple chains in parallel
3. scatterplot breadcrumbs
4. Gibbs sampler
5. Ensemble samplers
6. User-pluggable densities (Andrew really wants this for Stan where you’d choose two dimensions of N to visualize)

(2) Gaussian Process visualization

This one’s also really elegant.

• Chi Feng’s Gaussian process demo

It lets you lay down a few points and fit a 1D GP. It lets you choose kernel and hperparameters, and even sample regression functions from it conditioned on the data you provide.

Wish list

1. 3D (OK, I’m obssesed, but this one would be great on a 2D grid with a 3D visualization of a GP being fit)
2. Estimating hyperparameters (didn’t seem like it was doing this—may be a bit challenging for Javascript!)

Source code

• GitHub source repository for GP demo

The post Interactive visualizations of sampling and GP regression appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post Interactive visualizations of sampling and GP regression appeared first on All About Statistics.

I’ll be presenting at a webinar on Wednesday, December 13 at 1:00 PM Eastern. The title of the presentation is “Seven questions a statistician and answer for an attorney.”

I will discuss, among other things, when common sense applies and when correct analysis can be counter-intuitive. There will be ample time at the end of the presentation for Q & A.

If you’re interested in attending, you can

Please comment on the article here: Statistics – John D. Cook

The post How can a statistician help a lawyer? appeared first on All About Statistics.

A commmenter points to a chapter of E. T. Jaynes’s book on probability and inference that contains the following amazing bit:

The information we get from the TV evening news is not that a certain event actually happened in a certain way it is that some news reporter has claimed that it did. Even seeing the event on our screens can no longer convince us after recent revelations that all major US networks had faked some videotapes of alleged news events.

Scientists can reach agreement quickly because we trust our experimental colleagues to have high standards of intellectual honesty and sharp perception to detect possible sources of error. And this belief is justified but only about once in a decade is an experiment reported that turns out later to have been wrong. . . .

In politics we have a very different situation . . . We are convinced that virtually all news reporting is selective and distorted designed not to report the facts but to indoctrinate us in the reporter’s socio-political views. . . .

I have no idea what are those fake videotapes that Jaynes was talking about. But, beyond that, I don’t know what’s more disturbing: that Jaynes thinks that wrong experiments are reported only about once in a decade, or that he thinks “virtually all news reporting” is designed to indoctrinate.

I’m reminded of the Chestertonian position that extreme skepticism is a form of credulity: Once you take the position that virtually all news reporting is propaganda, then you can’t trust anything you read, so you make your way through the world based on whatever things you hear that sound good to you.

P.S. Don’t get me wrong here. Jaynes’s book is an entertaining and thought-provoking read, and I recommend it to all of you. It’s just funny to see someone like Jaynes, who had such strong opinions on technical matters, go off the deep end like this.

The post Ed Jaynes outta control! appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post Ed Jaynes outta control! appeared first on All About Statistics.