Please comment on the article here: Econometrics Beat: Dave Giles' Blog

The post June Reading List appeared first on All About Statistics.

From a few years ago, some quotes from the classic psychologist that fit within the worldview of the classic sabermetrician:

Faith means belief in something concerning which doubt is theoretically possible.

A chain is no stronger than its weakest link, and life is after all a chain.

A great many people think they are thinking when they are merely rearranging their prejudices.

A man has as many social selves as there are individuals who recognize him.

Acceptance of what has happened is the first step to overcoming the consequences of any misfortune.

An act has no ethical quality whatever unless it be chosen out of several all equally possible.

Be willing to have it so. Acceptance of what has happened is the first step to overcoming the consequences of any misfortune.

Belief creates the actual fact.

Compared to what we ought to be, we are half awake.

Do something everyday for no other reason than you would rather not do it, so that when the hour of dire need draws nigh, it may find you not unnerved and untrained to stand the test.

Great emergencies and crises show us how much greater our vital resources are than we had supposed.

Human beings can alter their lives by altering their attitudes of mind.

If any organism fails to fulfill its potentialities, it becomes sick.

If you want a quality, act as if you already had it.

In the dim background of mind we know what we ought to be doing but somehow we cannot start.

Individuality is founded in feeling; and the recesses of feeling, the darker, blinder strata of character, are the only places in the world in which we catch real fact in the making, and directly perceive how events happen, and how work is actually done.

It is only by risking our persons from one hour to another that we live at all. And often enough our faith beforehand in an uncertified result is the only thing that makes the result come true.

It is our attitude at the beginning of a difficult task which, more than anything else, will affect its successful outcome.

It is wrong always, everywhere, and for everyone, to believe anything upon insufficient evidence.

No matter how full a reservoir of maxims one may possess, and no matter how good one’s sentiments may be, if one has not taken advantage of every concrete opportunity to act, one’s character may remain entirely unaffected for the better.

Nothing is so fatiguing as the eternal hanging on of an uncompleted task.

Some are more Bill-James-like than others, but, as a whole, this list is kind of amazing.

The post A collection of quotes from William James that all could’ve come from . . . Bill James! appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post A collection of quotes from William James that all could’ve come from . . . Bill James! appeared first on All About Statistics.

This fall I’ll be again teaching the course, Communicating Data and Statistics.

Here’s the rough course plan. I’ll tinker with it between now and September but this is the basic idea. (The course listing is here, but that online description is out of date; the course plan linked above is more accurate.)

Here are the topics for the 13 weeks of the course:

1. Introducing yourself and telling a story
2. Principles of statistical graphics
3. Teaching
4. Making effective graphs
5. Communicating variation and uncertainty
6. Displaying fitted models
7. Giving a presentation
8. Dynamic graphics
9. Writing
10. Collaboration and the scientific community
11. Data processing and programming
12. Student projects
13. Student projects

Communication is central to your job as a quantitative researcher. Our goal in this course is for you to improve at all aspects of statistical communication, including writing, public speaking, teaching, informal conversation and collaboration, programming, and graphics.

Always keep in mind your goals and your audience.

Never forget, as one of our blog commenters reminds us: your closest collaborator is you six months ago . . . and she doesn’t reply to email!

The course is intended for Ph.D. students from all departments on campus; it is also open to some masters and undergraduate students who have particular interest in the topic.

This is my favorite course ever. As a student, you’ll get practice in all sorts of useful skills that are central to data and statistics, and you’ll participate in fast-moving conversations with fellow students with different backgrounds and experiences. In the interstices, you’ll learn all sorts of important ideas and methods in statistical design and analysis, that you’d never learn anywhere else. It’s the course where we first introduced statistics diaries. It’s the course where (a prototype version of) ShinyStan was one of the final projects!

You don’t want to miss this one.

The class will meet twice a week.

The post Hey—here are some tips on communicating data and statistics! appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post Hey—here are some tips on communicating data and statistics! appeared first on All About Statistics.

The trick is to consider graphs as comparisons.

Here’s the story. This post from several years ago shows a confusing and misleading pair of pie charts from a Kenyan election:

The quick reaction would be to say, ha ha, pie charts. But that’s not my point here. Sure, pie charts have problems and I think they’re almost never the right way to share data. But sometimes they’re not so bad.

To see the problem with the above display, we have to go back to first principles. And, with graphs, the first principle is always to consider what comparisons you would like to display, and what comparisons does the graph facilitate.

In this example, the main goal seems to be to compare the official results to the new exit poll. Thus, there are four comparisons to be made in parallel. A second goal is to compare the four categories within each dataset.

What about the pie charts? What comparisons do they easily allow?

The most salient point of any pie chart is that the percentages add up to 100%. So the first thing the graphs do is allow each of the proportions to be visually compared to the reference points of 100%, 50%, and 0%, and also 25% and 75%. We can see pretty clearly that in each dataset there are no candidates who received more than 50%, and there are two candidates who received between 25% and 50%.

But the pair of pie charts do not make it at all easy to compare each candidate from official results to the new poll. Part of that is perverse design choices, as the locations of the four “slices” have been permuted from one pie to another. But even if that had not been done—even if the four categories had been kept in pretty much the same position in each graph, and even if the coloring had been kept consistent—it would still be very difficult to compare angles of the different pies.

Basically the only way you can do it is to compare the numbers. And in that case a table would be clearer, as the relevant pairs could be written side by side. I’d prefer a dot plot.

Why, then, the pie? Part of this must be sheer convenience: whoever made these plots happened to know to make pies. But I think it’s more than that. I think the real appeal of the pie is that it graphically shows the percentages adding up to 100. And, sure, that’s something, but in this case it doesn’t really help with what I think is the key comparison of interest.

I had the same problem with Florence Nightingale’s clock plot. People think that graph is the coolest thing in the world—the wheel of time and all that—but it doesn’t facilitate the time-series comparisons that are the real goal in that example.

So, again, the point is not simply Don’t do pie charts (although I would endorse that message).

Rather, the point is: When making graphs, think about the comparisons you want readers to be able to visualize, and then evaluate possible displays based on their performance in making such comparisons clear.

The post You’ll never guess this one quick trick to diagnose problems with your graphs and then make improvements appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post You’ll never guess this one quick trick to diagnose problems with your graphs and then make improvements appeared first on All About Statistics.

Please comment on the article here: Gianluca Baio's blog

The post The code (and other stuff…) appeared first on All About Statistics.

Mike Betancourt pointed me to this news article by Alan Travis that is refreshingly positive regarding the use of sophisticated statistical methods in analyzing opinion polls.

Here’s Travis:

Leading pollsters have described YouGov’s “shock poll” predicting a hung parliament on 8 June as “brave” and the decision by the Times to splash it on its front page as “even braver”.

It is certainly rare for a polling company to produce a seats prediction. They usually leave that to psephologists and political scientists.

Good catch: YouGov’s chief scientist is Doug Rivers, who’s a political scientist. And the new method they’re using is multilevel regression and poststratification (MRP or Mister P), which was developed by . . . me! And I’m a political scientist too.

Travis continues:

But it is even more unusual for a company to suddenly employ a new polling model 10 days before a British general election.

Good point! That really would be an unusual practice. Fortunately, So, sure, MRP is not so new—our first paper on the method (“Poststratification into many categories using hierarchical logistic regression,” with Tom Little) was published 20 years ago.

Travis then supplies some details of YouGov’s forecast:

The Tories are in line to lose 20 seats, giving them 310, and Labour is set to gain 30, giving it 257 . . . But as Stephan Shakespeare, YouGov’s chief executive, notes in an accompanying analysis, that is only a central projection that “allows for a wide range of error” and he concedes: “However, these are just the midpoints and, as with any model, there is leeway either side. The Tories could end up with as many as 345 and Labour as few as 223, leading to an increased Conservative majority.” . . . The Times says the projection means the Tories “could get as many as 345 seats on a good night . . . and as few as 274 on a bad night”. That is a pretty wide range.

It’s good to see Travis explaining that realistic forecasts have wide ranges of uncertainty.

He continues with some details:

The methodology involved is described as “multi-level regression and post-stratification” analysis and is based on a substitute for traditional constituency polling, which it regards as “prohibitively expensive”. Shakespeare claimed YouGov tested it during last year’s EU referendum campaign and it produced leave leads every time. What a shame YouGov did not feel like sharing it with voters while their own published referendum polls showed a remain lead right up to polling day.

Travis missed a beat on this one! YouGov did share their MRP estimates during the EU referendum campaign. Ummm . . . let me google that . . . here it is: “YouGov uses Mister P for Brexit poll”: it’s an article by Ben Lauderdale and Doug Rivers which includes this graph showing Leave in the lead:

OK, it’s good we got that settled.

Travis concludes:

In an industry already suffering from an existential crisis of public confidence it is indeed a “brave” decision to come up with this one 10 days before a general election and promise to publish its results several times more before polling day.

“Brave” is perhaps overstating things, but, yes, I agree with Travis that presenting MRP estimates is the honorable way to go. Using the best available methods based on the best available knowledge, and giving appropriately wide uncertainty intervals: that’s how to do it.

It’s indeed a shame that Travis was not aware that YouGov had released a report with MRP in the lead-up to the Brexit vote. Also Travis, coming from the U.K., might not have realized that MRP was used in U.S. polls too. For example there was this Florida poll from September, 2016, which was analyzed by four different groups:

Charles Franklin, who estimated Clinton +3
Patrick Ruffini, who estimated Clinton +1
New York Times, who estimated Clinton +1
Sam Corbett-Davies, David Rothschild, and me, who estimated Trump +1. We used MRP.

That said, I don’t have any crystal ball. Shortly before the election I was promoting a forecast that gave Clinton a 92% chance of winning the election. That forecast didn’t use MRP; it aggregated information from state polls without appropriately accounting for expected levels of systematic error. I should’ve used MRP myself—I would’ve, but I wasn’t working with the raw data. In this upcoming U.K. election, YouGov does have the raw polling data so of course they are using MRP; it would at this point be a strange choice not to.

Anyway, it’s cool that Alan Travis conveyed some subtleties of polling to the British newspaper audience, even if he did miss one or two things.

Full disclosure: YouGov gives some financial support to the Stan project.

P.S. Thanks to Steven Johnson for the photo at the top of this post.

P.P.S. More on yougov’s use of MRP here.

The post U.K. news article congratulates YouGov on using modern methods in polling inference appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post U.K. news article congratulates YouGov on using modern methods in polling inference appeared first on All About Statistics.

Please comment on the article here: Junk Charts

The post Announcing a new venture appeared first on All About Statistics.

Saw this the other day:

In defense of wrapr::let() (originally part of replyr, and still re-exported by that package) I would say:

• let() was deliberately designed for a single real-world use case: working with data when you don’t know the column names when you are writing the code (i.e., the column names will come later in a variable). We can re-phrase that as: there is deliberately less to learn as let() is adapted to a need (instead of one having to adapt to let()).
• The R community already has months of experience confirming let() working reliably in production while interacting with a number of different packages.
• let() will continue to be a very specific, consistent, reliable, and relevant tool even after dpyr 0.6.* is released, and the community gains experience with rlang/tidyeval in production.

If rlang/tidyeval is your thing, by all means please use and teach it. But please continue to consider also using wrapr::let(). If you are trying to get something done quickly, or trying to share work with others: a “deeper theory” may not be the best choice.

An example follows.

In “base R” one can write:

d <- data.frame(x = 1:3)

If we know the column name we wish to add to this data frame we write:

d$newColumn <- 1

The above is “non-parameterized” evaluation in that the variable name is taken from the source code, and not from a variable carrying the information. This is great for ad-hoc analysis, but it would be hard to write functions, scripts and packages if this was the only way we had to manipulate columns.

This isn’t a problem as R supplies an additional parameterized notation as we show here. When we don’t know the name of the column (but expect it to be in a variable at run time) we write:

# code written very far away from us
variableHoldingColumnName <- 'newColumn' 

# our code
d[[variableHoldingColumnName]] <- 1

The purpose of wrapr::let() is to allow one to use the non-parameterized form as if it were parameterized. Obviously this is only useful if there is not a convenient parameterized alternative (which is the case for some packages). But for teaching purposes: how would wrapr::let() let us use the “$” as if it were parameterized (which we would have to do if [[]] and [] did not exist)?

With wrapr::let() we can re-write the “dollar-sign form” as:

wrapr::let(
   c(NEWCOL = variableHoldingColumnName),
   {
     d$NEWCOL <- 1
   }
)

The name “NEWCOL” is a stand-in name that we write all our code in terms of. The expression “c(NEWCOL = variableHoldingColumnName)” is saying: “NEWCOL is to be interpreted as being whatever name is stored in the variable variableHoldingColumnName.” Notice we can’t tell from the code what value is in variableHoldingColumnName, as that won’t be known until execution time. The alias list or vector can be arbitrarily long and built wherever you like (it is just data). The expression block can be arbitrarily large and complex (so you need only pay the mapping notation cost once per function, not once per line of code).

And that is wrapr::let().

If you don’t need parametric names you don’t need wrapr::let(). If you do need parametric names wrapr::let() is one of the most reliable and easiest to learn ways to get them.

A nice article from a user recording their experience trying different ways to parameterize their code (including wrapr::let()) can be found here.

If you wish to learn more we have a lot of resources available:

• A vignette: vignette('let', package='wrapr').
• The method help: help('let', package='wrapr')
• Examples: Using replyr::let to Parameterize dplyr Expressions.
• A video lecture: My recent BARUG talk: Parametric Programming in R with replyr.
• Some notes: Parametric Programming in R.
• The package introduction: The wrapr introduction.
• Comparison to the soon to deprecated SE underbar/underscore methods: Comparative examples using replyr::let.

And many more examples on our blog.

Please comment on the article here: Statistics – Win-Vector Blog

The post In defense of wrapr::let() appeared first on All About Statistics.

Uh oh, I’m starting to feel like that pizzagate guy . . .

Here’s the background. When I talk about my serious published errors, I talk about my false theorem, I talk about my empirical analysis that was invalidated by miscoded data, I talk my election maps whose flaws were pointed out by an angry blogger, and I talk about my near-miss regarding Portugal.

OK, fine. But then I was going through old blog posts and I’d found a published error of mine that I’d completely forgotten about! It was a statement in one of my most influential papers—just a small part of the paper, it didn’t change our main results, but we really were wrong, and arguably our mistake misled people. I’m glad that later researchers were suspicious of our statement, checked it, and pointed out how we’d been wrong.

The post Another serious error in my published work! appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post Another serious error in my published work! appeared first on All About Statistics.

Last week I was asked a simple question: "How do I choose a seed for the random number functions in SAS?" The answer might surprise you: use any seed you like. Each seed of a well-designed random number generator is likely to give rise to a stream of random numbers, so you can view the various streams as statistically equivalent.

Random means random

To be clear, I am talking about using a seed value to initialize a modern, high-quality, pseudorandom number generator (RNG). For example, in SAS you can use the STREAMINIT subroutine to initialize the Mersenne twister algorithm that is used by the RAND function. If you are still using the old-style RANUNI or RANNOR functions in SAS, please read the article "Six reasons you should stop using the RANUNI function to generate random numbers."

A seed value specifies a particular stream from a set of possible random number streams. When you specify a seed, SAS generates the same set of pseudorandom numbers every time you run the program. However, there is no intrinsic reason to prefer one stream over another. The stream for seed=12345 is just as random as the stream for the nine-digit prime number 937162211.

Some people see the number 937162211 and think that it looks "more random" than 12345. They then assume that the random number stream that follows from CALL STREAMINIT(937162211) is "more random" than the random number stream for CALL STREAMINIT(12345). Nope, random means random. In modern pseudorandom generators, the streams for different seeds should have similar statistical properties. Furthermore, many RNGs use the base-2 representation of the seed for initialization and (12345)₁₀ = (11000000111001)₂ looks pretty random! In fact, if you avoid powers of 2, the base-2 representations of most base-10 numbers "look random."

Initialization: Hard for researchers, easy for users

Researchers who specialize in random number generators might criticize what I've said as overly simplistic. There have been many research papers written about how to take a 32-bit integer and use that information to initialize a RNG whose internal state contains more than 32 bits. There have been cases where a RNG was published and the authors later modified the initialization routine because certain seeds did not result in streams that were sufficiently random. There have been many discussions about how to create a seed initialization algorithm that is easy to call and that almost always results in a high-quality stream of random numbers.

These are hard problems, but fortunately researchers have developed ways to initialize a stream from a seed so that there is a high probability that the stream will have excellent statistical properties. The relevant question for many SAS programmers is "can I use 12345 or my telephone number as seed values, or do I always need to type a crazy-looking nine-digit sequence?" My response is that there is no reason to prefer the crazy-looking seed over an easy-to-type sequence such as your phone number, your birthday, or the first few digits of pi.

Choosing a random seed

If you absolutely insist on using a "random seed," SAS can help. If you call the STREAMINIT subroutine with the value 0, then SAS will use the date, time of day, and possibly other information to manufacture a seed when you call the RAND function. SAS puts the seed value into the SYSRANDOM system macro variable. That means you can use %PUT to display the seed that SAS created, as follows:

data _null_;
call streaminit(0);   /* generate seed from system clock */
x = rand("uniform");
run;
%put &=SYSRANDOM;

SYSRANDOM=1971603567

Every time you run this program, you will get a different seed value that you can use as the seed for a next program.

A second method is to use the RAND function to generate a random integer between 1 and 2³¹-1, which is the range of valid seed values for the Mersenne twister generator in SAS 9.4m4. The following program generates a random seed value:

data _null_;
call streaminit(0);
seed = ceil( (2**31 - 1)*rand("uniform") ); 
put seed=;
run;

seed=1734176512

Both of these methods will generate a seed for you. However, the randomly generated seed does not provide any benefit. For a modern, high-quality, pseudorandom number generator, the stream should have good statistical properties regardless of the seed value. Using a random seed value does not make a stream "more random" than a seed that is easier to type.

The post How to choose a seed for generating random numbers in SAS appeared first on The DO Loop.

Please comment on the article here: The DO Loop

The post How to choose a seed for generating random numbers in SAS appeared first on All About Statistics.