Stephen Kosslyn’s principles of graphics and one more: There’s no need to cram everything into a single plot

December 6, 2012

(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)

Jerzy Wieczorek has an interesting review of the book Graph Design for the Eye and Mind by psychology researcher Stephen Kosslyn. I recommend you read all of Wieczorek’s review (and maybe Kosslyn’s book, but that I haven’t seen), but here I’ll just focus on one point. Here’s Wieczorek summarizing Kosslyn:

p. 18-19: the horizontal axis should be for the variable with the “most important part of the data.” See Kosslyn’s Figure 1.6 and 1.7 below. Figure 1.6 clearly shows that one of the sex-by-income groups reacts to age differently than the other three groups do. Figure 1.7 uses sex as the x-axis variable, making it much harder to see this same effect in the data.
As a statistician exploring the data, I might make several plots using different groupings… but for communicating my results to an audience, I would choose the one plot that shows the findings most clearly.

Those who know me well (or who have read the title of this post) will guess my reaction, which is that Kosslyn is trying to cram too much into a single graph. The circles and squares are hard to tell apart, the open and dark symbols are a bit confusing, and the lines are so thick that it’s hard to make out the symbols anyway. In addition, the y-axis seems a bit over-labled, with hash marks at every 100. As Kossyln himself notes, the purpose of a graph is to make comparisons, not to be used as a look-up table.

The structure of the (hypothetical) data being displayed is pretty simple: it’s a single continuous outcome as a function of three binary inputs. Displaying this as four lines is just too much. In general I prefer to put continuous predictors on the x-axis and discrete predictors as separate lines, so that would rule out Kosslyn’s second graph (Figure 1.7 above). But Figure 1.6 is too busy. Let’s try it as two graphs:

That looks better. (I’ve also removed the bit about $24,999, which to me just gives a bunch more meaningless numbers for someone to stare at, distracting them from the patterns in the data.) I kept the two graphs on a common scale so we can make comparisons between them if we’d like.

Or we could do it the other way:

That works too. Maybe it’s not quite as good as my other graph, from Kosslyn’s perspective of emphasizing that one discordant line. On the other hand, in real data, the slopes of different lines are estimated with error, and it’s not clear how much emphasis we really should be placing on a line that happens to have a different slope than some other lines whose slopes may not be statistically significantly different from zero in any case.

Here’s another point not mentioned by Kosslyn (or, to be precise, not mentioned in Wieczorek’s review): What’s with those binary age and income variables? That looks like something we’d see in an old-style statistics textbook. I’d prefer more granularity in the continuous variables. Why just “under 65″ and “65 and over”? Similarly, why only two income categories. I’d like to have at least three categories, maybe more, depending on how many data points are behind these numbers. In the above graph, it would be trivial to increase the number of age categories on the x-axis, and we could also increase the number of income categories by simply placing more graphs side by side. We could even introduce another background variable (for example, ethnicity) by stacking these graphs in rows (as we did for our maps of voting by income, ethnicity, and state).

The above is not meant to disparage Kosslyn’s work, nor am I suggesting that my redrawn graphs are perfect. Here I’m focusing on one single point, which is the virtue of small multiples (as Tufte puts it). I agree that psychology research should be central in helping us figure out how to convey information, to ourselves as well as to others (I don’t believe in the distinction between exploratory and presentation graphics). No amount of introspection, speculation, and theorizing by Tufte, me, or anybody else can substitute for hard research in perception. I just think that all of use get stuck in our ways of thinking, and I fear that Kosslyn has been stuck in the traditional idea that all the information should be conveyed in a single plot.

Hence I also object to Wieczorek’s statement, “for communicating my results to an audience, I would choose the one plot that shows the findings most clearly.” Sometimes one plot will do, but other times you can make a single display with several plots to better make your point. “A single display with several plots . . .”: that sounds complicated. But as the example above illustrates, the small-multiples display can be cleaner than the one graph.

Oddly enough, I think Kosslyn recognizes this point in some contexts, because in his book on powerpoint, he writes, in reference to the notorious tour de force graph of Napoleon’s troops in Russia:

I [Kosslyn] agree that M. Minard was amazingly clever and managed to cram a huge amount of information into a single display, but I can’t agree that this is an effective way to communicate; the display doesn’t present the facts so that they’re clear or easily absorbed. If you are in the mood, you may enjoy taking the time to study the display for the fun of solving a puzzle, pondering intricate details, or appreciating the graphic devices employed. But if you want the facts and want them in a clear, easily understood way, this display is not the solution.

I just think Kosslyn needs to take the next step and recognize that, in his own field, you can get a cleaner picture with small multiples than by trying to fit all the information on a single plot. As I tell my students: One slide, several plots. One page, several plots. Take advantage of our visual system’s ability and inclination to look around compare.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science