Criticism in visualization can be harsh, pedantic, and stupid. But it is also a useful tool that shows the thinking behind the seemingly simple graphical shapes we use, and teaches people things they might not be aware of. While I largely agree with Andy Kirk’s criticism of visualization criticism and the danger of scaring people away from visualization, his “grown-up criticism” argument cuts both ways: grown-ups can argue a point without…
Criticism in visualization can be harsh, pedantic, and stupid. But it is also a useful tool that shows the thinking behind the seemingly simple graphical shapes we use, and teaches people things they might not be aware of. While I largely agree with Andy Kirk’s criticism of visualization criticism and the danger of scaring people away from visualization, his “grown-up criticism” argument cuts both ways: grown-ups can argue a point without…
Some time ago I promised to blog about how to teach statistics with chocolate. Anyone who has watched my youtube videos may have noticed a recurring theme. Helen sells Choconutties. These are a fictitious chocolate bar, originally devised to require … Continue reading →
While traditional statistics courses teach students to calculate intervals and test for binomial proportions using a normal or t approximation, this method does not always work well. Agresti and Coull ("Approximate is better than "exact' for interval ...
Summary so far In Part One I described trends in market share of major religions in the U.S.: since 1988, the fraction of Protestants dropped from 60% to 51%, and the fraction of people with no religious affiliation increased from 8% t...
Stephen Senn Head of the Methodology and Statistics Group, Competence Center for Methodology and Statistics (CCMS), Luxembourg An issue sometimes raised about randomized clinical trials is the problem of indefinitely many confounders. This, for example is what John Worrall has to say: Even if there is only a small probability that an individual factor is [...]
Mark Patterson writes: I found a discussion at the Boston Review that I thought you’d be interested in, given your posts on the potentially dubious foundations of many neoclassical economics models. Michael Sandel cites a few examples of markets crowding out moral behavior. His longest discussion regards Frey and Oberholzer-Gee’s work demonstrating Swiss citizens’ willingness [...]
A Northwest Pipeline to Silicon Valley: The University of Washington’s reputation as an incubator of tech talent is growing — and so have the job offers from the famed area further down the West Coast.
A number of people seem to have objected to my post quoting Carl Sagan about skepticism (hi Paramita!) and I appreciate the comments. However, I wanted to clarify why I liked the quotation. I think in order to be successful in science three things are...
Come to your Census [spinifexgroup.com] was developed by Spinifex for the occasion of Vivid Sydney, a spectacular festival around the theme of light which happened about one month ago. The installation consisted of 3 different parts that visualized ...
Suppose that you have two data vectors, x and y, with the same number of elements. How can you rearrange the values of y so that they have the same relative order as the values of x? In other words, find a permutation, π, of the elements of y so [...]
Whatever happened to those evaluations that your airline asked you to complete after taking a flight? They ask you for a number of ratings about buying your ticket, attributes of the plane, the service you received, and if you were satisfied, if you wo...
Carl Bialik, i.e. the Numbers Guy at WSJ, wrote a nice piece (link) trying to explain something that is very difficult to explain to a general audience... the notion of statistical significance. He discusses this in relation to the experiments that have supposedly proved the existence of the Higgs boson. I won't repeat his entire piece here. I have these thoughts while reading the piece: The physicists talk about LEE…
I just finished reading "Dark Pools" by Scott Patterson. The book documents the foundation of the major electronic exchanges (from Island through BATS) and high-frequency trading firms, along with the related regulations. Ironically, the book says virtually nothing about actual dark pools (such as Goldman's Sigma X). Patterson makes this clear in the first note: [...]
Dear Reader: Over the next week, in addition to a regularly scheduled post by Professor Stephen Senn, we will be taking up two papers[i] from the contributions to the special topic: “Statistical Science and Philosophy of Science: Where Do (Should) They Meet in 2011 and Beyond?” in Rationality, Markets and Morals: Studies at the Intersection [...]
I cannot believe that I lived without emacs! Now I use emacs more than any application. The usage of emacs is going to increase as the days go. A non-emacs user may think that why an editor should get a … Continue reading →The post Why Emacs is important to me? : ESS and org-mode appeared first on Fiddling with data and code.
The Tyranny of Tuning Parameters We all know about the curse of dimensionality. The curse creates serious practical and theoretical difficulties in many aspects of statistics. But there is another pernicious problem that gets less attention. I call it: The Tyranny of Tuning Parameters. Many (perhaps most) data analysis methods involve one or more tuning [...]
I received the following note from someone who’d like to remain anonymous: I read your post on ethics and statistics, and the comments therein, with much interest. I did notice, however, that most of the dialogue was about ethical behavior of scientists. Herein I’d like to suggest a different take, one that focuses on the [...]
David Hogg on linear regression: … in almost all cases in which scientists fit a straight line to their data, they are doing something that is simultaneously wrong and unnecessary. It is wrong because … linear relationship is exceedingly rare.…Read more ›
I’ve recently taken delivery of a Raspberry Pi mini computer. For anyone who doesn’t know, this is a low cost, low power machine, costing around 20 GBP (25 USD) and consuming around 2.5 Watts of power (it is powered by micro-USB). This amazing little device can run linux very adequately, and so naturally I’ve been […]
This post shows how to create a timeline graph by using ggplot2. Let’s start by loading the ggplot2 library. Next let’s create a dataset which we will use to feed the graph. In the last column (y), I create random positive values for the first three rows (which will be Read more »The post Timeline graph with ggplot2 appeared first on FishyOperations.
Last week, scientists presented evidence for a "Higgs-like" particle, and this got science reporters really excited. Unfortunately, they got really excited about understanding the particle, but not the interpretation of the evidence for the particle.&#...
Reader Sushil B. offers this chart from Business Week on hedge fund returns. (link) Unmoored bubbles, slanted text, positive and negative returns undifferentiated, bubble within bubble, paired data scattered apart, and it's not even that attractive. Here is a Bumps-chart...
