Dear Reader: Over the next week, in addition to a regularly scheduled post by Professor Stephen Senn, we will be taking up two papers[i] from the contributions to the special topic: “Statistical Science and Philosophy of Science: Where Do (Should) They Meet in 2011 and Beyond?” in Rationality, Markets and Morals: Studies at the Intersection [...]
I cannot believe that I lived without emacs! Now I use emacs more than any application. The usage of emacs is going to increase as the days go. A non-emacs user may think that why an editor should get a … Continue reading →The post Why Emacs is important to me? : ESS and org-mode appeared first on Fiddling with data and code.
The Tyranny of Tuning Parameters We all know about the curse of dimensionality. The curse creates serious practical and theoretical difficulties in many aspects of statistics. But there is another pernicious problem that gets less attention. I call it: The Tyranny of Tuning Parameters. Many (perhaps most) data analysis methods involve one or more tuning [...]
I received the following note from someone who’d like to remain anonymous: I read your post on ethics and statistics, and the comments therein, with much interest. I did notice, however, that most of the dialogue was about ethical behavior of scientists. Herein I’d like to suggest a different take, one that focuses on the [...]
David Hogg on linear regression: … in almost all cases in which scientists fit a straight line to their data, they are doing something that is simultaneously wrong and unnecessary. It is wrong because … linear relationship is exceedingly rare.…Read more ›
I’ve recently taken delivery of a Raspberry Pi mini computer. For anyone who doesn’t know, this is a low cost, low power machine, costing around 20 GBP (25 USD) and consuming around 2.5 Watts of power (it is powered by micro-USB). This amazing little device can run linux very adequately, and so naturally I’ve been […]
This post shows how to create a timeline graph by using ggplot2. Let’s start by loading the ggplot2 library. Next let’s create a dataset which we will use to feed the graph. In the last column (y), I create random positive values for the first three rows (which will be Read more »The post Timeline graph with ggplot2 appeared first on FishyOperations.
Last week, scientists presented evidence for a "Higgs-like" particle, and this got science reporters really excited. Unfortunately, they got really excited about understanding the particle, but not the interpretation of the evidence for the particle.&#...
Reader Sushil B. offers this chart from Business Week on hedge fund returns. (link) Unmoored bubbles, slanted text, positive and negative returns undifferentiated, bubble within bubble, paired data scattered apart, and it's not even that attractive. Here is a Bumps-chart...
Thomas Basbøll writes: [Advertising executive] Russell Davies wrote a blog post called “The Tyranny of the Big Idea”. His five-point procedure begins: Start doing stuff. Start executing things which seem right. Do it quickly and do it often. Don’t cling onto anything, good or bad. Don’t repeat much. Take what was good and do it [...]
I saw this plot in the supplement of a recent paper comparing microarray results to RNA-seq results. Nothing earth-shattering in the paper - you've probably seen a similar comparison many times before - but I liked how they solved the overplotting...
I thank Dr. Vladimir Cherkassky for taking up my general invitation to comment. I don’t have much to add to my original post[i], except to make two corrections at the end of this post. I invite readers’ comments. As I could not participate in the discussion session on Sunday, I would like to address several [...]
As an illustration of how straight forward it is in JAGS/BUGS to fit non-linear trends to data, I estimated the parameters of a sinusoid-plus-linear trend when fit to average daily temperatures. The temperatures are for Madison, Wisconsin, in honor of ...
Ring Lardner, Jr.: [In 1936] I was already settled in Southern California, and it may have been that first exercise of the franchise that triggered the FBI surveillance of me that would last for decades. I had assumed, of course, that I was enjoying the vaunted American privilege of the secret ballot. On a wall [...]
Over the last few days, I've written a couple of posts (here, here) about creating coefficient plots. I like them way better than tables, but I don't really see a need for them.…
I guess we should not call Higgs Boson the “God Particle” any longer. Plenty of news have reported on this findings on July 4th, 2012. If you haven’t watched the original webcast announcement posted at CERN, the European Organization for Nuclear Research, you got to spend some time on it. A shorter version of the video summarizes the [...]
The Problem In the United States, health care costs have been going up for a number of years, even when adjusted for inflation. Not unlike a runaway freight train, this rampant inflation cannot continue indefinitely without crashing. ...
A timeline is a perfect tool for visualising complex information … and is sometimes quite complex itself. This example from evolutionoftheweb gives both. And it shows interesting details like the evolution of Flash and SVG over time. Flash SVG...
In an article catchily entitled, “I got more data, my model is more refined, but my estimator is getting worse! Am I just dumb?”, Meng and Xie write: Possibly, but more likely you are merely a victim of conventional wisdom. More data or better models by no means guarantee better estimators (e.g., with a smaller [...]
David Hogg on responsible data analysis: The key idea is that the result of responsible data analysis is not an answer but a distribution over answers. Data are inherently noisy and incomplete; they never answer your question precisely. So no…Read more ›
My serious stats book is officially published (in the UK at least). The US release date is next month (August 7th). I'm not sure why the release is later (possibly extra shipping time for the books). The earlier European release date is I suppose compe...
Weave (BETA 1.0) is a new web-based visualization platform designed to enable visualization of any available data by anyone for any purpose. Weave is an application development platform supporting multiple levels of user proficiency – novice to advan...
