## Creating a basic heat map in SAS

August 18, 2014
By

Heat maps have many uses. In a previous article, I showed how to use heat maps with a discrete color ramp to visualize matrices that have a small number of unique values, such as certain covariance matrices and sparse matrices. You can also use heat maps with a continuous color […]

## GEFCom 2014 energy forecasting competition is underway

August 18, 2014
By

GEFCom 2014 is the most advanced energy forecasting competition ever organized, both in terms of the data involved, and in terms of the way the forecasts will be evaluated. So everyone interested in energy forecasting should head over to the competition webpage and start forecasting: www.gefcom.org. This time, the competition is hosted on CrowdANALYTIX rather than Kaggle. […]

## Are P Values Error Probabilities? or, “It’s the methods, stupid!” (2nd install)

August 18, 2014
By

Despite the fact that Fisherians and Neyman-Pearsonians alike regard observed significance levels, or P values, as error probabilities, we occasionally hear allegations (typically from those who are neither Fisherian nor N-P theorists) that P values are actually not error probabilities. The denials tend to go hand in hand with allegations that P values exaggerate evidence against […]

## Teaching random variables and distributions

August 18, 2014
By

Why do we teach about random variables, and why is it so difficult to understand? Probability and statistics go together pretty well and basic probability is included in most introductory statistics courses. Often maths teachers prefer the probability section as … Continue reading →

August 17, 2014
By

Navigation gets you from where you are to where you want to be. Speaking of navigation, you can jump to selected sections of this post: Navigation; R-bloggers; Task views; Rdocumentation.org; sos package; ??; apropos; ls; methods; getAnywhere; :::; find; args; grep; %in%; str; getwd; file.choose; Spyglass summary; browser; See also. Overview Figure 1: A map […] The post 21 R navigation tools appeared first on Burns Statistics.

## Quicksort speed, just in time compiling and vectorizing

August 17, 2014
By

I was reading the Julia documentation the other day. They do speed comparisons to other languages. Obviously R does not come out very well. The R code for quicksort is here and I noticed it was not vectorized at all. So I wondered if it could be improv...

## My courses this fall at Columbia

August 16, 2014
By

Stat 6103, Bayesian Data Analysis, TuTh 1-2:30 in room 428 Pupin Hall: We’ll be going through the book, section by section. Follow the link to see slides and lecture notes from when I taught this course a couple years ago. This course has a serious workload: each week we have three homework problems, one theoretical, […] The post My courses this fall at Columbia appeared first on Statistical Modeling, Causal…

## Crowdsourcing resources for the Johns Hopkins Data Science Specialization

August 15, 2014
By

Since we began offering the Johns Hopkins Data Science Specialization we've noticed the unbelievable passion that our students have about our courses and the generosity they show toward each other on the course forums. Many students have created quality content around the subjects … Continue reading →

## How to use MCMC posterior as prior for future data

August 15, 2014
By

An emailer writes:Dear Prof. Kruschke,Hello. My name is ... and I am ... . I'm trying to apply Bayesian theorem in developing a model of ... . I used your code to estimate posterior distribution without any trouble. Here is my question. Would you ki...

## “Psychohistory” and the hype paradox

August 15, 2014
By

Lee Wilkinson writes: I thought you might be interested in this post. I was asked about this by someone at Skytree and replied with this link to Tyler Vigen’s Spurious Correlations. What’s most interesting about Vigen’s site is not his video (he doesn’t go into the dangers of correlating time series, for example), but his […] The post “Psychohistory” and the hype paradox appeared first on Statistical Modeling, Causal Inference,…

## Book reviews

August 15, 2014
By

I have neglected to acknowledge the good folks who have posted reviews of my books in the last several months. Thank you, readers! *** Tom Peters, the business guru and author of the business classic In Search of Excellence, tweeted about Numbersense: "On my 13-hour Boston-Dubai flight, I re-read cover-2-cover Kaiser Fung's superb-useful-fun book Number Sense". Numbers Rule Your World is #7 on Gregory Ciotti's list of 25 underrated books…

August 15, 2014
By

The New York Times Upshot team came up with a dataviz that is worth your time. This is a set of maps that gives a perspective on migration patterns within the US. The metric being portrayed is the birthplace of...

## Early Computing

August 15, 2014
By

During a recent visit to Harvard U. I came across this monstrosity thing of beauty in the Science Center - the building where the Dept. of Statistics is housed:It's an "Aiken-IBM Automatic Sequence Controlled Calculator - I", and dates from 1944. ...

## Mathematical Statistics Lesson of the Day – Markov’s Inequality

$Mathematical Statistics Lesson of the Day – Markov’s Inequality$

Markov’s inequality is an elegant and very useful inequality that relates the probability of an event concerning a non-negative random variable, , with the expected value of .  It states that where . I find Markov’s inequality to be beautiful for 2 reasons: It applies to both continuous and discrete random variables. It applies to any non-negative […]

## STEM forums

August 15, 2014
By

“I can calculate the movement of stars, but not the madness of men.” Isaac Newton When visiting the exhibition hall at JSM 2014, I spoke with people from STEM forums on the Springer booth. The concept of STEM (why STEM? Nothing to do with STAN! Nor directly with Biology. It stands as the accronym for Science, Technology, […]

## (Some) Spaces available

August 14, 2014
By

Requests for registration to our short course on Bayesian methods in Health Economics are coming in steadily \$-\$ in fact, we had started advertising quite in advance (the course is in November), but we're nearly booked up.We set a total of 30 part...

## Luck vs. skill in poker

August 14, 2014
By

The thread of our recent discussion of quantifying luck vs. skill in sports turned to poker, motivating the present post. 1. Can good poker players really “read” my cards and figure out what’s in my hand? For a couple years in grad school a group of us had a regular Thursday-night poker game, nickel-dime-quarter with […] The post Luck vs. skill in poker appeared first on Statistical Modeling, Causal Inference,…

## Welcome

August 14, 2014
By

Junk Charts is made by Kaiser Fung, the Web’s first data visualization critic. I discuss what makes graphics work, and how to make them better. Think chartjunk + junk art. Here is my first post from nine years ago. These...

## Intuition and Data at KeenCON

August 14, 2014
By

-+*I will be giving a talk “Bayesian statistics as a way to integrate intuition and data” at KeenCon, September 11, 2014 in San Francisco. Update: Use promo code KeenCon-JohnCook to get 75% off registration.

## new laptop with ubuntu 14.04

August 13, 2014
By

As I was getting worried about the chances of survival of my current laptop (bought in emergency upon my return from Kyoto!), I decided to use some available grant money to buy a new laptop without stepping through the emergency square. Thanks to my local computer engineer, Thomas, I found a local dealer selling light […]

## swirl and the little data scientist’s predicament

August 13, 2014
By

Editor's note: This is a repost of "R and the little data scientist's predicament". A brief idea for an update is presented at the end in italics.  I just read this fascinating post on _why, apparently a bit of a cult hero … Continue reading →

## oneSecond: Printing Every Tweet Created During a Single Second

August 13, 2014
By

#oneSecond [philippadrian.com] by graphic design student Philipp Adrian aggregates all the tweets sent at exactly 14:47:36 GMT of 9 November 2012. The 5522 Twitter messages are categorized and ordered in 4 different books. Every user is part of each...

## Davenport on targeted marketing

August 13, 2014
By

Tom Davenport is one of the leading voices on business analytics, and he has a new piece titled "Why are most 'targeted' marketing offers so bad?" in which he expanded on a question I raised in my HBR article. Tom's book Competing on Analytics is a classic. He has a great appreciation for the business of the data business. In the new feature, Davenport classifies marketing offers he gets into…