The 80/20 rule of statistical methods development

March 20, 2014
By
The 80/20 rule of statistical methods development

Developing statistical methods is hard and often frustrating work. One of the under appreciated rules in statistical methods development is what I call the 80/20 rule (maybe could even by the 90/10 rule). The basic idea is that the first … Continue reading →

Read more »

Seasonal, or periodic, time series

March 20, 2014
By

Monday, in our MAT8181 class, we’ve discussed seasonal unit roots from a practical perspective (the theory will be briefly mentioned in a few weeks, once we’ve seen multivariate models). Consider some time series , for instance traffic on ...

Read more »

The candy weighing demonstration, or, the unwisdom of crowds

March 20, 2014
By

From 2008: The candy weighing demonstration, or, the unwisdom of crowds My favorite statistics demonstration is the one with the bag of candies. I’ve elaborated upon it since including it in the Teaching Statistics book and I thought these tips might be useful to some of you. Preparation Buy 100 candies of different sizes and […]The post The candy weighing demonstration, or, the unwisdom of crowds appeared first on Statistical…

Read more »

Machine Learning Lesson of the Day – Overfitting and Underfitting

Machine Learning Lesson of the Day – Overfitting and Underfitting

Overfitting occurs when a statistical model or machine learning algorithm captures the noise of the data.  Intuitively, overfitting occurs when the model or the algorithm fits the data too well.  Specifically, overfitting occurs if the model or algorithm shows low bias but high variance.  Overfitting is often a result of an excessively complicated model, and […]

Read more »

Mathematical and Applied Statistics Lesson of the Day – The Central Limit Theorem Applies to the Sample Mean

Mathematical and Applied Statistics Lesson of the Day – The Central Limit Theorem Applies to the Sample Mean

Having taught and tutored introductory statistics numerous times, I often hear students misinterpret the Central Limit Theorem by saying that, as the sample size gets bigger, the distribution of the data approaches a normal distribution.  This is not true.  If your data come from a non-normal distribution, their distribution stays the same regardless of the […]

Read more »

MCMC for Econometrics Students – III

March 20, 2014
By
MCMC for Econometrics Students – III

As its title suggests, this post is the third in a sequence of posts designed to introduce econometrics students to the use of Markov Chain Monte Carlo (MCMC, or MC2) methods for Bayesian inference. The first two posts can be found here and here, and&n...

Read more »

fine-sliced Poisson [a.k.a. sashimi]

March 19, 2014
By
fine-sliced Poisson [a.k.a. sashimi]

As my student Kévin Guimard had not mailed me his own Poisson slice sampler of a Poisson distribution, I could not tell why the code was not working! My earlier post prompted him to do so and a somewhat optimised version is given below: As you can easily check by running the code, it does […]

Read more »

LEGO Calendar: a Tangible Wall-Mounted Planner that Can be Digitized

March 19, 2014
By
LEGO Calendar: a Tangible Wall-Mounted Planner that Can be Digitized

The LEGO Calendar [vitaminsdesign.com], developed by design and invention studio Vitamins, is a wall-mounted time planner that simply can be photographed to create an online, digital counterpart. The calendar is big, visible, tactile and flexible, as...

Read more »

The time traveler’s challenge.

March 19, 2014
By

Editor's note: This has nothing to do with statistics.  I do a lot of statistics for a living and would claim to know a relatively large amount about it. I also know a little bit about a bunch of other scientific … Continue reading →

Read more »

How Americans vote

March 19, 2014
By
How Americans vote

An interview with me from 2012: You’re a statistician and wrote a book, Red State, Blue State, Rich State, Poor State, looking at why Americans vote the way they do. In an election year I think it would be a good time to revisit that question, not just for people in the US, but anyone around […]The post How Americans vote appeared first on Statistical Modeling, Causal Inference, and Social Science.

Read more »

The closer you look, the more confused

March 19, 2014
By
The closer you look, the more confused

A twitter follower submitted this chart showing the shift in ethnicity in Texas: If you blinked, you probably took away the wrong message. Our "prior" tells us that the proportion of Hispanics has been rising quite rapidly in Texas. So,...

Read more »

What is numbersense?

March 19, 2014
By
What is numbersense?

It's Spring Break at NYU, which for professors, is not a break. I have been marking midterms for my business analytics class. Since I like to set open-ended questions (are there anything else in statistics?), I get a variety of answers. One of the questions helps clarify what I mean by numbersense. The question asks students to comment on the distribution of a variable (median income) in a dataset of…

Read more »

What is numbersense?

March 19, 2014
By
What is numbersense?

It's Spring Break at NYU, which for professors, is not a break. I have been marking midterms for my business analytics class. Since I like to set open-ended questions (are there anything else in statistics?), I get a variety of answers. One of the questions helps clarify what I mean by numbersense. The question asks students to comment on the distribution of a variable (median income) in a dataset of…

Read more »

Define functions with optional parameters in SAS/IML

March 19, 2014
By
Define functions with optional parameters in SAS/IML

Last month I blogged about defining SAS/IML functions that have default parameter values. This language feature, which was introduced in SAS/IML 12.1, enables you to skip arguments when you call a user-defined function. The same technique enables you to define optional parameters. Inside the function, you can determine whether the […]

Read more »

MCMC for Econometrics Students – II

March 19, 2014
By
MCMC for Econometrics Students – II

This is the second in a set of posts about Monte Carlo Markov Chain (MCMC, or MC2) methods in Bayesian econometrics. The background was provided in this first post, where the Gibbs sampler was introduced.The main objective of the present post is to con...

Read more »

HubCab: Mapping All Taxi Trips in New York during 2011

March 18, 2014
By
HubCab: Mapping All Taxi Trips in New York during 2011

The densely populated yet beautiful HubCab [hubcab.org] by MIT Senseable Lab is an interactive map that captures the more than 170 million unique taxi trips that were made by around 13,500 taxi cabs within the City of New York in 2011. The map shows...

Read more »

Those wacky anti-Bayesians used to be intimidating, but now they’re just pathetic

March 18, 2014
By

From 2006: Eric Archer forwarded this document by Nick Freemantle, “The Reverend Bayes—was he really a prophet?”, in the Journal of the Royal Society of Medicine: Does [Bayes's] contribution merit the enthusiasms of his followers? Or is his legacy overhyped? . . . First, Bayesians appear to have an absolute right to disapprove of any […]The post Those wacky anti-Bayesians used to be intimidating, but now they’re just pathetic appeared…

Read more »

Learn to Manage Data at useR! 2014 or Online April 25

March 18, 2014
By
Learn to Manage Data at useR! 2014 or Online April 25

Before you can analyze data, it must be in the right form. Join me on April 25th for a 4-hour webinar that shows how to perform the most commonly used data management tasks in R. We will work through hands-on examples of … Continue reading →

Read more »

Learn EDA (exploratory data analysis) from the experts

March 18, 2014
By

The Facebook data science team has put together a great course on EDA at Udacity. EDA stands for exploratory data analysis. It is the beginning of any data analysis when you have a pile of data (or datasets) and you...

Read more »

Where’s Rick at SAS Global Forum 2014?

March 18, 2014
By
Where’s Rick at SAS Global Forum 2014?

Once again I'll be at SAS Global Forum this year. The 2014 location is Washington, D. C., so I am looking forward to greeting many friends in the government and consulting sectors. I always enjoy talking with SAS customers about statistics, simulations, matrix computations, and the SAS/IML product, so here's […]

Read more »

Omitting Constant may Introduce Biased Coefficients

March 18, 2014
By
Omitting Constant may Introduce Biased Coefficients

It is well known that dropping the constant in regression analysis may introduce bias. However, bias is really not the deeper issue.  The deeper issue is that by omitting the constant, you are specifying a very specific form for the relationship b...

Read more »

Timeline charts with googleVis

March 18, 2014
By
Timeline charts with googleVis

Last year at the Google I/O conference Mitchell Foley presented new developments of the Google Chart Tools API and one of the new features he mentioned were timeline charts (about 6 min into the talk). Timeline charts are a great way of visualising di...

Read more »

Cover of my forecasting textbook

March 18, 2014
By
Cover of my forecasting textbook

We now have a cover for the print version of my forecasting book with George Athanasopoulos. It should be on Amazon in a couple of weeks. The book is also freely available online. This is a variation of the most popular one in the poll conducted a mon...

Read more »


Subscribe

Email:

  Subscribe