## A letter to reporters on the economy

August 19, 2013
By

The New York Times took over 1,000 words to tell us that Big Data won't change the economy (or is it the economists' profession?) ("Is Big Data an economic Big Dud?") I'm less pessimistic; I think the collection of vast troves of observational data is ultimately beneficial but only if (a) we set a high bar for analytics, such as requiring multiple corroborating data sources pointing to the same conclusion;…

## A letter to reporters on the economy

August 19, 2013
By

The New York Times took over 1,000 words to tell us that Big Data won't change the economy (or is it the economists' profession?) ("Is Big Data an economic Big Dud?") I'm less pessimistic; I think the collection of vast troves of observational data is ultimately beneficial but only if (a) we set a high bar for analytics, such as requiring multiple corroborating data sources pointing to the same conclusion;…

## Errors that cause SAS to "freeze"… and what to do about them

August 19, 2013
By

Even the best programmers make mistakes. For most errors, SAS software displays the nature and location of the error, returns control to the programmer, and awaits further instructions. However, there are a handful of insidious errors that cause SAS to think that a statement or program is not finished. For [...]

## Fitting psychometric functions using STAN

August 19, 2013
By

STAN is a new system for Bayesian inference, similar to BUGS and JAGS. I’ve played with it a bit and it’s quite promising, it really has the potential to make MCMC less of a pain (on simple models). I’ve written a short introduction to fitting psychometric functions using STAN and R, in case that’s useful […]

## Forecasting a Timeseries

August 19, 2013
By
$Forecasting a Timeseries$

Suppose you have decided on a suitable model for a timeseries. In this case, we have selected an ARIMA(2,1,3) model, using the Akaike Information Criteria (AIC) as our sole criterion for choosing between various models here, where we model the DJIA. Note: There are many criteria for choosing a model, and the AIC is only […]

## The Perfect Visualization

August 19, 2013
By

There are many rules about how to visualize data. We know how to encode specific types of data, what visual encodings work well, and what does not work so well. But is there such a thing as a perfect visualization for a given set of data? This is a topic that comes up every now […]

## The importance of being wrong

August 18, 2013
By

We don’t like to think we are wrong One of the key ideas in statistics is that sometimes we will be wrong. When we report a 95% confidence interval, we will be wrong 5% of the time. Or in other … Continue reading →

## Endogenous Spatial Lags for the Linear Regression Model

August 18, 2013
By

Over the past number of years, I have noted that spatial econometric methods have been gaining popularity. This is a welcome trend in my opinion, as the spatial structure of data is something that should be explicitly included in the empirical modelling procedure. Omitting spatial effects assumes that the location co-ordinates for observations are unrelated […]

## Endogenous Spatial Lags for the Linear Regression Model

August 18, 2013
By

Over the past number of years, I have noted that spatial econometric methods have been gaining popularity. This is a welcome trend in my opinion, as the spatial structure of data is something that should be explicitly included in the empirical modelling procedure. Omitting spatial effects assumes that the location co-ordinates for observations are unrelated […]

## A lot of statistical methods have this flavor, that they are a solution to a mathematical problem that has been posed without a careful enough sense of whether the problem is worth solving in the first place

August 18, 2013
By

Stuart Hurlbert writes: A colleague recently forwarded to me your 2012 paper with Hill and Yajima on the multiple comparison “non-problem”, as I call it. You and your colleagues might find of interest a 2012 paper [with Celia Lombardi] that reaches similar conclusions by a colleague and myself which is attached. Similar but not identical, […]The post A lot of statistical methods have this flavor, that they are a solution…

## Exercise in REML/Mixed model

August 18, 2013
By

I want to build a bit more experience in REML, so I decided to redo some of the SAS examples in R. This post describes the results of example 59.1 (page 5001, SAS(R)/STAT User guide 12.3 link). Following the list from freshbiostats I will analyze ...

## (That’s) Numbersense by Kaiser Fung

August 18, 2013
By

I recently received a copy of Numbersense by Kaiser Fung for review. Fung is the author of a blog I have a lot of respect for : Junk Charts. The current post at the top of Junk Charts is about...

## Rock’n'roll stars

August 17, 2013
By

We're spending a few days in the Tuscan seaside. But both Marta and I seem to have lost the Italian genes in us and can barely tolerate the hot weather \$-\$ XY is not too keen on swimming or even being in the sea, but he does enjoy rolling around in the...

## Somebody’s looking for a book on time series analysis in the style of Angrist and Pischke, or Gelman and Hill

August 17, 2013
By

Devrup Ghatak writes: I am a student of economics and recently read your review of Mostly Harmless Econometrics. In the review you mention that the book contains no time series. Given that your book on data analysis (Data Analysis using Regression) does not contain any time series material either, I wonder if you happen to […]The post Somebody’s looking for a book on time series analysis in the style of…

## More than macros

August 17, 2013
By

When explaining about the Excel automation work we do at Sharp Statistics often the initial response is ‘so you write VBA macros’. In fact we don’t use Visual Basic for Applications (VBA) macros to build our solutions but instead use ...

## Using Heatmaps to Uncover the Individual-Level Structure of Brand Perceptions

August 17, 2013
By

Heatmaps, when the rows and columns are appropriately ordered, provide insight into the data structure at the individual level.  In an earlier post I showed a cluster heatmap with dendrograms for both the rows and the columns.  In addition, I...

## BDA at 40% off!

August 16, 2013
By

Our publisher informs me of the exciting news that Amazon is now selling the 3rd edition of our book at 40% off! Enjoy. The post BDA at 40% off! appeared first on Statistical Modeling, Causal Inference, and Social Science.

## Learning about correlations using cross-sectional and over-time comparisons between and within countries

August 16, 2013
By

Antonio Rinaldi writes: Here in Italy an “hype” topic is the “staffetta tra generazioni”, handover between generations: since unemployment rate in young people is very high, someone in the government is thinking to encourage older people to anticipate their retirement to make more jobs available for youngs. I am not an economist and I don’t […]The post Learning about correlations using cross-sectional and over-time comparisons between and within countries appeared…

## More on AIC, WAIC, etc

August 15, 2013
By

Following up on our discussion from the other day, Angelika van der Linde sends along this paper from 2012 (link to journal here). And Aki pulls out this great quote from Geisser and Eddy (1979): This discussion makes clear that in the nested case this method, as Akaike’s, is not consistent; i.e., even if \$M_k\$ […]The post More on AIC, WAIC, etc appeared first on Statistical Modeling, Causal Inference, and…

## Blaming scientific fraud on the Kuhnians

August 15, 2013
By

I wouldn’t go that far, but I’ll send along this article by Ahti-Veikko Pietarinen that was sent to me by Lee Sechrest. Those of you who like this sort of thing might like this sort of thing. I neither endorse nor anti-endorse. Or, I should say, I am in sympathy with the author’s general attitude […]The post Blaming scientific fraud on the Kuhnians appeared first on Statistical Modeling, Causal Inference,…

## Beautiful spider loses its way

August 15, 2013
By

On Twitter, Andy C. (@AnkoNako) asked me to look at this pretty creation at NFL.com (link). There is a reason why you don't read much about spider charts (web charts, radar charts, etc.) here. While this chart is beautifully constructed,...

## Beautiful spider loses its way 2

August 15, 2013
By

A double post today. In the previous post, I talked about NFL.com's visualization of football player statistics. In this post, I offer a few different views of the same data. *** The first is a dot plot arranged in small...

## Numbersense Pros: Avinash Kaushik

August 15, 2013
By

Today, I have the honor of interviewing Avinash Kaushik, author of the bible known as Web Analytics 2.0, and a digital marketing evangelist at Google. He also has a must-read blog called Occam's Razor. Occam's Razor is a principle championed by statisticians that can be summarized as "as simple as possible but not too simple". It is a principle and therefore it also draws controversy from some quarters. Kaushik's blog…