Damn, I was off by a factor of 2!

December 16, 2014
By

I hate when that happens. Demography is tricky. Oh well, as they say in astronomy, who cares, it was less than an order of magnitude! The post Damn, I was off by a factor of 2! appeared first on Statistical Modeling, Causal Inference, and Social Scien...

Read more »

Learning R: Live Webinar, Interactive Self-Paced, or Site Visit?

December 15, 2014
By
Learning R: Live Webinar, Interactive Self-Paced, or Site Visit?

My recent blog post, Why R is Hard to Learn, must have hit a nerve as it was read by over 6,000 people in its first two days online.  If you’re using R to augment your work in SAS, SPSS … Continue reading →

Read more »

Another example of misleading time-based correlation

December 15, 2014
By
Another example of misleading time-based correlation

On my sister blog last week, I wrote about how to screw up a column chart. The chart designer apparently wanted to explore whether Rotten Tomato Scores are correlated with box office success, and whether the running time of a movie is correlated with box office success. In either case, the set of movies is a small one, those directed by Chris Nolan. Here is a better view of the…

Read more »

“Now the company appears to have screwed up badly, and they’ve done it in pretty much exactly the way you would expect a company to screw up when it doesn’t drill down into the data.”

December 15, 2014
By

Palko tells a good story: One of the accepted truths of the Netflix narrative is that CEO Reed Hastings is obsessed with data and everything the company does is data driven . . . Of course, all 21st century corporations are relatively data-driven. The fact that Netflix has large data sets on customer behavior does […] The post “Now the company appears to have screwed up badly, and they’ve done…

Read more »

On deck this week

December 15, 2014
By

Mon: “Now the company appears to have screwed up badly, and they’ve done it in pretty much exactly the way you would expect a company to screw up when it doesn’t drill down into the data.” Tues: Expectation propagation as a way of life Wed: I’d like to see a preregistered replication on this one […] The post On deck this week appeared first on Statistical Modeling, Causal Inference, and…

Read more »

Causal Modeling Update

December 15, 2014
By

In an earlier post on predictive modeling and causal inference, I mentioned my summer "reading list" for causal modeling:Re-read Pearl, and read the Heckman-Pinto critique.Re-read White et al. on settable systems and testing conditional indep...

Read more »

How students can improve their final grade without studying even more

December 15, 2014
By

Cross-post on my two blogs Here is something different: I wrote a piece on exam-taking tips. It's on a new website, Cafe, which has lots of good (non-quant) reads. The motivation for the piece is my observation that most American students are not taught how to take exams. As a professor, I notice that many students get lower scores than they deserve because of this. In this article, I describe…

Read more »

Elementwise minimum and maximum operators

December 15, 2014
By
Elementwise minimum and maximum operators

Like most programming languages, the SAS/IML language has many functions. However, the SAS/IML language also has quite a few operators. Operators can act on a matrix or on rows or columns of a matrix. They are less intuitive, but can be quite powerful because they enable you perform computations without […]

Read more »

Experient downdating algorithm for Leave-One-Out CV in RDA

December 15, 2014
By
Experient downdating algorithm for Leave-One-Out CV in RDA

In this post, I want to demonstrate a piece of experiment code for downdating algorithm for Leave-One-Out (LOO) Cross Validation in Regularized Discriminant Analysis [1]. In LOO CV, the program needs to calculate the inverse of \(\hat{\Sigma}_{k\v}(\la...

Read more »

Control Excel via SAS DDE & Python win32com

December 15, 2014
By
Control Excel via SAS DDE & Python win32com

Excel is probably the most used interface between human and data. Whenever you are dealing with business people, Excel is the de facto means for all things about data processing. I used to only use SAS and Python for number crunching but in one of my r...

Read more »

A time series classification contest

December 14, 2014
By
A time series classification contest

Amongst today’s email was one from someone running a private competition to classify time series. Here are the essential details. The data are measurements from a medical diagnostic machine which takes 1 measurement every second, and after 32–1000 seconds, the time series must be classified into one of two classes. Some pre-classified training data is […]

Read more »

The latest episode in my continuing effort to use non-sports analogies

December 14, 2014
By

In a unit about the law of large numbers, sample size, and margins of error, I used the notorious beauty, sex, and power example: A researcher, working with a sample of size 3000, found that the children of beautiful parents were more likely to be girls, compared to the children of less-attractive parents. Can such […] The post The latest episode in my continuing effort to use non-sports analogies appeared…

Read more »

Sunday data/statistics link roundup (12/14/14)

December 14, 2014
By

A very brief analysis suggests that economists are impartial when it comes to their liberal/conservative views. That being said, I'm not sure the regression line says what they think it does, particularly if you pay attention to the variance around the line (via Rafa). I am digging the simplicity of charted.co from the folks at Medium.

Read more »

The Rotterdam Model

December 14, 2014
By
The Rotterdam Model

Ken Clements (U. Western Australia) has sent me a copy of his paper, co-authored with Grace Gao this month, "The Rotterdam Demand Model Half a Century On". How appropriate it is to see this important landmark in econometrics honoured in this way. ...

Read more »

How to take exams

December 14, 2014
By

Here is something different: I wrote a piece on exam-taking tips. It's on a new website, Cafe, which has lots of good (non-quant) reads. The motivation for the piece is my observation that most American students are not taught how...

Read more »

I like the clever way they tell the story. It’s a straightforward series of graphs but the reader has to figure out where to click and what to do, which makes the experience feel more like a voyage of discovery.

December 14, 2014
By
I like the clever way they tell the story.  It’s a straightforward series of graphs but the reader has to figure out where to click and what to do, which makes the experience feel more like a voyage of discovery.

Jonathan Falk asks what I think of this animated slideshow by Matthew Klein on “How Americans Die”: Please click on the above to see the actual slideshow, as this static image does not do it justice. What do I think? Here was my reaction: It is good, but I was thrown off by the very […] The post I like the clever way they tell the story. It’s a straightforward…

Read more »

Monthly Weather in Netherlands

December 14, 2014
By
Monthly Weather in Netherlands

When I downloaded the KNMI meteorological data, the intention was to do something which takes more than just the computers memory. While it is clearly not big data, at the very least 100 years of daily data is not small either. So I took along a load o...

Read more »

When Did You Last Check Your Code?

December 13, 2014
By
When Did You Last Check Your Code?

Chris Blattman (Columbua U.) has a blog directed towards international development, economics, politics, and policy.In a post yesterday, Chris asks: "What happens when a very good political science journal checks the statistical code of its submissions...

Read more »

S. Stanley Young: Are there mortality co-benefits to the Clean Power Plan? It depends. (Guest Post)

December 13, 2014
By
S. Stanley Young: Are there mortality co-benefits to the Clean Power Plan? It depends. (Guest Post)

  S. Stanley Young, PhD Assistant Director Bioinformatics National Institute of Statistical Sciences Research Triangle Park, NC Are there mortality co-benefits to the Clean Power Plan? It depends. Some years ago, I listened to a series of lectures on finance. The professor would ask a rhetorical question, pause to give you some time to think, […]

Read more »

Don’t, don’t, don’t, don’t . . . We’re brothers of the same mind, unblind

December 13, 2014
By
Don’t, don’t, don’t, don’t . . . We’re brothers of the same mind, unblind

Hype can be irritating but sometimes it’s necessary to get people’s attention (as in the example pictured above). So I think it’s important to keep these two things separate: (a) reactions (positive or negative) to the hype, and (b) attitudes about the subject of the hype. Overall, I like the idea of “data science” and […] The post Don’t, don’t, don’t, don’t . . . We’re brothers of the same…

Read more »

ggRandomForests: Visually Exploring random forests. V1.1.1 release.

December 13, 2014
By
ggRandomForests: Visually Exploring random forests. V1.1.1 release.

Release early and often. http://cran.r-project.org/web/packages/ggRandomForests/index.html I may have been aggressive numbering the first CRAN release at v1.0, but there’s no going back now. The design of the feature set is complete even if the code has some catching up to… Continue reading →

Read more »

"The Error Term in the History of Time Series Econometrics"

December 12, 2014
By
"The Error Term in the History of Time Series Econometrics"

While we're on the subject of the history of econometrics ......... blog-reader Mark Leeds kindly drew my attention to this interesting paper published by Duo Qin and Christopher Gilbert in Econometric Theory in 2001.I don't recall reading this pa...

Read more »


Subscribe

Email:

  Subscribe