“History is the prediction of the present”

June 6, 2015
By

Ethan Bolker sent me an email with the above title and wrote: That’s the first sentence of a Louis Menand book review in the March 30 New Yorker. It touches on some ideas you play with. If you haven’t seen it, you might put it on your (long?) queue of things to read, maybe blog […] The post “History is the prediction of the present” appeared first on Statistical Modeling,…

Read more »

Big Data and Official Statistics

June 5, 2015
By
Big Data and Official Statistics

  Big Data is THE topic of the freshly published Statistical Journal of the IAOS – Volume 31, issue 2. . Five articles deal with Big-Data topics: International collaboration to understand the relevance of Big Data for official statistics Web scraping techniques to collect data on consumer electronics and airfares for Italian HICP compilation The production of … Continue reading Big Data and Official Statistics

Read more »

Should you get the blood transfusion?

June 5, 2015
By

Gur Huberman writes: Apropos Ethics & Logistic Regression, the piece you wrote with Madigan: In late 2001 I [Gur] broke my femur trying to rollerblade with my daughter. (No IQ award for that.) I had surgery and my recovery was slow. Every time I tried to get on crutches I’d collapse and faint. Diagnosis: Anemia. […] The post Should you get the blood transfusion? appeared first on Statistical Modeling, Causal…

Read more »

New package for image processing in R

June 5, 2015
By
New package for image processing in R

[update: imager is now on CRAN. install via install.packages(“imager”)] I’ve written a package for image processing in R, with the goal of providing a fast API in R that lets you do things in C++ if you need to. The package is called imager, and it’ on Github. The whole thing is based on CImg, […]

Read more »

A quick one

June 5, 2015
By

Fabio Rojas asks: Should I do Bonferroni adjustments? Pros? Cons? Do you have a blog post on this? Most social scientists don’t seem to be aware of this issue. My short answer is that if you’re fitting mutlilevel models, I don’t think you need mu...

Read more »

What Would Replication Research Under an Error Statistical Philosophy Be?

June 4, 2015
By
What Would Replication Research Under an Error Statistical Philosophy Be?

Around a year ago on this blog I wrote: “There are some ironic twists in the way psychology is dealing with its replication crisis that may well threaten even the most sincere efforts to put the field on firmer scientific footing” That’s philosopher’s talk for “I see a rich source of problems that cry out […]

Read more »

Logit, Probit, & Heteroskedasticity

June 4, 2015
By
Logit, Probit, & Heteroskedasticity

I've blogged previously about specification testing in the context of Logit and Probit models. For instance, see here and here. Testing for homoskedasticity in these models is especially important, for reasons that are outlined in t...

Read more »

Can Bradley Wiggins Do It? Welcome to the Thunder-Drome!

June 4, 2015
By
Can Bradley Wiggins Do It? Welcome to the Thunder-Drome!

Many have tried. Most have failed.Bradley Wiggins knows this. He also knows the ordeal he faces, knows the pain he will endure and knows the scrutiny he will face. It's nothing he hasn't experienced before, having raced and won the world's most pr...

Read more »

Interview with Class Central

June 4, 2015
By

Recently I sat down with Class Central to do an interview about the Johns Hopkins Data Science Specialization. We talked about the motivation for designing the sequence and and the capstone project. With the demand for data science skills greater than ever, the importance of the specialization is only increasing. See the full interview at

Read more »

We need a title for our Daily Beast column

June 4, 2015
By

Kaiser and I will soon start a weekly column for the Daily Beast, focusing on statistics that are cited in political and civic debates. The question is, what to call it? We have a few possibilities but aren’t thrilled with any of them. So we could use some help from the wisdom of the crowd. […] The post We need a title for our Daily Beast column appeared first on…

Read more »

Forty-eight Hillarys in some order

June 4, 2015
By
Forty-eight Hillarys in some order

Boston Globe has an eye-catching full-page poster about Hillary's current endorsements among 115 important New Hampshire people. (link) This is an excerpt of the poster: Each of the 115 people are represented by a circle, with their names, titles and...

Read more »

2015 French Open Djokovic-Nadal Tennis Graphic, and Some Explanation

June 4, 2015
By
2015 French Open Djokovic-Nadal Tennis Graphic, and Some Explanation

We now have the ability to produce our tennis graphic in near-real time. Here's Djokovic-Nadal from the French Open quarterfinals, 4 June 2015. The graph below plays quickly (click on it to enlarge and replay), but as usual on my web pag...

Read more »

Link: Disinformation Visualization

June 3, 2015
By

In his piece Disinformation Visualization: How to lie with datavis, Mushon Zer-Aviv makes some interesting points about how framing the same data differently in visualization can make a big difference. Using the example of the abortion debate, he shows the usual chart tricks, cherry-picking, subsetting, etc., that is done to make the data support a particular story. This is … Continue reading Link: Disinformation Visualization

Read more »

Of buggy whips and moral hazards; or, Sympathy for the Aapor

June 3, 2015
By
Of buggy whips and moral hazards; or, Sympathy for the Aapor

We’ve talked before about those dark-ages classical survey sampling types who say you can’t do poop with opt-in samples. The funny thing is, these people do all sorts of adjustment themselves, in the sampling or in post-data weighting or both, to deal with the inevitable fact that the people you can actually reach when you […] The post Of buggy whips and moral hazards; or, Sympathy for the Aapor appeared…

Read more »

What I said about data science at Princeton Reunions

June 3, 2015
By
What I said about data science at Princeton Reunions

Here was how I spent last weekend: At college reunions in beautiful Princeton on a glorious sunny day. I also spoke about data science at a Faculty-Alumni panel titled "Science Under Attack!". Here is what I said: In the past five to 10 years, there has been an explosion of interest in using data in business decision-making. What happens when business executives learn that the data do not support their…

Read more »

The mystery of the density curve that was too short

June 3, 2015
By
The mystery of the density curve that was too short

I was reading a statistics book when I encountered a histogram that caught my eye. The histogram looked similar to the one at the left. It contained a normal density estimate overlaid on a histogram, but the height of the density curve seemed too short when compared to the heights […] The post The mystery of the density curve that was too short appeared first on The DO Loop.

Read more »

R vs Autobox vs ForecastPro vs …

June 3, 2015
By
R vs Autobox vs ForecastPro vs …

Every now and then a commercial software vendor makes claims on social media about how their software is so much better than the forecast package for R, but no details are provided. There are lots of reasons why you might select a particular software solution, and R isn’t for everyone. But anyone claiming superiority should […]

Read more »

June Reading List

June 2, 2015
By
June Reading List

Andrews, I. and T. B. Armstrong, 2015. Unbiased instrumental variables estimation under known first-stage sign. Cowles Foundation Discussion Paper No. 1984R, Yale University,Bajari, P., D. Nekipelov, S. P. Ryan, and M. Yang. 2015. Demand estimation wit...

Read more »

Air Pollution (PM10 and PM2.5) in Different Cities using Interactive Charts

June 2, 2015
By

Gardiner Harris, who is a South Asia correspondent of the New York Times, shared a personal story of his son’s breathing troubles in New Delhi, India, in a recent dispatch titled Holding Your Breath in India. In this post, I use data from the World H...

Read more »

Statistical Models with a Point of View: First vs. Third Person

June 2, 2015
By
Statistical Models with a Point of View: First vs. Third Person

Marketing data can be collected in the first or third person, and we require different statistical models for each point of view.Netflix encourages you to adopt a third-person perspective when it surveys your taste preferences by asking how often you w...

Read more »

Cross-validation != magic

June 2, 2015
By

In a post entitled “A subtle way to over-fit,” John Cook writes: If you train a model on a set of data, it should fit that data well. The hope, however, is that it will fit a new set of data well. So in machine learning and statistics, people split their data into two parts. […] The post Cross-validation != magic appeared first on Statistical Modeling, Causal Inference, and Social…

Read more »

Back from R/Finance in Chicago

June 2, 2015
By
Back from R/Finance in Chicago

I had a great time at the R/Finance conference in Chicago last Friday/Saturday. Some brief takeaways for me were:From Emanuel Derman's talk: It is is important to distinguish between theories and models. Theories live in an abstract world and for a giv...

Read more »

Report: EuroVis 2015

June 2, 2015
By

I attended EuroVis 2015 last week in Cagliari, Sardinia. This is the second-most important conference in the academic visualization world, and there were plenty of good sessions to choose from (full and short papers, state-of-the-art reports, and industry sessions). As usual, this is a highly subjective and incomplete report. I did not see anywhere near all … Continue reading Report: EuroVis 2015

Read more »


Subscribe

Email:

  Subscribe