Statbusters: Games people play with the placebo effect

November 3, 2015
By

In the first two chapters of Numbersense, I discuss how people game statistics, and why gaming is inevitable. I have also written about the placebo effect before. Another article has appeared covering the same topic -- the industry doesn't like the fact that more and more drugs fail to clear the "placebo" hurdle; and the industry thinks the problem is that the placebo effect is mysteriously increasing over time. What…

Read more »

Learning to Love Bayesian Statistics

November 3, 2015
By
Learning to Love Bayesian Statistics

At Strata NYC 2015, O'Reilly Media's data science conference, I gave a talk called "Learning to Love Bayesian Statistics".  The video is available now:The sound quality is not great, and the clicker to advance the slides was a little wonky, but ot...

Read more »

My job here is done

November 3, 2015
By
My job here is done

It was cool, back in the day, to be mocked in the House of Commons: And of course I was happy a few months ago to be cited by the Supreme Court: But the high point of my journalistic career is being mentioned in Private Eye (see above). I can retir...

Read more »

RIP GFT

November 3, 2015
By

It appears that Google Flu Trends (GFT) has slipped quietly into the night. In a short post to the Google Research blog, the team behind GFT announced that they are "no longer publishing" flu estimates, effectively ending the seven-year-old experiment. The GFT home page now links to some historical datasets. The post was dated August 15, and it appears that mainstream media completely missed it. GFT was one of the…

Read more »

Loss Developments via Growth Curves and Stan

November 3, 2015
By
Loss Developments via Growth Curves and Stan

Last week I posted a biological example of fitting a non-linear growth curve with Stan/RStan. Today, I want to apply a similar approach to insurance data using ideas by David Clark [1] and James Guszcza [2].Instead of predicting the growth of dugongs (...

Read more »

Loss Developments via Growth Curves and Stan

November 3, 2015
By
Loss Developments via Growth Curves and Stan

Last week I posted a biological example of fitting a non-linear growth curve with Stan/RStan. Today, I want to apply a similar approach to insurance data using ideas by David Clark [1] and James Guszcza [2].Instead of predicting the growth of dugongs (...

Read more »

VIS 2015 – Friday

November 3, 2015
By
VIS 2015 – Friday

The final day of the conference was much shorter, only one full session and then the closing with the capstone. Here is a report on a few more papers, the capstone, a panel left over from Thursday, and a few random tidbits. Running Wrap-Up The running “club” started out well with five runners on Tuesday. … Continue reading VIS 2015 – Friday

Read more »

Keep eating those sausages

November 2, 2015
By

I am outsourcing this post to Aaron Carroll, whose Upshot column eviscerates the recent claim that eating meat will give you cancer, or that eating meat is the same as smoking cigarettes. While the media is partly culpable for spreading misinformation...

Read more »

Stop screaming already: Exaggeration of effects of fan distraction in NCAA basketball

November 2, 2015
By
Stop screaming already:  Exaggeration of effects of fan distraction in NCAA basketball

John Ezekowitz writes: I have been reading your work on published effect sizes, and I thought you might be interested in this example, which is of small consequence but grates me as a basketball and data fan. Kevin Quealy and Justin Wolfers published an analysis in The NYT on fans’ effectiveness in causing road teams […] The post Stop screaming already: Exaggeration of effects of fan distraction in NCAA basketball…

Read more »

Software from CSHL Genome Informatics 2015

November 2, 2015
By
Software from CSHL Genome Informatics 2015

I just returned from the Genome Informatics meeting at Cold Spring Harbor. This was, hands down, the best scientific conference I've been to in years. The quality of the talks and posters was excellent, and it was great meeting in person many...

Read more »

On deck this week

November 2, 2015
By

Mon: Stop screaming already: Exaggeration of effects of fan distraction in NCAA basketball Tues: My job here is done Wed: The tabloids strike again Thurs: Econometrics: Instrument locally, extrapolate globally Fri: I wish Napoleon Bonaparte had never been born Sat: This is a workshop you can’t miss: DataMeetsViz Sun: You won’t believe these stunning transformations: […] The post On deck this week appeared first on Statistical Modeling, Causal Inference, and…

Read more »

Statistics report raises mixed emotions

November 2, 2015
By
Statistics report raises mixed emotions

It's gratifying to live through the incredible rise of statistics as a discipline. In a recent report by the American Statistical Association (ASA), we learned that enrollment at all levels (bachelor, master and doctorate) has exploded in the last 5-10...

Read more »

Can’t find that data? Search all variables in all data sets

November 2, 2015
By
Can’t find that data? Search all variables in all data sets

Sometimes I can't remember where I put things. If I lose my glasses or garden tools, I am out of luck. But when I can't remember where I put some data, I have SAS to help me find it. When I can remember the name of the data set, my […] The post Can't find that data? Search all variables in all data sets appeared first on The DO Loop.

Read more »

VIS 2015 – Thursday

November 2, 2015
By
VIS 2015 – Thursday

Between the sessions, parties, discussions, running, and the occasional short sleep break, I fell a bit behind on the blogging about VIS 2015. After separate postings for Tuesday and Wednesday, this part covers Thursday. The posting about Friday will be short, but squeezing both days into one felt like a bit too much. BTW, the headings you see … Continue reading VIS 2015 – Thursday

Read more »

Our Differential Privacy Mini-series

November 2, 2015
By
Our Differential Privacy Mini-series

We’ve just finished off a series of articles on some recent research results applying differential privacy to improve machine learning. Some of these results are pretty technical, so we thought it was worth working through concrete examples. And some of the original results are locked behind academic journal paywalls, so we’ve tried to touch on … Continue reading Our Differential Privacy Mini-series

Read more »

Clustering Customer Satisfaction Ratings

November 1, 2015
By
Clustering Customer Satisfaction Ratings

We run our cluster analysis with great expectations, hoping to uncover diverse segments with contrasting likes and dislikes of the brands they use. Instead, too often, our K-means analysis returns the above graph of parallel lines indicating that the p...

Read more »

2 new thoughts on Cauchy priors for logistic regression coefficients

November 1, 2015
By

Aki noticed this paper, On the Use of Cauchy Prior Distributions for Bayesian Logistic Regression, by Joyee Ghosh, Yingbo Li, and Robin Mitra, which begins: In logistic regression, separation occurs when a linear combination of the predictors can perfectly classify part or all of the observations in the sample, and as a result, finite maximum […] The post 2 new thoughts on Cauchy priors for logistic regression coefficients appeared first…

Read more »

Vacancies in Europe

November 1, 2015
By
Vacancies in Europe

I like playing around with data from Eurostat. At this time the tools to do so are just so easy. There are tools to pull the data directly from the data base in R (eurostat package). Process it a bit using dplyr and before you know it, ggplot makes a p...

Read more »

Don’t use stats::aggregate()

October 31, 2015
By

When working with an analysis system (such as R) there are usually good reasons to prefer using functions from the “base” system over using functions from extension packages. However, base functions are sometimes locked into unfortunate design compromises that can now be avoided. In R’s case I would say: do not use stats::aggregate(). Read on … Continue reading Don’t use stats::aggregate()

Read more »

The Rachel Tanur Memorial Prize for Visual Sociology

October 31, 2015
By
The Rachel Tanur Memorial Prize for Visual Sociology

Judy Tanur writes: The Rachel Tanur Memorial Prize for Visual Sociology recognizes students in the social sciences who incorporate visual analysis in their work. The contest is open worldwide to undergraduate and graduate students (majoring in any social science). It is named for Rachel Dorothy Tanur (1958–2002), an urban planner and lawyer who cared deeply […] The post The Rachel Tanur Memorial Prize for Visual Sociology appeared first on Statistical…

Read more »

WHIPPING BOYS AND WITCH HUNTERS (ii)

October 31, 2015
By
WHIPPING BOYS AND WITCH HUNTERS (ii)

At least as apt today as 3 years ago…HAPPY HALLOWEEN! Memory Lane with new comments in blue.  In an earlier post I alleged that frequentist hypotheses tests often serve as whipping boys, by which I meant “scapegoats”, for the well-known misuses, abuses, and flagrant misinterpretations of tests (both simple Fisherian significance tests and Neyman-Pearson tests, although in different ways)—as […]

Read more »

Competing views on Argentina’s Frontrunner

October 31, 2015
By
Competing views on Argentina’s Frontrunner

It might be too early to state anything solid, but the few polls prospecting the runoff election in Argentina seem to be telling two distinct histories. Hugo Haime and IPSOS suggest Daniel Scioli is going up while Elypsis and Polldata say he’s sin...

Read more »

Agile Web Mining at Bing

October 30, 2015
By
Agile Web Mining at Bing

The web search industry is making great progress in transitioning from building tools for finding pages and sites to building tools that leverage and surface facts and knowledge. The local search space - where I work in Bing - is...

Read more »


Subscribe

Email:

  Subscribe