How Could Classification Trees Be So Fast on Categorical Variables?

December 8, 2015
By
How Could Classification Trees Be So Fast on Categorical Variables?

I think that over the past months, I have been saying non-correct things about classification with categorical covariates. Because I never took time to look at it carefuly. Consider some simulated dataset, with a logistic regression, > n=1e3 > set.seed(1) > X1=runif(n) > q=quantile(X1,(0:26)/26) > q[1]=0 > X2=cut(X1,q,labels=LETTERS[1:26]) > p=exp(-.1+qnorm(2*(abs(.5-X1))))/(1+exp(-.1+qnorm(2*(abs(.5-X1))))) > Y=rbinom(n,size=1,p) > df=data.frame(X1=X1,X2=X2,p=p,Y=Y) Here, we use some continuous covariate, except that is considered as not-observed. Instead, we have a categorical covariate…

Read more »

Many rules of statistics are wrong

December 8, 2015
By
Many rules of statistics are wrong

There are two kinds of people who violate the rules of statistical inference: people who don't know them and people who don't agree with them.  I'm the second kind.The rules I hold in particular contempt are:The interpretation of p-values: Suppose...

Read more »

Hierarchical modeling when you have only 2 groups: I still think it’s a good idea, you just need an informative prior on the group-level variation

December 8, 2015
By

Dan Chamberlain writes: I am working on a Bayesian analysis of some data from a randomized controlled trial comparing two different drugs for treating seizures in children. I have been using your book as a resource and I have a question about hierarchical modeling. If you have the time, I would greatly appreciate any advice […] The post Hierarchical modeling when you have only 2 groups: I still think it’s…

Read more »

Probabilistic Integration

December 8, 2015
By
Probabilistic Integration

Mark Girolami sends along a new paper by Francois-Xavier Briol, Chris Oates, Michael Osborne, Dino Sejdinovic, and himself. The idea is to consider numerical integration as a statistical problem, to say that the integral being estimated is an unknown parameter and then to perform inference about it. This is related to ideas of Xiao-Li Meng, […] The post Probabilistic Integration appeared first on Statistical Modeling, Causal Inference, and Social Science.

Read more »

Write unit tests!

December 7, 2015
By
Write unit tests!

Since 2000, I’ve been working on R/qtl, an R package for mapping the genetic loci (called quantitative trait loci, QTL) that contribute to variation in quantitative traits in experimental crosses. The Bioinformatics paper about it is my most cited; also see my 2014 JORS paper, “Fourteen years of R/qtl: Just barely sustainable.” It’s a bit […]

Read more »

Two peas in a pod

December 7, 2015
By
Two peas in a pod

Earlier today I've seen this post about Frank Wilcoxon's work on non-parametric statistics, on the Significance website. I've only very recently become involved in using some non-parametric methods (notably for our work on the EVPPI...

Read more »

Statbusters: standing may or may not stand a chance

December 7, 2015
By

In our latest Statbusters column for the Daily Beast, we read the research behind the claim that "standing reduces odds of obesity". Especially at younger companies, it is trendy to work at standing desks because of findings like this. We find a variety of statistical issues calling for better studies. For example, the observational dataset used provides no clue as to whether sitting causes obesity or obesity leads to more…

Read more »

Use of Jeffreys prior in estimating climate sensitivity

December 7, 2015
By
Use of Jeffreys prior in estimating climate sensitivity

William Morris writes: A discussion of the use of Bayesian estimation in calculating climate sensitivity (to doubled CO2) occurred recently in the comments at the And Then There’s Physics (ATTP) blog. One protagonist, ‘niclewis’, a well known climate sensitivity researcher, uses the Jeffreys prior in his estimations. His estimations are always at the low end […] The post Use of Jeffreys prior in estimating climate sensitivity appeared first on Statistical…

Read more »

On deck this week

December 7, 2015
By

Mon: Use of Jeffreys prior in estimating climate sensitivity Tues: Hierarchical modeling when you have only 2 groups: I still think it’s a good idea, you just need an informative prior on the group-level variation Wed: I definitely wouldn’t frame it as “To determine if the time series has a change-point or not.” The time […] The post On deck this week appeared first on Statistical Modeling, Causal Inference, and…

Read more »

Why doesn’t PROC UNIVARIATE support certain common distributions?

December 7, 2015
By
Why doesn’t PROC UNIVARIATE support certain common distributions?

A SAS customer asked: Why isn't the chi-square distribution supported in PROC UNIVARIATE? That is an excellent question. I remember asking a similar question when I first started learning SAS. In addition to the chi-square distribution, I wondered why the UNIVARIATE procedure does not support the F distribution. These are […] The post Why doesn't PROC UNIVARIATE support certain common distributions? appeared first on The DO Loop.

Read more »

NYC Stan meetup 12 December

December 7, 2015
By

The next NYC Stan meetup is on Saturday: Feel free to bring things you’re working on or join in on projects some of the others are working on. A couple of the developers will be around to answer questions and help out. If you don’t have anything to work on, the Stan team could use […] The post NYC Stan meetup 12 December appeared first on Statistical Modeling, Causal Inference,…

Read more »

New Review of Forecasting at Bank of England

December 7, 2015
By

Check it out here. It's thorough and informative.  It's interesting and unfortunate that even the Bank of England, the great "fan chart pioneer," produces density forecasts for only three of eleven variables forecasted (p. 15). In my view, the mos...

Read more »

Cannabis/IQ follow-up: Same old story

December 6, 2015
By

Ole Rogeberg writes: The way researchers respond to criticism is a recurring theme on your blog, so you might find this amusing as a brief follow-up on the cannabis/IQ discussion you’ve covered before: The Dunedin longitudinal study has now been going for 40 years, and the lead researchers in charge of the study recently published an […] The post Cannabis/IQ follow-up: Same old story appeared first on Statistical Modeling, Causal Inference,…

Read more »

Venezuelan Parliamentary Election: What do the Polls Say?

December 6, 2015
By
Venezuelan Parliamentary Election: What do the Polls Say?

There is not a huge population of opinion polls covering this parliamentary election in Venezuela, but all I’ve can be used to gauge the public opinion by the local polling houses. This posting begs an obvious question: how has the mood in Venezuela ...

Read more »

Venezuelan Parliamentary Election: What do the Polls Say?

December 6, 2015
By
Venezuelan Parliamentary Election: What do the Polls Say?

There is not a huge population of opinion polls covering this parliamentary election in Venezuela, but all I’ve can be used to gauge the public opinion by the local polling houses. This posting begs an obvious question: how has the mood in Venezuela ...

Read more »

Venezuelan Parliamentary Election: What do the Polls Say?

December 6, 2015
By
Venezuelan Parliamentary Election: What do the Polls Say?

There is not a huge population of opinion polls covering this parliamentary election in Venezuela, but all I’ve can be used to gauge the public opinion by the local polling houses. This posting begs an obvious question: how has the mood in Venezuela ...

Read more »

Venezuelan Parliamentary Election: What do the Polls Say?

December 6, 2015
By
Venezuelan Parliamentary Election: What do the Polls Say?

There is not a huge population of opinion polls covering this parliamentary election in Venezuela, but all I’ve can be used to gauge the public opinion by the local polling houses. This posting begs an obvious question: how has the mood in Venezuela ...

Read more »

Venezuelan Parliamentary Election: What do the Polls Say?

December 6, 2015
By
Venezuelan Parliamentary Election: What do the Polls Say?

There is not a huge population of opinion polls covering this parliamentary election in Venezuela, but all I've can be used to gauge the public opinion by the local polling houses. This posting begs an obvious question: how has the mood in Venezuel...

Read more »

Venezuelan Parliamentary Election: What do the Polls Say?

December 6, 2015
By
Venezuelan Parliamentary Election: What do the Polls Say?

There is not a huge population of opinion polls covering this parliamentary election in Venezuela, but all I've can be used to gauge the public opinion by the local polling houses. This posting begs an obvious question: how has the mood in Venezuel...

Read more »

Beware of questionable front page articles warning you to beware of questionable front page articles (2)

December 5, 2015
By
Beware of questionable front page articles warning you to beware of questionable front page articles (2)

Such articles have continued apace since this blogpost from 2013. During that time, meta-research, replication studies, statistical forensics and fraudbusting have become popular academic fields in their own right. Since I regard the ‘programme’ (to use a Lakatosian term) as essentially a part of the philosophy and methodology of science, I’m all in favor of it—I employed the term “metastatistics” […]

Read more »

Judea Pearl and I briefly discuss extrapolation, causal inference, and hierarchical modeling

December 5, 2015
By

OK, I guess it looks like the Buzzfeed-style headlines are officially over. Anyway, Judea Pearl writes: I missed the discussion you had here about Econometrics: Instrument locally, extrapolate globally, which also touched on my work with Elias Bareinboim. So, please allow me to start a new discussion about extrapolation and external validity. First, two recent […] The post Judea Pearl and I briefly discuss extrapolation, causal inference, and hierarchical modeling…

Read more »

Stan at NIPS 2015

December 5, 2015
By
Stan at NIPS 2015

With NIPS 2014 a distant memory, we have a web page covering all the Stan-related activity at NIPS 2015 Including how to score a nifty Stan sticker. If you have something else to add to the list, let us know in the comments. The post Stan at NIPS 20...

Read more »

Syllabus for my course on design and analysis of sample surveys

December 4, 2015
By

Here’s last year’s course plan. Maybe I’ll change it a bit, haven’t decided yet. The course number is Political Science 4365, and it’s also cross-listed in Statistics. The post Syllabus for my course on design and analysi...

Read more »


Subscribe

Email:

  Subscribe