How does Brad Cooper analyze hierarchical survey data with post-stratification?

December 1, 2015
By

Laura Holder writes: I am working on a project involving a large survey data set and am interested in applying a model-based approach in the context of post-stratification (as you frequently discuss). I’m attempting to determine the most suitable approach for my circumstance. The data set I am working with is collected via a two […] The post How does Brad Cooper analyze hierarchical survey data with post-stratification? appeared first…

Read more »

Notes from Warsaw R meetup

December 1, 2015
By
Notes from Warsaw R meetup

I had the great pleasure time to attend the Warsaw R meetup last Thursday. The organisers Olga Mierzwa and Przemyslaw Biecek had put together an event with a focus on R in Insurance (btw, there is a conference with the same name), discussing examples o...

Read more »

Internet use and religion, part five

November 30, 2015
By
Internet use and religion, part five

[If you are jumping into the middle of this series, you might want to start with this article, which explains the methodological approach I am taking.]In the previous article, I show results from two regression models that predict religious affiliation...

Read more »

7 tips for work-life balance

November 30, 2015
By
7 tips for work-life balance

A student writes in: Dear Sherri: I am **, a PhD student in ** in ** University. I am trying to work productively. For example, I tried to use some todo list software, such as, remember the milk and omnifocus, and read related books. But feeling of overwhelming happens every day, and no achievements were […] The post 7 tips for work-life balance appeared first on Statistical Modeling, Causal Inference,…

Read more »

Size matters: Preserving the aspect ratio of the data in ODS graphics

November 30, 2015
By
Size matters: Preserving the aspect ratio of the data in ODS graphics

When creating a statistical graphic such as a line plot or a scatter plot, it is sometimes important to preserve the aspect ratio of the data. For example, if the range of the X and Y variables are equal, it can be useful to display the data in a square […] The post Size matters: Preserving the aspect ratio of the data in ODS graphics appeared first on The DO…

Read more »

The hidden benefits of open-source software

November 29, 2015
By
The hidden benefits of open-source software

I’ve been having discussions with colleagues and university administration about the best way for universities to manage home-grown software. The traditional business model for software is that we build software and sell it to everyone willing to pay. Very often, that leads to a software company spin-off that has little or nothing to do with the university that nurtured the […]

Read more »

Comedy book with surefire can’t-miss formula, misses

November 29, 2015
By

The other day at the library I noticed a pink-covered book, “We Killed: The Rise of Women in American Comedy.” It was filled with interviews. Cool! I checked it out and . . . jeez was it boring. It’s hard to imagine you could interview a bunch of comedians and come up with something so […] The post Comedy book with surefire can’t-miss formula, misses appeared first on Statistical Modeling,…

Read more »

The proliferation of useless data

November 29, 2015
By

One of the secrets of great data analysis is thoughtful data collection. Great data collection is necessary but not sufficient for great data analysis. I recently had the unfortunate need to select a new doctor. Every time I had to do this, it has been an exercise in frustration and desperation. And after wasting hours and hours perusing the "data" on doctors, inevitably I give up and just throw a…

Read more »

Attention please!

November 29, 2015
By
Attention please!

They all use statistics …  in the media, in politics, in sports. But they mostly forget that statistics, especially official statistics, are made by professionals in a quite demanding, time- and resource-consuming process. The WO/MAN-IN-THE-MIDDLE, the professionals, providing information and knowledge from facts remain hidden (despite Googles’ statement that statistician will be ‘the sexy job in … Continue reading Attention please!

Read more »

Wind in Netherlands II

November 29, 2015
By
Wind in Netherlands II

Two weeks ago I plotted how wind measurements on the edge of the North Sea changed in the past century. This week the same dataset is used for hypothesis testing.DataThe most important things to reiterate from previous post is that the data is from KNM...

Read more »

Additional thoughts about ‘Lorenz curves’ to compare models

November 29, 2015
By
Additional thoughts about ‘Lorenz curves’ to compare models

A few month ago, I did mention a graph, of some so-called Lorenz curves to compare regression models, see e.g. Progressive’s slides (thanks Guillaume for the reference) The idea is simple. Consider some model for the pure premium (in insurance, it is the quantity that we like to model), i.e. the conditional expected valeur On some dataset, we have our predictions, as well as observed quantities, . The curve are obtained simply…

Read more »

Return to the Comedy Hour: P-values vs posterior probabilities (1)

November 29, 2015
By
Return to the Comedy Hour: P-values vs posterior probabilities (1)

Some recent criticisms of statistical tests of significance have breathed brand new life into some very old howlers, many of which have been discussed on this blog. One variant that returns to the scene every decade I think (for 50+ years?), takes a “disagreement on numbers” to show a problem with significance tests even from a “frequentist” perspective.  Since it’s […]

Read more »

We got mooks

November 28, 2015
By
We got mooks

Columbia University’s Data Science Institute is releasing some mooks, and I’m part of it. I’ll first give the official announcement and then share some of my thoughts. The official announcement: The Data Science Institute at Columbia University is excited to announce the launch of its first online-education series, Data Science and Analytics in Context, on […] The post We got mooks appeared first on Statistical Modeling, Causal Inference, and Social…

Read more »

You’ll never believe what this girl wrote in her diary (NSFW)

November 27, 2015
By

Arber Tasimi heard about our statistics diaries and decided to try it out in the psychology class he was teaching. The students liked his class but a couple of them pushed back against the diaries, describing the assignment as pointless or unhelpful in their learning. This made me think that it may be that a […] The post You’ll never believe what this girl wrote in her diary (NSFW) appeared…

Read more »

“iPredict to close after Govt refuses anti-money laundering law exemption”

November 26, 2015
By

Richard Barker points us to an update on ipredict, the New Zealand political prediction market. From the news article by Hamish Rutherford: The site, run by Victoria University of Wellington’s commercialisation arm, VicLink, issued a statement to its website and on Twitter on Thursday. According to the iPredict statement, Associate Justice Minister Simon Bridges refused […] The post “iPredict to close after Govt refuses anti-money laundering law exemption” appeared first…

Read more »

Boston Stan meetup 1 Dec

November 25, 2015
By

Here’s the announcement: Using Stan for variational inference, plus a couple lightning talks Dustin Tran will give a talk on using Stan for variational inference, then we’ll have a couple lightening (5 minute-ish) talks on projects. David Sparks will talk, I will talk about some of my work and we’re looking for 1-2 more volunteers. […] The post Boston Stan meetup 1 Dec appeared first on Statistical Modeling, Causal Inference,…

Read more »

A thanksgiving dplyr Rubik’s cube puzzle for you

November 25, 2015
By

Nick Carchedi is back visiting from DataCamp and for fun we came up with a dplyr Rubik's cube puzzle. Here is how it works. To solve the puzzle you have to make a 4 x 3 data frame that spells Thanksgiving like this: View the code on Gist. To solve the puzzle you need to pipe this

Read more »

Gary McClelland agrees with me that dichotomizing continuous variables is a bad idea. He also thinks my suggestion of dividing a variable into 3 parts is also a mistake.

November 25, 2015
By
Gary McClelland agrees with me that dichotomizing continuous variables is a bad idea.  He also thinks my suggestion of dividing a variable into 3 parts is also a mistake.

In response to some of the discussion that inspired yesterday’s post, Gary McClelland writes: I remain convinced that discretizing a continuous variable, especially for multiple regression, is the road to perdition. Here I explain my concerns. First, I don’t buy the motivation that discretized analyses are easier to explain to lay citizens and the press. […] The post Gary McClelland agrees with me that dichotomizing continuous variables is a bad…

Read more »

3 YEARS AGO (NOVEMBER 2012): MEMORY LANE

November 25, 2015
By
3 YEARS AGO (NOVEMBER 2012): MEMORY LANE

MONTHLY MEMORY LANE: 3 years ago: November 2012. I mark in red three posts that seem most apt for general background on key issues in this blog.[1]. Please check out others that didn’t make the “bright red cut”. If you’re interested in the Likelihood Principle, check “Blogging Birnbaum” and “Likelihood Links”. If you think P-values are hard to explain, see how […]

Read more »

Even the tiniest error messages can indicate an invalid statistical analysis

November 25, 2015
By
Even the tiniest error messages can indicate an invalid statistical analysis

The other day, I was reading in a data set in R, and the function indicated that there was a warning about a parsing error on one line. I went ahead with the analysis anyway, but that small parsing error kept bothering me. I thought it was just one lin...

Read more »

Extracting elements from a matrix: rows, columns, submatrices, and indices

November 25, 2015
By
Extracting elements from a matrix: rows, columns, submatrices, and indices

A matrix is a convenient way to store an array of numbers. However, often you need to extract certain elements from a matrix. The SAS/IML language aupports two ways to extract elements: by using subscripts or by using indices. Use subscripts when you are extracting a rectangular portion of a […] The post Extracting elements from a matrix: rows, columns, submatrices, and indices appeared first on The DO Loop.

Read more »

a programming bug with weird consequences

November 24, 2015
By
a programming bug with weird consequences

One student of mine coded by mistake an independent Metropolis-Hastings algorithm with too small a variance in the proposal when compared with the target variance. Here is the R code of this implementation: It produces outputs of the following shape which is quite amazing because of the small variance. The reason for the lengthy freezes […]

Read more »

Internet use and religion, part four

November 24, 2015
By
Internet use and religion, part four

[If you are jumping into the middle of this series, you might want to start with this article, which explains the methodological approach I am taking.]In the previous article, I presented preliminary results from a study of relationships between I...

Read more »


Subscribe

Email:

  Subscribe