Statistics – StatsBlogs

My talk at the Metascience symposium Fri 6 Sep

Andrew — Wed, 04 Sep 2019 17:31:23 +0000

The meeting is at Stanford, and here’s my talk: Embracing Variation and Accepting Uncertainty: Implications for Science and Metascience The world would be pretty horrible if your attitude on immigration could be affected by a subliminal smiley face, if elections were swung by shark attacks and college football games, if how you vote depended on […]

The post My talk at the Metascience symposium Fri 6 Sep appeared first on StatsBlogs.

Advanced Data Reshaping in Python and R

John Mount — Wed, 04 Sep 2019 16:59:43 +0000

This note is a simple data wrangling example worked using both the Python data_algebra package and the R cdata package. Both of these packages make data wrangling easy through he use of coordinatized data concepts (relying heavily on Codd’s “rule of access”). The advantages of data_algebra and cdata are: The user specifies their desired transform … Continue reading Advanced Data Reshaping in Python and R

The post Advanced Data Reshaping in Python and R appeared first on StatsBlogs.

Quiet mode

John — Wed, 04 Sep 2019 15:43:34 +0000

When you start a programming language like Python or R from the command line, you get a lot of initial text that you probably don’t read. For example, you might see something like this when you start Python. Python 2.7.6 (default, Nov 23 2017, 15:49:48) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" […]

The post Quiet mode appeared first on StatsBlogs.

More golf putting, leading to a discussion of how prior information can be important for an out-of-sample prediction or causal inference problem, even if it’s not needed to fit existing data

Andrew — Wed, 04 Sep 2019 13:33:07 +0000

Steve Stigler writes: I saw a piece on your blog about putting. It suggests to me that you do not play golf, or you would not think this was a model – length is much more important than you indicate. I attach an old piece by a friend that is indeed the work of a […]

The post More golf putting, leading to a discussion of how prior information can be important for an out-of-sample prediction or causal inference problem, even if it’s not needed to fit existing data appeared first on StatsBlogs.

More bc weirdness

John — Wed, 04 Sep 2019 12:41:18 +0000

As I mentioned in a footnote to my previous post, I just discovered that variable names in the bc programming language cannot contain capital letters. I think I understand why: Capital letters are reserved for hexadecimal constants, though in a weird sort of way. At first variable names in bc could only be one letter […]

The post More bc weirdness appeared first on StatsBlogs.

Asimov’s question about π

John — Wed, 04 Sep 2019 12:20:51 +0000

In 1977, Isaac Asimov [1] asked how many terms of the slowly converging series π = 4 – 4/3 + 4/5 – 4/7 + 4/9 – … would you have to sum before doing better than the approximation π ≈ 355/113. A couple years later Richard Johnsonbaugh [2] answered Asimov’s question in the course of […]

The post Asimov’s question about π appeared first on StatsBlogs.

Le Monde puzzle [#1109]

xi'an — Tue, 03 Sep 2019 22:19:10 +0000

A digital problem as Le Monde current mathematical puzzle: Noble numbers are such that they only involve different digits and are multiple of all their digits. What is the largest noble number? Hmmmm…. Brute force? Since the maximal number of digits is 10, one may as well try: k=soz=9 for (t in 1:1e3){ sol=1 while […]

The post Le Monde puzzle [#1109] appeared first on StatsBlogs.

National Drug Code (NDC)

John — Tue, 03 Sep 2019 19:53:29 +0000

The US Food and Drug Administration tracks drugs using an identifer called the NDC or National Drug Code. It is described as a 10-digit code, but it may be more helpful to think of it as a 12-character code. An NDC contains 10 digits, separated into three segments by two dashes. The three segments are […]

The post National Drug Code (NDC) appeared first on StatsBlogs.

100ième victime

xi'an — Tue, 03 Sep 2019 18:19:15 +0000

The post 100ième victime appeared first on StatsBlogs.

The methods playroom: Mondays 11-12:30

Andrew — Tue, 03 Sep 2019 17:13:29 +0000

Each Monday 11-12:30 in the Lindsay Rogers room (707 International Affairs Bldg, Columbia University): The Methods Playroom is a place for us to work and discuss research problems in social science methods and statistics. Students and others can feel free to come to the playroom and work on their own projects, with the understanding that […]

The post The methods playroom: Mondays 11-12:30 appeared first on StatsBlogs.

“There is no way to prove that [an extreme weather event] either was, or was not, affected by global warming.”

Phil — Tue, 03 Sep 2019 16:08:55 +0000

This post is by Phil, not Andrew. It’s hurricane season, which means it’s time to see the routine disclaimer that no single weather event can be attributed to global warming. There’s a sense in which that is true, and a sense in which it is very wrong. I’ll start by going way back to 2005. […]

The post “There is no way to prove that [an extreme weather event] either was, or was not, affected by global warming.” appeared first on StatsBlogs.

The Wife

Andrew — Tue, 03 Sep 2019 13:57:54 +0000

I was on the plane a few months ago and watched on that tiny screen some movies, the best of which was The Wife, starring Glenn Close. It wasn’t a great movie, but it was OK, the acting was good, and my main thought was: This seems like a much better story for a book […]

The post The Wife appeared first on StatsBlogs.

Prefix code examples

John — Tue, 03 Sep 2019 11:56:38 +0000

In many offices, you can dial a three digit number to reach someone else in the office. In such offices, you usually have to dial 9 to to reach an outside number. There’s no ambiguity because no one can have an extension that begins with 9. After you’ve entered three digits, the phone system knows […]

The post Prefix code examples appeared first on StatsBlogs.

an independent sampler that maximizes the acceptance rate of the MH algorithm

xi'an — Mon, 02 Sep 2019 22:19:16 +0000

An ICLR 2019 paper by Neklyudov, Egorov and Vetrov on an optimal choice of the proposal in an independent Metropolis algorithm I discovered via an X validated question. Namely whether or not the expected Metropolis-Hastings acceptance ratio is always one (which it is not when the support of the proposal is restricted). The paper mentions […]

The post an independent sampler that maximizes the acceptance rate of the MH algorithm appeared first on StatsBlogs.

He says it again, but more vividly.

Andrew — Mon, 02 Sep 2019 13:41:59 +0000

We’ve discussed Clarke’s third law (“Any sufficiently crappy research is indistinguishable from fraud”) and that, to do good science, honesty and transparency are not enough. James Heathers says it again, vividly. I don’t know if Heathers has ever written anything about the notorious study in which participants were invited to stick 51 pins into a […]

The post He says it again, but more vividly. appeared first on StatsBlogs.

Number of possible Unicode characters

John — Mon, 02 Sep 2019 12:36:38 +0000

How many? The previous post showed how the number of Unicode characters has grown over time. You’ll notice there was a big jump between versions 3.0 and 3.1. That will be important later on. Unicode started out relative small then became much more ambitious. Are they going to run out of room? How many possible […]

The post Number of possible Unicode characters appeared first on StatsBlogs.

Growth of Unicode over time

John — Mon, 02 Sep 2019 02:12:27 +0000

My previous post quoted Randall Munroe saying Unicode “started out just trying to unify a couple different character sets” and grew much more ambitious. The first version of Unicode, published in 1991, had 7,191 characters. Now the latest version has 137,994 characters and so is about 19 times bigger. Here’s a plot of the number […]

The post Growth of Unicode over time appeared first on StatsBlogs.

New Getting Started with vtreat Documentation

John Mount — Mon, 02 Sep 2019 00:20:00 +0000

Win Vector LLC‘s Dr. Nina Zumel has just released some new vtreat documentation. vtreat is a an all-in one step data preparation system that helps defend your machine learning algorithms from: Missing values Large cardinality categorical variables Novel levels from categorical variables I hoped she could get the Python vtreat documentation up to parity with … Continue reading New Getting Started with vtreat Documentation

The post New Getting Started with vtreat Documentation appeared first on StatsBlogs.

off to SimStat2019, Salzburg

xi'an — Sun, 01 Sep 2019 22:19:04 +0000

Today, I am off to Salzburg for the SimStat 2019 workshop, or more formally the 10th International Workshop on Simulation and Statistics, where I give a talk on ABC. The program of the workshop is quite diverse and rich and so I do not think I will have time to take advantage of the Hohe […]

The post off to SimStat2019, Salzburg appeared first on StatsBlogs.

Bayesian post-selection inference

Andrew — Sun, 01 Sep 2019 13:52:06 +0000

Richard Artner, Francis Tuerlinckx, and Wolf Vanpaemel write: We are currently researching along the lines of model selection/averaging/misspecification and post-selection inference. As far as we understand your approach to Bayesian statistical analysis looks (drastically simplified) like this: 1. A series of models is sequentially fitted (with an increase in model complexity) whereby the types of […]

The post Bayesian post-selection inference appeared first on StatsBlogs.