Heaps’ law says that the number of unique words in a text of n words is approximated by V(n) = K nβ where K is a positive constant and β is between 0 and 1. According to the Wikipedia article on Heaps’ law, K is often between 10 and 100 and β is often between 0.4 […]
Category: Statistics
So You Want to Start a Podcast
Podcasting has gotten quite a bit easier over the past 10 years, due in part to improvements to hardware and software. I wrote about both how I edit and record both of my podcasts about 2 years ago and, while not much has changed since then, I though…
sunrise on Tenjikuyama [jatp]
Introducing data_algebra
This article introduces the data_algebra project: a data processing tool family available in R and Python. These tools are designed to transform data either in-memory or on remote databases. In particular we will discuss the Python implementation (also called data_algebra) and its relation to the mature R implementations (rquery and rqdatatable). Introduction Parts of the … Continue reading Introducing data_algebra
More on the piranha problem, the butterfly effect, unintended consequences, and the push-a-button, take-a-pill model of science
The other day we had some interesting discussion that I’d like to share. I started by contrasting the butterfly effect—the idea that a small, seemingly trivial, intervention at place A can potentially have a large, unpredictable effect at place B—with the “PNAS” or “Psychological Science” view of the world, in which small, seemingly trivial, intervention […]
the three Kumano shrines [jatp]
Forming a hyper-precise numerical summary during a research crisis can improve an article’s chance of achieving its publication goals.
Speaking of measurement and numeracy . . . Kevin Lewis pointed me to this published article with the following abstract that starts out just fine but kinda spirals out of control: Forming a military coalition during an international crisis can improve a state’s chances of achieving its political goals. We argue that the involvement of […]
The importance of talking about the importance of measurement: It depends on the subfield
An interesting point came up in a comment thread the other day and you might have missed it, so I’ll repeat it here. Dan Goldstein wrote to me: Many times I’ve heard you say people should improve the quality of their measurements. Have you considered that people may be quite close to the best quality […]
the (forty-)seven samurai (赤穂浪士)
During my vacations in Japan, I read the massive (1096p) book by Osaragi Jiro on the Akō incident, with occidental title the 47 rōnins. Which I had bought in Paris before leaving. This is a romancized version of an historical event that took part in 1701 in the Genroku era. Where 47 rōnin (leaderless samurai) […]
Inferring a continuous distribution from binned data
I show how modelling the distribution of an underlying continuous variable that has been clouded by binning is much better way of understanding the data than crude methods dealing directly with the binned counts.
More on why Cass Sunstein should be thanking, not smearing, people who ask for replications
Recently we discussed law professor and policy intellectual Cass Sunstein’s statement that people who ask for social science findings to be replicated are like the former East German secret police. In that discussion I alluded to a few issues: 1. The replication movement is fueled in large part by high-profile work, lauded by Sunstein and […]
bye and thank you for all the fish!
Yes, you can include prior information on quantities of interest, not just on parameters in your model
Nick Kavanagh writes: I studied economics in college and never heard more than a passing reference to Bayesian stats. I started to encounter Bayesian concepts in the workplace and decided to teach myself on the side. I was hoping to get your advice on a problem that I recently encountered. It has to do with […]
Mickey Mouse, Batman, and conformal mapping
A conformal map between two regions in the plane preserves angles [1]. If two curves meet at a given angle in the domain, their images will meet at the same angle in the range. Two subsets of the plane are conformally equivalent if there is a conformal map between them. The Riemann mapping theorem says […]
Eliminating Tail Calls in Python Using Exceptions
I was working through Kyle Miller‘s excellent note: “Tail call recursion in Python”, and decided to experiment with variations of the techniques. The idea is: one may want to eliminate use of the Python language call-stack in the case of a “tail calls” (a function call where the result is not used by the calling … Continue reading Eliminating Tail Calls in Python Using Exceptions
Hyppocratic oath for maths?
On a free day in Nachi-Taksuura, I came across this call for a professional oath for mathematicians (and computer engineers and scientists in related fields) . By UCL mathematician Hannah Fry. The theme is the same as with Weapons of math destruction, namely that algorithms have a potentially huge impact on everyone’s life and that […]
Multilevel structured (regression) and post-stratification
My enemies are all too familiar. They’re the ones who used to call me friend – Jawbreaker Well I am back from Australia where I gave a whole pile of talks and drank more coffee than is probably a good idea. So I’m pretty jetlagged and I’m supposed to be writing my tenure packet, so […]
Stancon is happening now.
Hi, everyone!
Star-crossed lovers
A story in The New Yorker quotes the following explanation from Arthur Eddington regarding the speed of light. Suppose that you are in love with a lady on Neptune and that she returns the sentiment. It will be some consolation for the melancholy separation if you can say to yourself at some—possibly prearranged—moment, “She is […]
Coney Island
Inspired by this story (“Good news! Researchers respond to a correction by acknowledging it and not trying to dodge its implications”): Coming down from Psych Science Stopping off at PNAS Out all day datagathering And the craic was good Stopped off at the old lab Early in the morning Drove through Harvard taking pictures And […]