## Source for the marketAgent R package

May 22, 2016
By

I recently gave a talk at the R in Finance conference in which I introduced the marketAgent package for R. Here is the source for the package if you’d like to play with it: marketAgent_0.000.tar I’ll be giving more details of the talk real ...

## “Stop the Polling Insanity”

May 21, 2016
By

Norman Ornstein and Alan Abramowitz warn against over-interpreting poll fluctuations: In this highly charged election, it’s no surprise that the news media see every poll like an addict sees a new fix. That is especially true of polls that show large and unexpected changes. Those polls get intense coverage and analysis, adding to their presumed […] The post “Stop the Polling Insanity” appeared first on Statistical Modeling, Causal Inference, and…

## Nick and Nate and Mark on Leicester and Trump

May 20, 2016
By

Just following up on our post the other day on retrospective evaluations of probabilistic predictions: For more on Leicester City, see Nick Goff on Why did bookmakers lose on Leicester? and What price SHOULD Leicester have been? (forwarded to me by commenter Iggy). For more on Trump, see Nate Silver on How I Acted Like […] The post Nick and Nate and Mark on Leicester and Trump appeared first on…

## Simulating a Weibull conditional on time-to-event is greater than a given time

May 20, 2016
By

Recently, I had to simulate a time-to-event of subjects who have been on a study, are still ongoing at the time of a data cut, but who are still at risk of an event (e.g. progressive disease, cardiac event, death). This requires the simulation of a con...

## Hazard Functions for U.S. Expansions

May 20, 2016
By

Glenn Rudebusch has a very nice 2016 FRBSF Letter, "Will the Economic Recovery Die of Old Age?".  He draws on perspective and results from our joint work of 25 years ago (including a paper we did with Dan Sichel -- see below), and he applies them ...

## R/Finance 2016: Applied Finance with R

May 20, 2016
By

At R/Finance 2016: Applied Finance with R.  Interesting group, with many constituencies, and interesting program, which appears below (or go to http://www.rinfinance.com/agenda/).Friday, May 20th, 201608:00 - 09:00Optional Pre-...

## ABC random forests for Bayesian parameter inference

May 19, 2016
By

Before leaving Helsinki, we arXived [from the Air France lounge!] the paper Jean-Michel presented on Monday at ABCruise in Helsinki. This paper summarises the experiments Louis conducted over the past months to assess the great performances of a random forest regression approach to ABC parameter inference. Thus validating in this experimental sense the use of […]

## Will transparency damage science?

May 19, 2016
By

Jonathan Sterne sent me this opinion piece by Stephan Lewandowsky and Dorothy Bishop, two psychology researchers who express concern that the movement for science and data transparency has been abused. It would be easy for me to dismiss them and take a hard-line pro-transparency position—and I do take a hard-line pro-transparency position—but, as they remind […] The post Will transparency damage science? appeared first on Statistical Modeling, Causal Inference, and…

## Bias against women in academia

May 19, 2016
By

I’m not the best one to write about this: to the extent that there’s bias in favor of men, I’ve been a beneficiary. Also I’m not familiar with the research on the topic. I know there are some statistical difficulties in setting up these causal questions, comparable to the difficulties arising in using “hedonic regression” […] The post Bias against women in academia appeared first on Statistical Modeling, Causal Inference,…

## Using MCMC output to efficiently estimate Bayes factors

May 18, 2016
By
$Using MCMC output to efficiently estimate Bayes factors$

As I was checking for software to answer a query on X validated about generic Bayes factor derivation, I came across an R software called BayesFactor, which only applies in regression settings and relies on the Savage-Dickey representation of the Bayes factor when the null hypothesis writes as θ=θ⁰ (and possibly additional nuisance parameters with […]

## Learning to Love Bayesian Statistics

May 18, 2016
By

I did a webcast earlier today about Bayesian statistics.  Some time in the next week, the video should be available from O'Reilly.  In the meantime, you can see my slides here: And here's a transcript of what I said:Thanks everyone for joinin...

## Law versus ethics: how the new overtime pay rule will play out

May 18, 2016
By

In any discussion of ethics , someone is bound to bring up the law: it is okay to do something if something is not against the law. The logical conclusion of this belief is that ethics is irrelevant in the presence of the law. The recent stories relating to minimum wage and the overtime pay rule changes (for example, link) argue strongly against such a point of view. The law…

## Cepstrum, quefrency, and pitch

May 18, 2016
By

John Tukey coined many terms that have passed into common use, such as bit (a shortening of binary digit) and software. Other terms he coined are well known within their niche: boxplot, ANOVA, rootogram, etc. Some of his terms, such as jackknife and vacuum cleaner, were not new words per se but common words he […]

## OpenVis 2016 Talk Videos and Pointers

May 18, 2016
By

The OpenVis Conference had another great selection of talks this year. Here is a list of my favorites, with talk videos and pointers to some additional materials. The conference venue was the IMAX theater of the Boston Aquarium, which was great. Not only was the huge screen and theater seating a definite plus, but the ticket also included … Continue reading OpenVis 2016 Talk Videos and Pointers

## Birthday analysis—Friday the 13th update, and some model checking

May 18, 2016
By

Carl Bialik and Andrew Flowers at fivethirtyeight.com (Nate Silver’s site) ran a story following up on our birthdays example—that time series decomposition of births by day, which is on the cover of the third edition of Bayesian Data Analysis using data from 1968-1988, and which then Aki redid using a new dataset from 2000-2014. Friday […] The post Birthday analysis—Friday the 13th update, and some model checking appeared first on…

## All I really need to know about Newton’s method I learned in primary school

May 18, 2016
By

I was eleven years old when I first saw Newton's method. No, I didn't go to a school for geniuses. I didn't even know it was Newton's method until decades later. However, in sixth grade I learned an iterative algorithm that taught me (almost) everything I need to know about […] The post All I really need to know about Newton's method I learned in primary school appeared first on…

## What is software engineering for data science?

May 18, 2016
By

Editor’s note: This post is a chapter from the book Executive Data Science: A Guide to Training and Managing the Best Data Scientists, written by myself, Brian Caffo, and Jeff Leek. Software is the generalization of a specific aspect of a data analy...

## Does Trivers-Willard apply to people?

May 17, 2016
By

Today I am working on another "one-day paper", although this one is a bit of a cheat, since I'm a few hours past the deadline.  Nevertheless, the question of the day is whether the Trivers-Willard effect applies to people.According to Wikipedia, t...

## Is fraac Scott Adams?

May 17, 2016
By

tl;dr: If you value your time, don’t read this post. In favor of the fraac=Adams hypothesis: 1. Fraac came to our attention with a burst of comments on my 2011 post on Scott Adams and Charlie Sheen. Here’s fraac, defending Adams in a very Adams-like way: Besides the fact that fraac is one of the […] The post Is fraac Scott Adams? appeared first on Statistical Modeling, Causal Inference, and…

## Statistical Machine Learning Circa 1989

May 17, 2016
By

I've always been a massive fan of statisticians whose work is rigorous yet practical, with emphasis on modeling. People like Box, Cox, Hastie, and Tibshirani obviously come to mind.  So too, of course, do Leo Brieman and Jerry Friedman.  I ha...

## Graduate Econometrics Exam

May 17, 2016
By

Occasionally readers ask about the exams that I set in my graduate econometrics courses.The elective graduate econometrics course that I taught this past semester was one titled "Themes in Econometrics". The topics that are covered vary from year to ye...

## Principal Components Regression, Pt.1: The Standard Method

May 17, 2016
By

In this note, we discuss principal components regression and some of the issues with it: The need for scaling. The need for pruning. The lack of “y-awareness” of the standard dimensionality reduction step. The purpose of this article is to set the stage for presenting dimensionality reduction techniques appropriate for predictive modeling, such as y-aware … Continue reading Principal Components Regression, Pt.1: The Standard Method

## Sharp-R May Update

May 16, 2016
By

Another update has been made to Sharp-R bringing it to version 1.2. The main changes are, Multiple function files. Built in standard functions. Changes to the XML function files. Bug fixes. We have increased the number of function files that can loaded...