I recently gave a talk at the R in Finance conference in which I introduced the marketAgent package for R. Here is the source for the package if you’d like to play with it: marketAgent_0.000.tar I’ll be giving more details of the talk real ...

I recently gave a talk at the R in Finance conference in which I introduced the marketAgent package for R. Here is the source for the package if you’d like to play with it: marketAgent_0.000.tar I’ll be giving more details of the talk real ...

Norman Ornstein and Alan Abramowitz warn against over-interpreting poll fluctuations: In this highly charged election, it’s no surprise that the news media see every poll like an addict sees a new fix. That is especially true of polls that show large and unexpected changes. Those polls get intense coverage and analysis, adding to their presumed […] The post “Stop the Polling Insanity” appeared first on Statistical Modeling, Causal Inference, and…

Just following up on our post the other day on retrospective evaluations of probabilistic predictions: For more on Leicester City, see Nick Goff on Why did bookmakers lose on Leicester? and What price SHOULD Leicester have been? (forwarded to me by commenter Iggy). For more on Trump, see Nate Silver on How I Acted Like […] The post Nick and Nate and Mark on Leicester and Trump appeared first on…

At R/Finance 2016: Applied Finance with R. Interesting group, with many constituencies, and interesting program, which appears below (or go to http://www.rinfinance.com/agenda/).Friday, May 20th, 201608:00 - 09:00Optional Pre-...

Before leaving Helsinki, we arXived [from the Air France lounge!] the paper Jean-Michel presented on Monday at ABCruise in Helsinki. This paper summarises the experiments Louis conducted over the past months to assess the great performances of a random forest regression approach to ABC parameter inference. Thus validating in this experimental sense the use of […]

Jonathan Sterne sent me this opinion piece by Stephan Lewandowsky and Dorothy Bishop, two psychology researchers who express concern that the movement for science and data transparency has been abused. It would be easy for me to dismiss them and take a hard-line pro-transparency position—and I do take a hard-line pro-transparency position—but, as they remind […] The post Will transparency damage science? appeared first on Statistical Modeling, Causal Inference, and…

I’m not the best one to write about this: to the extent that there’s bias in favor of men, I’ve been a beneficiary. Also I’m not familiar with the research on the topic. I know there are some statistical difficulties in setting up these causal questions, comparable to the difficulties arising in using “hedonic regression” […] The post Bias against women in academia appeared first on Statistical Modeling, Causal Inference,…

As I was checking for software to answer a query on X validated about generic Bayes factor derivation, I came across an R software called BayesFactor, which only applies in regression settings and relies on the Savage-Dickey representation of the Bayes factor when the null hypothesis writes as θ=θ⁰ (and possibly additional nuisance parameters with […]

In any discussion of ethics , someone is bound to bring up the law: it is okay to do something if something is not against the law. The logical conclusion of this belief is that ethics is irrelevant in the presence of the law. The recent stories relating to minimum wage and the overtime pay rule changes (for example, link) argue strongly against such a point of view. The law…

John Tukey coined many terms that have passed into common use, such as bit (a shortening of binary digit) and software. Other terms he coined are well known within their niche: boxplot, ANOVA, rootogram, etc. Some of his terms, such as jackknife and vacuum cleaner, were not new words per se but common words he […]

The OpenVis Conference had another great selection of talks this year. Here is a list of my favorites, with talk videos and pointers to some additional materials. The conference venue was the IMAX theater of the Boston Aquarium, which was great. Not only was the huge screen and theater seating a definite plus, but the ticket also included … Continue reading OpenVis 2016 Talk Videos and Pointers

Carl Bialik and Andrew Flowers at fivethirtyeight.com (Nate Silver’s site) ran a story following up on our birthdays example—that time series decomposition of births by day, which is on the cover of the third edition of Bayesian Data Analysis using data from 1968-1988, and which then Aki redid using a new dataset from 2000-2014. Friday […] The post Birthday analysis—Friday the 13th update, and some model checking appeared first on…

I was eleven years old when I first saw Newton's method. No, I didn't go to a school for geniuses. I didn't even know it was Newton's method until decades later. However, in sixth grade I learned an iterative algorithm that taught me (almost) everything I need to know about […] The post All I really need to know about Newton's method I learned in primary school appeared first on…

Editor’s note: This post is a chapter from the book Executive Data Science: A Guide to Training and Managing the Best Data Scientists, written by myself, Brian Caffo, and Jeff Leek. Software is the generalization of a specific aspect of a data analy...

tl;dr: If you value your time, don’t read this post. In favor of the fraac=Adams hypothesis: 1. Fraac came to our attention with a burst of comments on my 2011 post on Scott Adams and Charlie Sheen. Here’s fraac, defending Adams in a very Adams-like way: Besides the fact that fraac is one of the […] The post Is fraac Scott Adams? appeared first on Statistical Modeling, Causal Inference, and…

In this note, we discuss principal components regression and some of the issues with it: The need for scaling. The need for pruning. The lack of “y-awareness” of the standard dimensionality reduction step. The purpose of this article is to set the stage for presenting dimensionality reduction techniques appropriate for predictive modeling, such as y-aware … Continue reading Principal Components Regression, Pt.1: The Standard Method

Another update has been made to Sharp-R bringing it to version 1.2. The main changes are, Multiple function files. Built in standard functions. Changes to the XML function files. Bug fixes. We have increased the number of function files that can loaded...