## Hastie-Tibshirani Statistical Learning Course Now Open

January 29, 2014
By

Machine learning is hot, hot, hot. I can't imagine better instructors (or scholars) in the area than H&T (great videos), and the course is also a fine way to learn R. It's happening now (started just last week) and runs through late March. Just go ...

## Stupid R Tricks: Random Scope

January 29, 2014
By

Andrew and I have been discussing how we’re going to define functions in Stan for defining systems of differential equations; see our evolving ode design doc; comments welcome, of course. About Scope I mentioned to Andrew I would prefer pure lexical, static scoping, as found in languages like C++ and Java. If you’re not familiar […]The post Stupid R Tricks: Random Scope appeared first on Statistical Modeling, Causal Inference, and…

## Not teaching computing and statistics in our public schools will make upward mobility even harder

January 29, 2014
By

In his book Average Is Over, Tyler Cowen predicts that as automatization becomes more common, modern economies will eventually be composed of two groups: 1) a highly educated minority involved in the production of  automated services and 2) a vast majority … Continue reading →

## “Questioning The Lancet, PLOS, And Other Surveys On Iraqi Deaths, An Interview With Univ. of London Professor Michael Spagat”

January 29, 2014
By

Mike Spagat points to this interview, which, he writes, covers themes that are discussed on the blog such as wrong ideas that don’t die, peer review and the statistics of conflict deaths. I agree. It’s good stuff. Here are some of the things that Spagat says (he’s being interviewed by Joel Wing): In fact, the […]The post “Questioning The Lancet, PLOS, And Other Surveys On Iraqi Deaths, An Interview With…

## Sample with replacement in SAS

January 29, 2014
By

Randomly choosing a subset of elements is a fundamental operation in statistics and probability. Simple random sampling with replacement is used in bootstrap methods (where the technique is called resampling), permutation tests and simulation. Last week I showed how to use the SAMPLE function in SAS/IML software to sample with [...]

## Data mining with R course in the Netherlands taught by Luis Torgo

January 29, 2014
By

In the course of this year, Dr. Luis Torgo will teach a Data Mining with R course together with the DIKW Academy in Nieuwegein, The Netherlands. Dr. Torgo is an Associate Professor at the department of Computer Science at the… See more ›

## What do I do? How do I apply statistics in my job? How did I get started?

January 29, 2014
By

I've been invited to a panel discussion by the UCLA undergraduate statistics club. Some of the questions I was told to expect are down below. By answering the questions here, there's a chance of a more literate answer and other students will be able to...

## Applied Statistics Lesson of the Day – Blocking and the Randomized Complete Blocked Design (RCBD)

A completely randomized design works well for a homogeneous population - one that does not have major differences between any sub-populations.  However, what if a population is heterogeneous? Consider an example that commonly occurs in medical studies.  An experiment seeks to determine the effectiveness of a drug on curing a disease, and 100 patients are recruited […]

## The Mirrored Line Chart Is A Bad Idea

January 29, 2014
By

The mirrored line chart is a pet peeve of mine. It’s very common close to elections when there are two parties or candidates: one’s gains are at the other’s expense. But it becomes even more egregious when there are two categories that have to sum up to 100% by their very definition. In her coverage […]

## BOSTON COLLOQUIUM FOR PHILOSOPHY OF SCIENCE: Revisiting the Foundations of Statistics

January 29, 2014
By

BOSTON COLLOQUIUM FOR PHILOSOPHY OF SCIENCE 2013–2014 54th Annual Program Download the 54th Annual Program REVISITING THE FOUNDATIONS OF STATISTICS IN THE ERA OF BIG DATA: SCALING UP TO MEET THE CHALLENGE Cosponsored by the Department of Mathematics & Statistics at Boston University. Friday, February 21, 2014 10 a.m. – 5:30 p.m. Photonics Center, 9th Floor Colloquium […]

## Inference for AR(p) Time Series

January 29, 2014
By
$Y_t =\varphi_1 Y_{t-1}+\varphi_2 Y_{t-2}+\varepsilon_t$

Consider a (stationary) autoregressive process, say of order 2, for some white noise with variance . Here is a code to generate such a process, > phi1=.25 > phi2=.7 > n=1000 > set.seed(1) > e=rnorm(n) > Z=rep(0,n) > for(t in 3:n) Z[t]=phi1*Z[t-1]+phi2*Z[t-2]+e[t] > Z=Z[800:1000] > n=length(Z) > plot(Z,type="l") Here, we have to estimate two sets of parameters: the autoregressive coefficients, and the variance of the innovation process . Several techniques…

## cut, baby, cut!

January 28, 2014
By

At MCMSki IV, I attended (and chaired) a session where Martyn Plummer presented some developments on cut models. As I was not sure I had gotten the idea [although this happened to be one of those few sessions where the flu had not yet completely taken over!] and as I wanted to check about a […]

## Time series data in R

January 28, 2014
By

There is no shortage of time series data available on the web for use in student projects, or self-learning, or to test out new forecasting algorithms. It is now relatively easy to access these data sets directly in R. M Competition data The 1001 series from the M-competition and the 3003 series from the M3-competition are available as part of the Mcomp package in R. DataMarket and Quandl Both DataMarket…

## Binomial testing with buttered toast

January 28, 2014
By

Rasmus' post of last week on binomial testing made me think about p-values and testing again. In my head I was tossing coins, thinking about gender diversity and toast. The toast and tossing a buttered toast in particular was the most helpful thought experiment, as I didn't have a fixed opinion on the probabilities for a toast to land on either side. I have yet to carry out some real…

## Context Matters When Modeling Human Judgment and Choice

January 28, 2014
By

Herbert Simon was succinct when he argued that judgment and choice "is shaped by a scissor whose two blades are the structure of the task environment and the computational capabilities of the actor" (Simon, 1990, p.7). As a marketing researcher, I take...

## Bias of Hill Estimators

January 28, 2014
By
$\overline{F}(x)=C x^{-\alpha}$

In the MAT8595 course, we’ve seen yesterday Hill estimator of the tail index. To be more specific, we did see see that if , with , then Hill estimators for are given by for . Then we did say that satisfies some consistency in the sense that if , but not too fast, i.e. (under additional assumptions on the rate of convergence, it is possible to prove that ). Further,…

## Announcing the Release of swirl 2.0

January 28, 2014
By

Editor's note: This post was written by Nick Carchedi, a Master's degree student in the Department of Biostatistics at Johns Hopkins. He is working with us to develop the Data Science Specialization as well as software for interactive learning of … Continue reading →

## Defining Properly MA(∞) Time Series

January 28, 2014
By
$MA(\infty)$

In order to properly define series, we need to get back on some properties of infinite sequences, as briefly mentioned yesterday in the MAT8181 course. Consider some sequence . The sequence is said to be summable if is convergent, i.e. if the limit of  exists when . From Cauchy criterion,  converges if and only if for each , there is  for which when . The sequence  is said to be…

## History is too important to be left to the history professors

January 28, 2014
By

From Thomas Laqueur, the Helen Fawcett professor of history at the University of California, reviewing a book by Christopher Clark: [As of 6 July 1914, the German] army made no plans for a general war; the kaiser believed the war would be localized. . . . A last small chance at least to contain a […]The post History is too important to be left to the history professors appeared first…

## Marie Curie says stop hating on quilt plots already.

January 28, 2014
By

"There are sadistic scientists who hurry to hunt down error instead of establishing the truth." -Marie Curie (http://en.wikiquote.org/wiki/Marie_Curie) Thanks to Kasper H. for that quote. I think it is a perfect fit for today's culture of academic put down as … Continue reading →

## Video Tutorial: Breaking Down the Definition of the Hazard Function

$Video Tutorial: Breaking Down the Definition of the Hazard Function$

The hazard function is a fundamental quantity in survival analysis.  For an event occurring at some time on a continuous time scale, the hazard function, , for that event is defined as , where is the time, is the time of the occurrence of the event. However, what does this actually mean?  In this Youtube […]

## Using Last.fm to data mine my music listening history

January 28, 2014
By

I've (passively) been keeping meticulous records of almost every song I've listened to since January of 2008. Since I opened my last.fm account 6 years ago, they've accumulated a massive detailed dataset of the 107,222 songs I've listened to since then. The best thing is that they're willing to share this data with me! I »more

## Online collaborative writing

January 28, 2014
By

Everyone who has written a paper with another author will know it can be tricky making sure you don’t end up with two versions that need to be merged. The good news is that the days of sending updated drafts by email backwards and forwards is finally over (having lasted all of 25 years — I can barely imagine writing papers before email). LaTeX solutions There has been a lot…