Statistics Blogs
In this article we will discuss composing standard-evaluation interfaces (SE) and composing non-standard-evaluation interfaces (NSE) in R. In R the package tidyeval/rlang is a tool for building domain specific languages intended to allow easier composition of NSE interfaces. To use it you must know some of its structure and notation. Here are some details paraphrased … Continue reading Non-Standard Evaluation and Function Composition in R
Here is an absolutely horrible way to confuse yourself and get an inflated reported R-squared on a simple linear regression model in R. We have written about this before, but we found a new twist on the problem (interactions with categorical variable encoding) which we would like to call out here. First let’s set up … Continue reading An easy way to accidentally inflate reported R-squared in linear regression models
This note describes a useful replyr tool we call a "join controller" (and is part of our "R and Big Data" series, please see here for the introduction, and here for one our big data courses). When working on real world predictive modeling tasks in production, the ability to join data and document how you … Continue reading Use a Join Controller to Document Your Work
The third and (possibly) final round of the workshops of our introductory workshops was overbooked in April, but we have managed to arrange some additional dates in June.There are still places left on these. More details at: http://www.p...
Maths trauma and earthquakes Trauma is a deeply distressing or disturbing experience. Many people in my home town of Christchurch still suffer from post traumatic stress disorder (PTSD) as a result of our earthquakes five or so years ago. I … Continue reading →
So: for once I woke up this morning feeling slightly quite tired for the late night, but also rather upbeat after an election. The final results of the general election are out and have produced quite some shock. Throughout yesterday, it look...
In our latest “R and big data” article we show how to manage intermediate results in non-trivial Apache Spark workflows using R, sparklyr, dplyr, and replyr. Handle management Many Sparklyr tasks involve creation of intermediate or temporary tables. This can be through dplyr::copy_to() and through dplyr::compute(). These handles can represent a reference leak and eat … Continue reading Managing intermediate results when using R/sparklyr
Our free video course Campaign Response Testing is no longer published on Udemy. It remains available for free on YouTube with all source code available from GitHub. I’ll try to correct bad links as I find them. Please read on for the reasons. Udemy recently unilaterally instituted a new policy on free courses: “When a … Continue reading Campaign Response Testing no longer published on Udemy
Recall the tall-wide-dense (T, K, m) Big Data taxonomy. One might naively assert that tall data (big time dimension, T) are not really a part of the Big Data phenomenon, insofar as T has not started growing more quickly in recent years. But...
If you don't follow Marc Bellemare's blog, you should do.And if you read only one other blog post this week, it should be this one from Marc, titled, "How to Publish in Academic Journals". Read his slides that are linked in the post.Great advice that i...
