I came across this post by blogger Echidne slamming psychology professor Roy Baumeister. I’d first heard about the Baumeister in the context of his seeming inability to handle scientific criticism. I hadn’t realized that Baumeister had a sideline in pseudoscientific anti-political-correctness.

One aspect of all this that interests me is the way that Baumeister, and other scholars like him, seem to take some of the worst of the traditional left and right. From the 60’s-style left, you get a kind of mystical attitude that reality isn’t important, a mind-over-matter perspective exemplified by his view that “flair” and “intuition” are more important than boring number crunching. From the right (or, I guess we say now, the “alt-right”), there’s science-style justifications of traditional sex roles, racial inequality, and a general feeling that rich people deserve to keep what they have.

Or, to move to political science, the claims that elections are decided by shark attacks and college football games can be given a leftist spin—We’re not really living in a democracy! Voters are being manipulated!—or can be taken to support rightist positions: If voters are really so easily distracted, maybe the scope of democracy should be restricted to people such as business owners who have a real stake in the system.

In discussing the straddling of left/right themes of business school professor and plagiarist Karl Weick, here’s what Thomas Basbøll and I had to say:

Unmoored to its original source, the story gets altered by the tellers so that it can be used to make any point that people want to make from it.

We conclude with some comments on political ideology. Storytelling has been championed by a wide range of scholars who would like to escape the confines of rigor. On the academic left, storytelling is sometimes viewed as a humane alternative to the impersonal number crunching of economists, while the academic right uses stories to connect to worldly business executives who have neither the time nor patience for dry scholasticism. Karl Weick seems to us to express an unstable mix of these attitudes, championing the creative humanism of story-based social reasoning while offering his theories as useful truths for the business world. And indeed he may be correct in both these views . . .

Maybe it’s nothing special, it’s just the usual story that people will use the tropes available for them. After all, why should someone have to be on the political left to feel like a 60’s-style rebel? Cultural commentators ranging from P. J. O’Rourke (on the right) to Thomas Frank (on the left) have been talking about this sort of thing for decades.

So I’m not sure where this leads.

The post Pseudoscience and the left/right whiplash appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post Pseudoscience and the left/right whiplash appeared first on All About Statistics.

Our article "Let’s Have Some Sympathy For The Part-time R User" includes two points:

• Sometimes you have to write parameterized or re-usable code.
• The methods for doing this should be easy and legible.

The first point feels abstract, until you find yourself wanting to re-use code on new projects. As for the second point: I feel the wrapr package is the easiest, safest, most consistent, and most legible way to achieve maintainable code re-use in R.

In this article we will show how wrapr makes code-rewriting even easier with its new let x=x automation.

Let X=X

There are very important reasons to choose a package that makes things easier. One is debugging:

Everyone knows that debugging is twice as hard as writing a program in the first place. So if you’re as clever as you can be when you write it, how will you ever debug it?

Brian Kernighan, The Elements of Programming Style, 2nd edition, chapter 2

Let’s take the monster example from "Let’s Have Some Sympathy For The Part-time R User".

The idea was that perhaps one had worked out a complicated (but useful and important) by-hand survey scoring method:

suppressPackageStartupMessages(library("dplyr"))
library("wrapr")

d <- data.frame(
  subjectID = c(1,                   
                1,
                2,                   
                2),
  surveyCategory = c(
    'withdrawal behavior',
    'positive re-framing',
    'withdrawal behavior',
    'positive re-framing'
  ),
  assessmentTotal = c(5,                 
                      2,
                      3,                  
                      4),
  stringsAsFactors = FALSE
)

scale <- 0.237

d %>%
  group_by(subjectID) %>%
  mutate(probability =
           exp(assessmentTotal * scale)/
           sum(exp(assessmentTotal * scale))) %>%
  arrange(probability, surveyCategory) %>%
  mutate(isDiagnosis = row_number() == n()) %>%
  filter(isDiagnosis) %>%
  ungroup() %>%
  select(subjectID, surveyCategory, probability) %>%
  rename(diagnosis = surveyCategory) %>%
  arrange(subjectID)

## # A tibble: 2 x 3
##   subjectID           diagnosis probability
##       <dbl>               <chr>       <dbl>
## 1         1 withdrawal behavior   0.6706221
## 2         2 positive re-framing   0.5589742

The presumption is that the above pipeline is considered reasonable (but long, complicated, and valuable) dplyr, and our goal is to re-use it on new data that may not have the same column names as our original data.

We are making the huge simplifying assumption that you have studied the article and the above example is now familiar.

The question is: what to do when one wants to process the same type of data with different column names? For example:

d <- data.frame(
  PID = c(1,                   
          1,
          2,                   
          2),
  DIAG = c(
    'withdrawal behavior',
    'positive re-framing',
    'withdrawal behavior',
    'positive re-framing'
  ),
  AT = c(5,                 
         2,
         3,                  
         4),
  stringsAsFactors = FALSE
)

print(d)

##   PID                DIAG AT
## 1   1 withdrawal behavior  5
## 2   1 positive re-framing  2
## 3   2 withdrawal behavior  3
## 4   2 positive re-framing  4

The new table has the following new column definitions:

subjectID       <- "PID"
surveyCategory  <- "DIAG"
assessmentTotal <- "AT"
isDiagnosis     <- "isD"
probability     <- "prob"
diagnosis       <- "label"

We could "reduce to a previously solved problem" by renaming the columns to names we know, doing the work, and then renaming back (which is actually a service that replyr::replyr_apply_f_mapped() supplies).

In "Let’s Have Some Sympathy For The Part-time R User" I advised editing the pipeline to have obvious stand-in names (perhaps in all-capitals) and then using wrapr::let() to perform symbol substitution on the pipeline.

Dr. Nina Zumel has since pointed out to me: if you truly trust the substitution method you can use the original column names and adapt the original calculation pipeline as is (without alteration). Let’s try that:

let(
  c(subjectID = subjectID,
    surveyCategory = surveyCategory, 
    assessmentTotal = assessmentTotal,
    isDiagnosis = isDiagnosis,
    probability = probability,
    diagnosis = diagnosis),
  d %>%
    group_by(subjectID) %>%
    mutate(probability =
             exp(assessmentTotal * scale)/
             sum(exp(assessmentTotal * scale))) %>%
    arrange(probability, surveyCategory) %>%
    mutate(isDiagnosis = row_number() == n()) %>%
    filter(isDiagnosis) %>%
    ungroup() %>%
    select(subjectID, surveyCategory, probability) %>%
    rename(diagnosis = surveyCategory) %>%
    arrange(subjectID))

## # A tibble: 2 x 3
##     PID               label      prob
##   <dbl>               <chr>     <dbl>
## 1     1 withdrawal behavior 0.6706221
## 2     2 positive re-framing 0.5589742

That works! All we did was: paste the original code into the block and the adapter did all of the work, with no user edits of the code.

It is a bit harder for the user to find which symbols are being replaced, but in some sense they don’t really need to know (it is R‘s job to perform the replacements).

wrapr has a new helper function mapsyms() that automates all of the "let x = x" steps from the above example.

mapsyms() is a simple function that captures variable names and builds a mapping from them to the names they refer to in the current environment. For example we can use it to quickly build the assignment map for the let block, because the earlier assignments such as "subjectID <- "PID"" allow mapsyms() to find the intended re-mappings. This would also be true for other cases, such as re-mapping function arguments to values. Our example becomes:

print(mapsyms(subjectID,
              surveyCategory, 
              assessmentTotal,
              isDiagnosis,
              probability,
              diagnosis))

## $subjectID
## [1] "PID"
## 
## $surveyCategory
## [1] "DIAG"
## 
## $assessmentTotal
## [1] "AT"
## 
## $isDiagnosis
## [1] "isD"
## 
## $probability
## [1] "prob"
## 
## $diagnosis
## [1] "label"

This allows the solution to be re-written and even wrapped into a function in a very legible form with very little effort:

computeRes <- function(d,
                       subjectID,
                       surveyCategory, 
                       assessmentTotal,
                       isDiagnosis,
                       probability,
                       diagnosis) {
  let(
    mapsyms(subjectID,
            surveyCategory, 
            assessmentTotal,
            isDiagnosis,
            probability,
            diagnosis),
    d %>%
      group_by(subjectID) %>%
      mutate(probability =
               exp(assessmentTotal * scale)/
               sum(exp(assessmentTotal * scale))) %>%
      arrange(probability, surveyCategory) %>%
      mutate(isDiagnosis = row_number() == n()) %>%
      filter(isDiagnosis) %>%
      ungroup() %>%
      select(subjectID, surveyCategory, probability) %>%
      rename(diagnosis = surveyCategory) %>%
      arrange(subjectID)
  )
}

computeRes(d,
           subjectID       = "PID",
           surveyCategory  = "DIAG",
           assessmentTotal = "AT",
           isDiagnosis     = "isD",
           probability     = "prob",
           diagnosis       = "label")

## # A tibble: 2 x 3
##     PID               label      prob
##   <dbl>               <chr>     <dbl>
## 1     1 withdrawal behavior 0.6706221
## 2     2 positive re-framing 0.5589742

The idea is: instead of having to mark what instances of symbols are to be replaced (by quoting or de-quoting indicators), we instead declare what symbols are to be replaced using the mapsyms() helper.

mapsyms() is a stand-alone helper function (just as ":=" is). It works not because it is some exceptional corner-case hard-wired into other functions, but because mapsyms()‘s reasonable semantics happen to synergize with let()‘s reasonable semantics. mapsyms() behaves as a replacement target controller (without needing any cumbersome direct quoting or un-quoting notation!).

Please comment on the article here: Statistics – Win-Vector Blog

The post Let X=X in R appeared first on All About Statistics.

StanCon is happening at the beautiful Asilomar conference facility at the beach in Monterey California for three days starting January 10, 2018. We have space for 200 souls and this will sell out.

If you don’t already know, Stan is the rising star of probabilistic modeling with Bayesian analysis. If you do statistics, machine learning or data science then you need to know about Stan.

StanCon offers a full schedule of invited talks, submitted papers, and tutorials unavailable in any other format. Balancing the intellectual intensity of cutting edge statistical modeling are fun activities like indoor R/C airplane building/flying/designing and non-snobby blind wine tasting for after dinner activities. We will have the first ever “wear your poster” reception–see the call for posters below. And no parallel sessions–you get the entire StanCon2018, not a slice.

Go to http://mc-stan.org/events/stancon2018 and register.

Invited Talks

• Andrew Gelman
  Department of Statistics and Political Science, Columbia University
• Susan Holmes
  Department of Statistics, Stanford University
• Frank Harrell, Jr.
  School of Medicine and Department of Biostatistics, Vanderbilt University
• Sophia Rabe-Hesketh
  Educational Statistics and Biostatistics, University of California, Berkeley
• Sean Taylor and Ben Letham
  Facebook Core Data Science
• Manuel Rivas
  Department of Biomedical Data Science, Stanford University
• Talia Weiss
  Department of Physics, Massachusetts Institute of Technology

These rock stars have agreed to leave their entourages, groupies and bad habits at home and will start their shows talks on time and leave you wanting more.

Submitted talks:

We have 18 accepted talks ranging from public policy viewed through Bayesian analysis to painful theory papers. And we have Facebook, and space people from NASA. Talks are self-contained knitr or Jupyter notebooks that will be made publicly available after the conference.

Tutorials

We have tutorials that start at the crack of 8am for those desiring further edification beyond the awesome program. Total time ranges from 6 hours to 1 hour depending on topic—these will be parallel but don’t conflict with the main conference.

• Introduction to Stan
  Know how to program? Know basic statistics? Curious about Bayesian analysis and Stan? This is the course for you. Hands on, focused and an excellent way to get started working in Stan. 2 hours every morning 8am to 10am.
• Executive decision making the Bayesian way
  This is for nontechnical managers to learn the core of decision making under uncertainty and how to interpret the talks that they will be attending the rest of the day. 1 hour/day every day.
• Advanced Modeling in Stan
  The hard stuff led by the best of the best. Very interactive, very intense. Varying topics, every day 1-2 hours.

Poster call for participation

We will take poster submissions on a rolling basis until December 5th. One page exclusive of references is the desired format but anything that gives us enough information to make a decision is fine. We will accept/reject within 48 hours. Send to stancon2018@mc-stan.org.

The only somewhat odd requirement is that your poster must be “wearable” to the 5pm reception where you will be a walking presentation. Great way to network, signboard supplies will be available so you need only have sheets of paper which can be attached to signboard material which coincidentally will be the source airframe material for the R/C airplane activities following dinner.

Fun Stuff

Learning is fun but we anticipate that blowing off a little steam will be called for.

• R/C Airplanes
  After dinner on day 1 we will provide designs and building materials to create your own R/C airplane. The core design can be scratch built in 90 minutes or less at which point, and weather dependent, we will learn to fly our planes indoors or outdoors. See http://brooklynaerodrome.com for an idea of the style of airplane. You can also create your own designs and we will have night illumination gear.
• Snob-free Blind Wine Tasting
  By day 2 you will have gotten to know your fellow attendees so some social adventure is called for. This activity has proved wildly successful at DARPA conferences and they invented the internet so it can’t be all bad. Participants taste wines without knowing what they are.
  That’s it! StanCon2018 is going to be a pressure cooker of learning and fun. Don’t miss it.

Early registration

Early bird registration ends 10 November 2017.

Go to http://mc-stan.org/events/stancon2018 and register.

StanCon Organizing Committee

The post StanCon2018 Early Registration ends Nov 10 appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post StanCon2018 Early Registration ends Nov 10 appeared first on All About Statistics.

We had some discussion yesterday about this Gallup poll that asked respondents to guess the percentage of Americans who are gay. The average response was 23%—and this stunningly high number was not just driven by outliers: more than half the respondents estimated the proportion gay as 20% or more.

All this is in stark contrast to direct estimates from surveys that 3 or 4% of Americans are gay.

One thing that came up in comments is that survey respondents with minority sexual orientations might not want to admit it. So maybe that 3-4% is an underestimate. Here’s an informative news article by Samantha Allen which suggests that the traditionally-cited 10% number might not be so far off.

But even if the real rate is 10% (including lots of closeted people), that’s still much less than the 23% from the survey.

Or, to look at it another way, it’s no surprise that respondents grossly overestimate the rate of gays, given that they also grossly overestimate the rate of African Americans, Hispanic Americans, Asian Americans, and immigrants in this and other countries. Or similar overestimates if you ask people what fraction of the Federal budget goes to various well-known but relatively tiny programs.

From this perspective, the problem doesn’t seem to have anything to do with gays but rather a more general lack of numeracy, a lack of understanding about small proportions.

Suppose we wanted to ask the survey question in a way to elicit more accurate responses. One way would be to break up the majority category into subgroups. For example, what percentage of American adults are: Straight and married; Straight, unmarried, in a committed relationship; Straight, never married, and single; Straight and divorced; Straight and widowed; Gay. I’m sure these categories could be phrased better, also you have to figure out what to do with various intermediate categories. The point is that it could make a difference if you set up enough alternative categories.

Similarly for the ethnicities. What if, instead of asking black/Hispanic/Asian/white/other, you ask something like this: black, Hispanic, Asian, American Indian, English, Irish, German, Italian, Polish, Russian, etc.? I don’t know how this would go, but I could see it making a difference.

On the other hand, you shouldn’t have to do it that way to get a reasonable answer.

Another way would be to take a more direct approach: Ask the survey respondent about his or her 10 or 20 closest family members and friends: How many are gay, etc.? (You’d want them to identify the 10 or 20 people first, before saying why you’re asking, otherwise it would be too easy to recall one or more gay person.) Then you could ask about the rest of the population: Do you think you know many more gay people, more gay people, about the same, fewer gay people, or many fewer gay people, compared to the average American.

The point here is not to trick respondents into giving more accurate answers but rather to connect population questions of interest to respondents’ direct experience.

I wonder what Gerd Gigerenzer thinks about all this.

P.S. David Landy writes:

I saw your post today on overestimations of LGBT individuals, and its supposed relationship to innumeracy. Funnily enough, I’ve been meaning to write you on that specific topic (which you’ve posted on before)! I think your interpretation, which is the common one, is either wrong or simply too vague: My colleagues and I published a paper on this topic this year, and have one more in the works*. The short version is that we argue that these overestimations are a direct result of individual-level psychophysical response curves: they are a part of how people estimate proportions of any kind, and have little or nothing to do with people’s perceptions of particular subgroups.

* Here’s a dropbox link to a PDF version of the individual-level data and analyses, that confirm the pattern of responses within single individuals, for a wider variety of demographic items, including LGBT populations. It’s a lot of slides (but just a 20 minute talk)—but the most important/convincing are the data and models on pages 58-62).

Second *: Also in a rank ordering task we conducted, Mechanical Turkers actually seem to underestimate the LGBT population, specifically. This is important because rank ordering distinguishes between general psychophysical biases and specific population-specific misconceptions. Again, we’re prepping for publication, but here’s the key figure:

The post More thoughts on that “What percent of Americans would you say are gay or lesbian?” survey appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post More thoughts on that “What percent of Americans would you say are gay or lesbian?” survey appeared first on All About Statistics.

“And then there was Yodeling Elaine, the Queen of the Air. She had a dollar sign medallion about as big as a dinner plate around her neck and a tiny bubble of spittle around her nostril and a little rusty tear, for she had lassoed and lost another tipsy sailor“– Tom Waits

It turns out I turned thirty two and became unbearable. Some of you may feel, with an increasing sense of temporal dissonance, that I was already unbearable. (Fair point) Others will wonder how I can look so good at my age. (Answer: Black Metal) None of that matters to me because all I want to do is talk about the evils of marketing like the 90s were a vaguely good idea. (Narrator: “They were not. The concept of authenticity is just another way for the dominant culture to suppress more interesting ones.”)

The thing is, I worry that the real problem in academic statistics in 2017 is not a reproducibility crisis, so much as that so many of our methods just don’t work. And to be honest, I don’t really know what to do about that, other than suggest that we tighten our standards and insist that people proposing new methods, models, and algorithms work harder to sketch out the boundaries of their creations. (What a suggestion. Really. Concrete proposals for concrete change. But it’s a blog. If ever there was a medium to be half-arsed in it’s this one. It’s like twitter for people who aren’t pithy.)

Berätta för mig om det är sant att din hud är doppad i honung

So what is the object of my impotent ire today. Well nothing less storied than the Bayesian Lasso.

It should be the least controversial thing in this, the year of our lord two thousand and seventeen, to point out that this method bears no practical resemblance to the Lasso. Or, in the words of Law and Order: SVU, “The [Bayesian Lasso] is fictional and does not depict any actual person or event”.

Who do you think you are?

The Bayesian Lasso is a good example of what’s commonly known as the Lupita Nyong’o fallacy, which goes something like this: Lupita Nyong’o had a break out role in Twelve Years a Slave, she also had a heavily disguised role in one of the Star Wars films (the specific Star Wars film is not important. I haven’t seen it and I don’t care). Hence Twelve Years a Slave exists in the extended Star Wars universe.

The key point is that the (classical) Lasso plays a small part within the Bayesian Lasso (it’s the MAP estimate) in the same way that Lupita Nyong’o played a small role in that Star Wars film. But just as the presence of Ms Nyong’o does not turn Star Wars into Twelve Years a Slave, the fact that the classical Lasso can be recovered as the MAP estimate of the Bayesian Lasso does not make the Bayesian Lasso useful.

And yet people still ask if they can be fit in Stan. In that case, Andrew answered the question that was asked, which is typically the best way to deal with software enquiries. (It’s usually a fool’s game to try to guess why people are asking particular questions. It probably wouldn’t be hard for someone to catalogue the number of times I’ve not followed my advice on this, but in life as in statistics, consistency is really only a concern if everything else is going well.) But I am brave and was not asked for my opinion, so I’m going to talk about why the Bayesian Lasso doesn’t work.

Hiding all away

So why would anyone not know that the Bayesian Lasso doesn’t work?  Well, I don’t really know. But I will point out that all of the results that I’ve seen in this directions (not that I’ve been looking hard) have been published in the prestigious but obtuse places like Annals of Statistics, the journal we publish in when we either don’t want people without a graduate degree in mathematical statistics to understand us or when we want to get tenure.

By contrast, the original paper is very readable and published in JASA, where we put papers when we are ok with people who do not have a graduate degree in mathematical statistics being able to read them, or when we want to get tenure.

To be fair to Park and Casella, they never really say that the Baysian Lasso should be used for sparsity. Except for one sentence in the introduction where they say the median gives approximately sparse estimators and the title which links it to the most prominent and popular method for estimating a sparse signal. Marketing eh. (See, I’m Canadian now).

The devil has designed my death and is waiting to be sure

So what is the Bayesian LASSO (and why did I spend 600 words harping on about something before defining it? The answer will shock you. Actually the answer will not shock you, it’s because it’s kinda hard to do equations on this thing.)

For data observed with Gaussian error, the Bayesian Lasso takes the form

where, instead of putting a Normal prior on we put independent Laplace priors

Here the tuning parameter where is the number of covariates, is the number of observations, is the number of “true” non-zero elements of , is known, and is an unknown scaling parameter that should be .

Important Side note: This isn’t the exact same model as Park and Castella used as they didn’t use the transformation

but rather just dealt with as the parameter. From a Bayesian viewpoint, it’s a much better idea to put a prior on than on directly. Why? Because a prior on needs to depend on n, p, , and X and hence needs to be changed for each problem, while a prior on can be used for many problems.  One possible option is , which is a rate optimal parameter for the (non-Bayesian) Lasso. Later, we’ll do a back-of-the-envelope calculation that suggests we probably don’t need the square root  around the logarithmic term.

Edit: I’ve had some questions about this scaling, so I’m going to try to explain it a little better.  The idea here is that the Bayesian Lasso uses the i.i.d. Laplace priors with scaling parameter on to express the substantive belief that the signal is approximately sparse.  The reason for scaling the prior is that not every value of is consistent with this belief.  For example, will not give an approximately sparse signal.  While we could just use a  prior for that has a very heavy right tail (something like an inverse gamma), this is at odds with a good practice principle of making sure all of thee parameters in your models are properly scaled to make them order 1.  Why do we do this? Because it makes it much much easier to set sensible priors.

Other important side note: Some of you may have noticed that the scaling  can depend on the unknown sparsity .  This seems like cheating. People who do asymptotic theory call this sort of value for an oracle value, mainly because people studying Bayesian asymptotics are really really into databasing software. The idea is that this is the value of that gives the model the best chance of working. When maths-ing, you work out the properties of the posterior with the oracle value of and then you use some sort of smoothness argument to show that the actual method that is being used to select (or average over) the parameter gives almost the same answer.

Only once in Sheboygan. Only once.

So what’s wrong with the Bayesian Lasso? Well the short version is that the Laplace prior doesn’t have enough mass near zero relative to the mass in the tails to allow for a posterior that has a lot of entries that are almost zero and some entries that are emphatically not zero.  Because the Bayesian Lasso prior does not have a spike at zero, none of the entries will be a priori  exactly zero, so we need some sort of rule to separate the “zero” entries from the “non-zero” entries.  The way that we’re going to do this is to choose a cutoff where we assume that if , then .

So how do we know that the Lasso prior doesn’t put enough mass in important parts of the parameter space? Well there are two ways. I learnt it during the exciting process of  writing a paper that the reviewers insisted should have  an extended section about sparsity (although this was at best tangential to the rest of the paper), so I suddenly needed to know about Bayesian models of sparsity. So I read those Annals of Stats papers. (That’s why I know I should be scaling !).

What are the key references? Well all the knowledge that you seek is here and here.

But a much easier way to work out that the Bayesian Lasso is bad is to do some simple maths. Because the are a priori independent, we get a prior on the effective sparsity 

For the Bayesian Lasso, that probability can be computed as

.

Ideally, the distribution of this effective sparsity would be centred on the true sparsity.  That is, we’d like to choose so that

.

A quick re-arrangement suggests that

.

Now, we are interested in signals with , i.e. where only a very small number of the are non-zero.  This suggests we can safely ignore the second term.  Some other theory  suggests that we need to take to depend on in such a way that it goes to zero faster than as gets large. (Edit: Oops – this paper used and .  For the general case, their reasoning leads to .)

This means that we need to take in order to ensure that we have our prior centred on sparse vectors (in the sense that the prior mean for the number of non-zero components is always much less than ).

Show some emotion

So for the Bayesian Lasso, a sensible parameter is , which will usually have a large number of components less than the threshold and a small number that are larger.  But this is still not any good.  To see this, let’s consider the prior probability of seeing a larger than one:

.

This is the problem with the Bayesian Lasso: in order to have a lot of zeros in the signal, you are also forcing the non-zero elements to be very small. A plot of this function is above, and it’s clear that even for very small values of this probability is infinitesimally small.

Basically, the Bayesian Lasso can’t give enough mass to both small and large signals simultaneously.  Other Bayesian models (such as the horseshoe and the Finnish horseshoe) can support both simultaneously and this type of calculation can show that (although it’s harder. See Theorem 6 here).

Side note: The scaling that I derived in the previous section is a little different to the standard Lasso scaling of , but the same result holds: for large the probability of seeing a large signal is vanishingly small.

Maybe I was mean, but I really don’t think so

Now, obviously this is not what we see with real data when we fit the Bayesian Lasso in Stan with an unknown .  What happens is the model tries to strike a balance between the big signals and the small signals, shrinking the former too much, and letting the latter be too far from zero.  You will see this in the event that you try to fit the Bayesian Lasso in Stan.

Update: I’ve put some extra notes in the above text to make it hopefully a little clearer in one case and a little more correct in the other. Please accept this version of the title track as way of an apology

(Also this stunning version, done wearing even more makeup)

The post The king must die appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post The king must die appeared first on All About Statistics.

[cat picture]

For the chapter in Regression and Other Stories that includes nonlinear regression, I’d like a couple homework problems where the kids have to construct and fit models to real data. So I need some examples. We already have the success of golf putts as a function of distance from the hole, and I’d like some others. One thing that came to mind today, because I happened to see a safety warning poster on the bus reminding people not to drive too fast, is data on speed and traffic accidents.

But I’m interested in other examples too. Just about anything interesting with data on x and y where there’s no simple linear relation, and where log and logit transformations don’t do the trick either. The outcome can be discrete or continuous, either way.

There’s gotta be lots and lots of good examples but for some reason I’m drawing a blank. So I could use your help. I need not just the examples but also the data.

Thanks! Anyone who comes up with a good suggestion will be rewarded with a free Stan sticker.

The post Looking for data on speed and traffic accidents—and other examples of data that can be fit by nonlinear models appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post Looking for data on speed and traffic accidents—and other examples of data that can be fit by nonlinear models appeared first on All About Statistics.

Karl Broman was surprised that x[[c(5, 3)]] in R was the same as x[[5]][[3]]. And it was in 2013. It reminded me of a similar thing that surprised me a couple of months ago. I have used R for more than 10 years, and for the first time I had discovered that for a matrix x and a two-column integer index matrix i (each row of i is a pair of row and column indices to index x), x[i] returns the vector c(x[i[1, 1], i[1, 2]], x[i[2, 1], i[2, 2]], ..., x[i[n, 1], i[n, 2]]). Actually I had been searching for this way of indexing for years, and I didn’t know it was this simple! In the past, I was using an awkward loop like:

n = nrow(i)
y = numeric(n)
for (k in seq_len(n)) {
  y[k] = x[i[k, 1], i[k, 2]]
}
y

That was both ugly and inefficient. After someone asked me this question again a couple of months ago, I came up with a slightly more elegant solution but perhaps even more inefficient:

diag(x[i[, 1], i[, 2]])

The perfect answer is x[i], both elegant and efficient. I learned this from another person’s answer to the same question. Then I looked at the help page ?`[`, and realized it was well-documented:

When indexing arrays by [ a single argument i can be a matrix with as many columns as there are dimensions of x; the result is then a vector with elements corresponding to the sets of indices in each row of i.

The example I gave above was just a special case: x could be an array of more than two dimensions, and does not have to be a matrix (a matrix is a special case of an array).

R help pages have too much hidden power. You have to carefully unfold and read them.

Please comment on the article here: Yihui's Blog on Yihui Xie | 谢益辉

The post x[matrix(ncol = 2)] appeared first on All About Statistics.

The last and final Le Monde puzzle is a bit of a disappointment, to wit:

A 4×4 table is filled with positive and different integers. A 3×3 table is then deduced by adding four adjacent [i.e. sharing a common corner] entries of the original table. Similarly with a 2×2 table, summing up to a unique integer. What is the minimal value of this integer? And by how much does it increase if all 29 integers in the tables are different?

For the first question, the resulting integer writes down as the sum of the corner values, plus 3 times the sum of the side values, plus 9 times the sum of the 4 inner values [of the 4×4 table]. Hence, minimising the overall sum means taking the inner values as 1,2,3,4, the side values as 5,…,12, and the corner values as 13,…,16. Resulting in a total sum of 352. As checked in this computer code in APL by Jean-Louis:

This configuration does not produce 29 distinct values, but moving one value higher in one corner does: I experimented with different upper bounds on the numbers and 17 always provided with the smallest overall sum, 365.

firz=matrix(0,3,3)#second level
thirz=matrix(0,2,2)#third level
for (t in 1:1e8){
flor=matrix(sample(1:17,16),4,4)
for (i in 1:3) for (j in 1:3)
firz[i,j]=sum(flor[i:(i+1),j:(j+1)])
for (i in 1:2) for (j in 1:2)
thirz[i,j]=sum(firz[i:(i+1),j:(j+1)])
#last
if (length(unique(c(flor,firz,thirz)))==29)
solz=min(solz,sum(thirz))}

and a further simulated annealing attempt did not get me anywhere close to this solution.

Please comment on the article here: R – Xi'an's Og

The post Le Monde [last] puzzle [#1026] appeared first on All About Statistics.

Blake McShane sent along this paper by himself and David Gal, which begins:

In light of recent concerns about reproducibility and replicability, the ASA issued a Statement on Statistical Significance and p-values aimed at those who are not primarily statisticians. While the ASA Statement notes that statistical significance and p-values are “commonly misused and misinterpreted,” it does not discuss and document broader implications of these errors for the interpretation of evidence. In this article, we review research on how applied researchers who are not primarily statisticians misuse and misinterpret p-values in practice and how this can lead to errors in the interpretation of evidence. We also present new data showing, perhaps surprisingly, that researchers who are primarily statisticians are also prone to misuse and misinterpret p-values thus resulting in similar errors. In particular, we show that statisticians tend to interpret evidence dichotomously based on whether or not a p-value crosses the conventional 0.05 threshold for statistical significance. We discuss implications and offer recommendations.

The article is published in the Journal of the American Statistical Association along with discussions:

A p-Value to Die For, by Don Berry

The Substitute for p-Values, by William Briggs

Some Natural Solutions to the p-Value Communication Problem—and Why They Won’t Work, by Andrew Gelman and John Carlin

Statistical Significance and the Dichotomization of Evidence: The Relevance of the ASA Statement on Statistical Significance and p-Values for Statisticians, by Eric Laber and Kerby Shedden

and a rejoinder by McShane and Gal.

Good stuff. Read the whole thing.

For some earlier blog discussions of the McShane and Gal paper and related work, see here (More evidence that even top researchers routinely misinterpret p-values), here (Some natural solutions to the p-value communication problem—and why they won’t work), here (When considering proposals for redefining or abandoning statistical significance, remember that their effects on science will only be indirect!), and here (Abandon statistical significance).

The post Statistical Significance and the Dichotomization of Evidence (McShane and Gal’s paper, with discussions by Berry, Briggs, Gelman and Carlin, and Laber and Shedden) appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post Statistical Significance and the Dichotomization of Evidence (McShane and Gal’s paper, with discussions by Berry, Briggs, Gelman and Carlin, and Laber and Shedden) appeared first on All About Statistics.

In my [Keith] previous post that criticised a publish paper, the first author commented they wanted some time to respond and I agreed. I also suggested that if the response came in after most readers have moved on I would re-post their response as a new post pointing back to the previous. So here we are.

Now there has been a lot of discussion on this blog about public versus private criticism and their cost and benefit trade offs. One change I am making is to refer to first, second or third author rather than names. Here I should also clarify that I have previously worked with the first and second author (so they are not strangers) and that the first author posted the paper on my blog post (they brought it to my attention).  Now my three main points in that original post were – 1. failing to distinguish what something is versus what to make of it, 2. ignoring the ensemble of similar studies (completed, ongoing and future) and 3. neglecting important non-random errors.  So when the first author brought the paper to my attention, I thought is was going to be an example of not neglecting those three things. But when I read the paper I felt it pretty much did neglect 1 and 3.

One of the main points in the response by all three authors was clarification of the goal of the paper (all other responses were just by the first author). They claimed the goal was to simply clarify that the fixed effects estimate is an estimate of some population’s average though not necessary one that would be of interest (depending on the context). Quoting the response “Given the didactic goal of the paper, the issue is not so much whether such a population is of interest, but just the realization that the analysis is informing us about such a population.” I fully agree with that (with a caveat below). In my experience, getting an adeqaute sense of that population and generalising from it to a population of interest – is a very big stretch. The first author responded that his experience is different and especially given the epidemiologists he works with, it is often doable. Fair enough – experiences and research team expertise differ. Now my reading of the paper was that it suggested more than just that clarification and gave the impression fixed effects should be used much more and often it was more scientifically relevant. But those are just my interpretations and that can vary as does our apparent views on what is meant by scientifically relevant if not also a population. So I was expecting the paper not to fail to distinguish what something is versus what to make of it, but apparently the authors never intended to.

One of my points the first author chose not to respond to was my review of Rubin’s conceptual ideas. Again fair enough. However that is where I believe there is a serious technical disagreement. This became clearer in the first author’s response e.g. “[Keith referring to] varying study quality… [first author] This is a misconception. The paper goes to considerable lengths to allow for underlying effects to differ”. That is the caveat I referred to above in that fixed effects estimate is an estimate of some populations average only if the between study variation is not importantly driven by design (AKA study quality or methodological) variation. This kind of variation is usually/mostly the result of haphazard biases and has different implications for what is to be made of the variation and expectation. Briefly, the variation needs to be included in the uncertainty quantification and the expectation is no longer of direct interest (more below).  So while science (AKA clinical or biological) driven variation allows it to be excluded in the uncertainty quantification and the expectation is of direct interest for the average of some population. These are very different.

Fisher was one of the first to bring attention to this issue (that is why I gave the reference), the Rubin references discusses it and it is even in the Cochrane Handbook (section 9.5.1) which was edited by the second author (I believe I wrote the first draft for this section and  the second author revised it). There also was full discussion of this issue at an RSS meeting in 2002. A simple example may make the issue clear. If a fixed object is measured with three measuring instruments with differing (haphazard) biases, there will be variation in the measurements that is not from the object and the average won’t be estimating the object but the object plus average bias (whatever that is). With three fixed objects each measured with the same unbiased measuring instrument there will be variation in the measurements that is “real” but the average of the three objects is fixed. Here, the average measurement will be estimating the average of the three objects and the measurement variation (above the  instrument’s variance) need not be included in the uncertainty of the estimated average. And this would be true for any population involving various proportions of the three objects – fixed population average that the properly weighted the average would estimate it. Now one could argue the expectation with varying bias is of indirect interest being the populations true average plus some weighted combination of the biases. But then one should clearly warn of the presence of such biases even in the absence of how to address those biases.  So here the paper is neglecting important non-random errors. The authors, if they agree, may wish to consider adding a note to their paper about this issue.

One of my other points the first author choose not to respond to was to work through the likelihood mechanics involved.  Again fair enough. But they referred to a paper by Danyu Lin and co-author and claimed that it was pivotal and that suggested it may not be as straightforward as I thought. When I read the paper I saw exactly the likelihood mechanics being worked as I expected but there was something I had not seen before. The paper worked through the properties of a mis-specified model which is what a fixed effects model implemented with the full raw data actually is. You consider the effects to vary by study but in the model implemented with all the raw data you purposely (mis)specify the parameter as being exactly the same (give it the exact same symbol) for all studies. Now when I first started doing meta-analysis in the 1980,s the outcomes were usually binary. I had just taken a generalised linear models course, so I implemented meta-analysis using logistic regression with a common odds parameter and differing control rate parameters by study. To formally test for heterogeneity (that we explicitly argued should not be depended upon)  the common odds parameter would be replaced with differing odds ratio parameters by study. (See Model 4: Partial Pooling (Log Odds) in Bob’s extensive tutorial for a full Bayesian approach to this.) But I knew with binary outcomes whether I code the data by individual patient (0 or 1) versus number of failures and successes – the answers would be exactly the same (given the same parameter specifications and the magic of sufficiency)  so this summary data versus raw data meta-analysis quest (given you had both) seemed completely moot to me.

So I learned something. The fixed effects model implemented with the raw data actually is mis-specified – that is wrong – but it estimates the correct average for some population (depends on sufficiency and other technicalities – but it often does). So that must have been puzzling to many and that needed to be sorted out. No doubt there is also asymptotic things that needed to be sorted out but Charlie Geyer has convinced me that such considerations are not a good use of my time.

The post What I missed on fixed effects (plural). appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post What I missed on fixed effects (plural). appeared first on All About Statistics.