Sir David Cox

David Cox sent me a letter relating to my post of Oct.5, 2013. He has his own theory as to who might have been doing the teasing! I’m posting it  here, with his permission: 

Dear Deborah

I was interested to see the correspondence about Jeffreys and the possible teasing by Neyman’s associate. It brought a number of things to mind.

1. While I am not at all convinced that any teasing was involved, if there was it seems to me much more likely that Jeffreys was doing the teasing. He, correctly surely, disapproved of that definition and was putting up a highly contrived illustration of its misuse.
2. In his work he was not writing about a subjective view of probability but about objective degree of belief. He did not disapprove of more physical definitions, such as needed to describe radioactive decay; he preferred to call them chances.
3.  In assessing his work it is important that the part on probability was perhaps 10% of what he did. He was most famous for The earth (1924) which is said to have started the field of geophysics. (The first edition of his 1939 book on probability was in a series of monographs in physics.) The later book with his wife, Bertha,  Methods of mathematical physics is a masterpiece.
4. I heard him speak from time to time and met him personally on a couple of occasions. He was superficially very mild and said very little. He was involved in various controversies but, and I am not sure about this, I don’t think they ever degenerated into personal bitterness. He lived to be 98 and, a mark of his determination is that in his early 90’s he cycled in Cambridge having a series of minor accidents. He was stopped only when Bertha removed the tires from his bike. Bertha was a highly respected teacher of mathematics.
5.  He and R.A.Fisher were not only towering figures in statistics in the first part of the 20^th century but surely among the major applied mathematicians of that era in the world.
6. Neyman was not at all Germanic, in the sense that one of your correspondents described. He could certainly be autocratic but not in personal manner. While all the others at Berkeley were Professor this or Dr that, he insisted on being called Mr Neyman.
7. The remarks [i] about how people addressed one another 50 plus years ago in the UK are  broadly accurate, although they were not specific to Cambridge and certainly could be varied. From about age 11 boys in school, students and men in the workplace addressed one another by surname only. Given names were for family and very close friends. Women did use given names or  were Miss or Mrs, certainly never Madam unless they were French aristocrats. Thus in 1950 or so I worked with, published with and was very friendly with two physical scientists, R.C. Palmer and S.L. Anderson. I have no idea what their given names were; it was irrelevant. To address someone you did not know by name you used Sir or Madam. It would be very foolish to think that meant unfriendliness or that the current practice of calling absolutely everyone by their given name means universal benevolence.

Best wishes

David

D.R.Cox
Nuffield College
Oxford
UK

[i]In comments to this post.

Please comment on the article here: Error Statistics Philosophy » Statistics

The post Sir David Cox: a comment on the post, “Was Hosiasson pulling Jeffreys’ leg?” appeared first on All About Statistics.

I’ve written a few posts now about using parallel tempering to sample from complicated multi-modal target distributions but there are also other benefits and uses to this algorithm. There is a nice post on Darren Wilkinson’s blog about using tempered posteriors for marginal likelihood calculations. There is also another area where parallel tempering finds application, namely in stochastic optimization. I first encountered parallel tempering whilst doing my MPhys degree at the University of Warwick but at that time it was employed as a stochastic optimization algorithm to find the minimum energy configuration of a Lennard-Jones cluster as opposed to a sampling algorithm. All that is required is one observation to turn this sampling algorithm into a stochastic optimization algorithm. Lets break this observation down into a few steps.
Consider sampling from a simple exponential distribution $$f(E)\propto e^{-\beta E}1_{(0,\infty )}(E),$$
with rate parameter beta. For now lets fix beta=5. One could sample from this distribution using the same Rmpi parallel tempering code given in my previous post by simply changing the target distribution to the exponential above. The histograms of mcmc draws from four tempered distribution would then look something like this:

Note the scale on the x-axis. The two important observations mentioned earlier are

The second point is important because although the sampling algorithm is creating draws that are not the minimum value of E, by increasing the rate parameter one can force these draws to be arbitrarily close to E-min.

A Uni-modal Optimization Function

How does this relate to optimization? Consider setting $$E(\theta)=(\theta-40)^2$$ Whereas before where using the Metropolis algorithm one would propose a new value of E, say E’, now the proposal is made in θ, and θ’ is accepted based on u < f(E(θ')) / f(E(θ)). By construction the algorithm gives draws close to E-min, which occurs when θ=40. The traceplot of θ is shown below:

Click here for the code.

A Harder Optimization Function

The above quadratic was an easy uni-modal example. Let’s try a harder function. Consider the minimum of $$ E(\theta)=3sin(\theta)+(0.1\theta-3)^2,$$ which looks like this:

This function has infinitely many local minima but one global minimum around 30. Local minima make optimization challenging and many optimization algorithms get stuck in these regions as locally it appears the minimum has been reached. This is where the parallel tempering really helps. The traceplots of theta are shown for six tempered distributions below:

Click here for the code.

I’m currently working on another example just for fun, namely finding the lowest energy configuration of an n-particle Lennard-Jones cluster. This is a nice example because one can visualize the process using vmd and it also provides some insight into the origins of such terminology as “tempering”, “annealing” and “temperature” which always look somewhat out of place in the statistics literature.

Li Y., Protopopescu V.A., Arnold N., Zhang X. & Gorin A. (2009). Hybrid parallel tempering and simulated annealing method, Applied Mathematics and Computation, 212 (1) 216-228. DOI: 10.1016/j.amc.2009.02.023

The post Stochastic Optimization in R by Parallel Tempering appeared first on Lindons Log.

Please comment on the article here: Lindons Log » Statistics

The post Stochastic Optimization in R by Parallel Tempering appeared first on All About Statistics.

Dean Eckles writes:

Given your ongoing discussion of info viz for different goals, you might be interested in Sinan Aral’s new article:

This touches on several info viz themes:
- Viz for yourself (or your team) vs. visualizations to share the final conclusions
- Viz for identifying promising features for use in modeling
- Viz and statistical significance, especially when the data has plenty of dependence structure

Also, these cascade visualizations are perhaps worth comparing with some of very large cascades on Facebook made by my colleagues Alex Dow, Lada Adamic, and Adrien Friggeri.

I like those graphs but on the international maps I would make the country boundaries thinner and I would get rid of Greenland and Antarctica, they’re distracting.

(I think that’s what Bob would call a “bike shed” comment.)

The post Visualization, “big data”, and EDA appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

The post Visualization, “big data”, and EDA appeared first on All About Statistics.

Made on ggplot

I have a standard code for ggplot2 which I use to make line graphs, scatter plots, and histograms.

For lines or scatters:

p<- ggplot(x, aes(x=Year, y=Rank, colour=Uni, group=Uni)) #colour lines by variable Uni #group Uni labelled variables in the same line

Then: 

p + #you get an error if not for this step
geom_line(size=1.2) +
geom_point(data=QS[QS[,2]==”2013″,]) +
geom_text(data=QS[QS[,2]==”2013″&QS[,1]!=”Princeton”,],aes(label=paste(paste(Rank,”.”,sep=”"),Uni)),hjust=-0.2)+
ylim(17,0.5) +
scale_x_continuous(limit=c(2004,2014),breaks=seq(2004,2014,1)) +
theme(legend.position=”none”) +
ggtitle(“QS University Rankings 2008-2013″) +
theme(plot.title=element_text(size=rel(1.5))) +
theme_bw() +
theme(panel.grid.major=element_blank(), panel.grid.minor=element_blank()) +
geom_text(aes(label=country),size=6,vjust=-1) +
annotate(“text”,x=2011,y=16.5,label=”Abbas Keshvani”)

For a bar chart:

ggplot(Dist, aes(x=B,y=C,fill=A)) +  #stacked bars, column A contains stacks
geom_bar(stat=”identity”, width=0.9) +

Abbas Keshvani

Please comment on the article here: CoolStatsBlog

The post Using ggplot2 appeared first on All About Statistics.

I just upload on http://hal.archives-ouvertes.fr/hal-00871883 a joint paper entitled Modeling earthquake dynamics.

In this paper, we investigate questions arising in Parsons & Geist (2012). Pseudo causal models connecting magnitudes and waiting times are consider, through generalized regression. We do use conditional model (magnitude given previous waiting time, and conversely) as an extension to joint distribution model described in Nikoloulopoulos & Karlis (2008). On the one hand, we fit a Pareto distribution for earthquake magnitudes, where the tail index is a function of waiting time following previous earthquake; on the other hand, waiting times are modeled using a Gamma or a Weibull distribution, where parameters are function of the magnitude of the previous earthquake. We use those two models, alternatively, to generate the dynamics of earthquake occurrence, and to estimate the probability of occurrence of several earthquakes within a year, or a decade.

Please comment on the article here: Freakonometrics » Statistics

The post Earthquake dynamics appeared first on All About Statistics.

Betsy Sinclair, The Social Citizen: Peer Networks and Political Behavior

Introspectively, it seems pretty obvious that our political behavior and attitudes are influenced by our friends and family; this is an attempt to document and quantify that influence, in the context of modern American politics. Sinclair looks at influence on going to the polls, on donating to campaigns, on who we vote for, and on party identification.

The evidence on voter turnout is experimental: randomly encouraging one person to vote makes those in the same household, who were not experimented on, more likely to vote themselves. This has the same compelling logic as any good experiment: manipulating A makes B move, so there must be some mechanism connecting A to B.

For donations, candidate choice, and party identification, Sinclair's evidence is observational, from surveys. These studies generally do not, in fact, collect social networks, but rather rely on people reporting on attributes of their networks and their friends and family, or even proxying social networks by demographic information about neighborhoods. What Sinclair reports here is intuitively plausible, but all of it is vulnerable to confounding due to homophily. I don't have a better idea about how to address these problems in observational studies, and Sinclair and her co-authors have done about¹ as well as anyone could have with this data, but it's still only suggestive.

[1]: Assume for the sake of argument that every attribute on which people are homophilous is accurately measured by the survey and included in the regression. To support the sort of strong scientific interpretation that Sinclair (reasonably!) wants to give the regression results, we'd also have to know that the functional form of the regression was well-specified. This is can be checked, but apparently wasn't. (This lapse is no less serious for being so very common.) I also think there is a logical error in the discussion of the panel data on candidate preferences (pp. 95ff). I see no reason why homophily couldn't lead to voters selecting friends who will respond similarly to what happens during a political campaign, which would tend to make their vote choices increasingly aligned. ^

(Disclaimer: Prof. Sinclair was kind enough to invite me to a conference she organized on this subject earlier this year. I should maybe also mention that I'm happy with how this book cites my work.)

Manning Marable, Malcolm X: A Life of Reinvention

A really magnificent biography of a remarkable American, by a great scholar coming from a viewpoint close enough to be sympathetic but distant enough to be sharply critical.

Tarquin Hall, The Case of the Missing Servant: From the Files of Vish Puri, Most Private Investigator; The Case of the Man Who Died Laughing; The Case of the Deadly Butter Chicken

Mind candy: humorous murder mysteries from contemporary India, principally Delhi.

Books to Read While the Algae Grow in Your Fur; Pleasures of Detection, Portraits of Crime; The Beloved Republic; Networks; Commit a Social Science

Please comment on the article here: Three-Toed Sloth

The post Books to Read While the Algae Grow in Your Fur, September 2013 appeared first on All About Statistics.

Have some delayed book-chat from April, May, June, July, August and September.

Please comment on the article here: Three-Toed Sloth

The post October First Is Too Late appeared first on All About Statistics.

Attention conservation notice: Only relevant if you (1) really care about statistics, and (2) will be in Pittsburgh on Monday.

Through a fortuitous concourse of calendars, we will have three outstanding talks on Monday, 14 October 2013. In chronological order:

Michael I. Jordan, "On the Computational and Statistical Interface and 'Big Data'" (special joint statistics/ML seminar)

Abstract: The rapid growth in the size and scope of datasets in science and technology has created a need for novel foundational perspectives on data analysis that blend the statistical and computational sciences. That classical perspectives from these fields are not adequate to address emerging problems in "Big Data" is apparent from their sharply divergent nature at an elementary level---in computer science, the growth of the number of data points is a source of "complexity" that must be tamed via algorithms or hardware, whereas in statistics, the growth of the number of data points is a source of "simplicity" in that inferences are generally stronger and asymptotic results can be invoked. Indeed, if data are a data analyst's principal resource, why should more data be burdensome in some sense? Shouldn't it be possible to exploit the increasing inferential strength of data at scale to keep computational complexity at bay? I present three research vignettes that pursue this theme, the first involving the deployment of resampling methods such as the bootstrap on parallel and distributed computing platforms, the second involving large-scale matrix completion, and the third introducing a methodology of "algorithmic weakening," whereby hierarchies of convex relaxations are used to control statistical risk as data accrue.

(Joint work with Venkat Chandrasekaran, Ariel Kleiner, Lester Mackey, Purna Sarkar, and Ameet Talwalkar.)

Time and place: Noon, Rangos 2, University Center

David Choi, "Testing for Coordination and Peer Influence in Network Data" (machine learning and the social sciences seminar)

Abstract: Many tests have been proposed for the detection of "viral" peer influence in observational studies involving social network data. However, these tests typically make strong (and sometimes unstated) modeling assumptions on participant behavior. We propose a test which holds under less restrictive assumptions, and which controls for unobserved homophily variables that are unaccounted for in existing methods. We discuss conditions under which the test is valid, and give preliminary results on its effectiveness.

Time and place: 3 pm in Gates Hall 4405

Genevera Allen, "Sparse and Functional Principal Components Analysis"

Abstract: Regularized principal components analysis, especially Sparse PCA and Functional PCA, has become widely used for dimension reduction in high-dimensional settings. Many examples of massive data, however, may benefit from estimating both sparse AND functional factors. These include neuroimaging data where there are discrete brain regions of activation (sparsity) but these regions tend to be smooth spatially (functional). Here, we introduce an optimization framework that can encourage both sparsity and smoothness of the row and/or column PCA factors. This framework generalizes many of the existing approaches to Sparse PCA, Functional PCA and two-way Sparse PCA and Functional PCA, as these are all special cases of our method. In particular, our method permits flexible combinations of sparsity and smoothness that lead to improvements in feature selection and signal recovery as well as more interpretable PCA factors. We demonstrate our method on simulated data and a neuroimaging example of EEG data. This work provides a unified optimization framework for regularized PCA that can form the foundation for a cohesive approach to regularization in high-dimensional multivariate analysis.

Time and place: 4 pm in Doherty Hall 1212

As always, the talks are free and open to the public.

Enigmas of Chance; Networks

Please comment on the article here: Three-Toed Sloth

The post Triple Header (Next Week at the Statistics / Machine Learning Seminars) appeared first on All About Statistics.

Lecture 10: Functions in R are objects, just like everything else, and so can be both arguments to and return values of functions, with no special machinery required. Examples from math (especially calculus) of functions with other functions as arguments. Some R syntax relating to functions. Examples with curve. Using sapply to extend functions of single numbers to functions of vectors; its combination with curve. We write functions with lower-level functions as arguments to abstract out a common pattern of operations. Example: calculating a gradient. Numerical gradients by first differences, done two different ways. (Limitations of taking derivatives by first differences.) Incorporating this as a part of a larger algorithm, such as gradient descent. Using adapters, like wrapper functions and anonymous functions, to fit different functions together. Examples from math (especially calculus) of operators, which turn one function into another. The importance of scoping when using functions as return values. Example: creating a linear predictor. Example: implementing the gradient operator (two different ways). Example: writing surface, as a two-dimensional analog to the standard curve. The use of eval and substitute to control when and in what context an expression is evaluated. Three increasingly refined versions of surface, employing eval.

R for examples.

Introduction to Statistical Computing

Please comment on the article here: Three-Toed Sloth

The post Functions as Objects (Introduction to Statistical Computing) appeared first on All About Statistics.

In which we use Tukey's rule for identifying outliers as an excuse to learn about debugging and testing.

Assignment

Introduction to Statistical Computing

Please comment on the article here: Three-Toed Sloth

The post Lab: Testing Our Way to Outliers (Introduction to Statistical Computing) appeared first on All About Statistics.