In our article How robust is logistic regression? we pointed out some basic yet deep limitations of the traditional full-step Newton-Raphson or Iteratively Reweighted Least Squares methods of solving logistic regression problems (such as in R‘s standard glm() implementation). In fact in the comments we exhibit a well posed data fitting problem that can not [...] Related posts: How robust is logistic regression? The equivalence of logistic regression and maximum…
Logistic Regression is a popular and effective technique for modeling categorical outcomes as a function of both continuous and categorical variables. The question is: how robust is it? Or: how robust are the common implementations? (note: we are using robust in a more standard English sense of performs well for all inputs, not in the [...] Related posts: The equivalence of logistic regression and maximum entropy models Learn Logistic Regression…
What does a generalized linear model do? R supplies a modeling function called glm() that fits generalized linear models (abbreviated as GLMs). A natural question is what does it do and what problem is it solving for you? We work some examples and place generalized linear models in context with other techniques.For predicting a categorical [...] Related posts: How robust is logistic regression? Modeling Trick: Impact Coding of Categorical Variables…
Dr. Nina Zumel recently published an excellent tutorial on a modeling technique she called impact coding. It is a pragmatic machine learning technique that has helped with more than one client project. Impact coding is a bridge from Naive Bayes (where each variable’s impact is added without regard to the known effects of any other [...] Related posts: Modeling Trick: Impact Coding of Categorical Variables with Many Levels The equivalence…
Related posts: Modeling Trick: Impact Coding of Categorical Variables with Many Levels Modeling Trick: the Signed Pseudo Logarithm Learn Logistic Regression (and beyond)
Related posts: A Personal Perspective on Machine Learning The differing perspectives of statistics and machine learning Kernel Methods and Support Vector Machines de-Mystified
Related posts: A Personal Perspective on Machine Learning Why you can not to use statistics to dispute magic Statistics to English Translation, Part 2b: Calculating Significance
Much of the data that the analyst uses exhibits extraordinary range. For example: incomes, company sizes, popularity of books and any “winner takes all process”; (see: Living in A Lognormal World). Tukey recommended the logarithm as an important “stabilizing transform” (a transform that brings data into a more usable form prior to generating exploratory statistics, [...] Related posts: Modeling Trick: Masked Variables Modeling Trick: Impact Coding of Categorical Variables with…
Introduction To implement many numeric simulations you need a sophisticated source of instances of random variables. The question is: how do you generate them? The literature is full of algorithms requiring random samples as inputs or drivers (conditional random fields, Bayesian network models, particle filters and so on). The literature is also full of competing [...] Related posts: What is a large enough random sample? Kernel Methods and Support Vector…
