(This article was originally published at The Chemical Statistician » Statistics, and syndicated at StatsBlogs.)
After training a binary classifier, calculating its various values of sensitivity and specificity, and constructing its receiver operating characteristic (ROC) curve, we can use the ROC curve to assess the predictive accuracy of the classifier. A minimum standard for a good ROC curve is being better than the line of no discrimination. On a plot of 
on the vertical axis and 
on the horizontal axis, the line of no discrimination is the line that passes through the points 
and 
. In other words, the line of discrimination is the diagonal line that runs from the bottom left to the top right. This line shows the performance of a binary classifier that predicts the class of the target variable purely by the outcome of a Bernoulli random variable with 0.5 as its probability of attaining the “Success” category. Such a classifier does not use any of the predictors to make the prediction; instead, its predictions are based entirely on random guessing, with the probabilities of predicting the “Success” class and the “Failure” class being equal.
If we did not have any predictors, then we can rely on only random guessing, and a random variable with the distribution 
is the best that we can use for such guessing. If we do have predictors, then we aim to develop a model (i.e. the binary classifier) that uses the information from the predictors to make predictions that are better than random guessing. Thus, a minimum standard of a binary classifier is having an ROC curve that is higher than the line of no discrimination. (By “higher“, I mean that, for a given value of , the of the binary classifier is higher than the of the line of no discrimination.)
Filed under:
Applied Statistics,
Machine Learning,
Machine Learning Lesson of the Day,
Predictive Modelling,
Statistics Tagged:
Bernoulli distribution,
binary classification,
binary classifier,
classification,
line of no discrimination,
machine learning,
random variable,
receiver operating characteristic curve,
ROC curve,
sensitivity,
specificity,
supervised learning 
Please comment on the article here: The Chemical Statistician » Statistics
Tags: applied statistics, Bernoulli distribution, binary classification, binary classifier, classification, line of no discrimination, Machine learning, Machine Learning Lesson of the Day, Predictive Modelling, Random variable, receiver operating characteristic curve, ROC curve, sensitivity, specificity, statistics, supervised learning
