Logistic Regression

To predict yes or no, squeeze a straight line through an S-curve so the output becomes a probability between 0 and 1. The steepness says how confident the model is.

How it works

The sigmoid and probability

In one sentence

Logistic regression models the probability of class 1 as $p(x) = 1/(1 + e^{-(wx + b)})$ , the sigmoid of a linear score, with decision boundary $wx + b = 0$ .

General formula

{\htmlData{sym=0}{p(x)}} = \frac{1}{1 + e^{-({\htmlData{sym=1}{w}}{\htmlData{sym=3}{x}} + {\htmlData{sym=2}{b}})}}

$wx + b$ is a score that can be any number; the sigmoid squashes it into a probability between $0$ and $1$ .

What the symbols mean

$p(x)$
Predicted probability
The model's chance that the answer is yes, between 0 and 1.
$w$
Weight
How strongly the input pushes the prediction up or down.
$b$
Bias
Shifts where the curve crosses 50%.
$x$
Input
The measured feature, like hours studied.

$wx + b$ can be any number. The sigmoid maps it into $(0, 1)$ .
The decision boundary, where $p = 0.5$ , sits at $x = -b/w$ .
Training maximizes the probability of the observed labels, with gradient descent.

Learn the math

Applications to Linear ModelsLin Alg · Lesson 36, Orthogonality and Least Squares Logistic RegressionStats · Lesson 40, Regression and Correlation