Logistic Regression

Logistic regression is simply a way to predict the chance of a yes-or-no outcome, like whether a loan defaults, from some measurement. It's named after the logistic function, the S-shaped curve it uses. The first formula looks scary, but read it inside out: $b_0 + b_1x$ (an ordinary straight line, just like the last few lessons), plugged into $\frac{1}{1 + e^{-(\dots)}}$ (a squashing step that turns any number into something between 0 and 1). The second formula says the same thing backward: the log-odds (a way of rescaling probability, explained next) is a straight line in $x$. It's a line in disguise, so bear with the notation.

Why not just a line? Suppose you fit an ordinary regression line to 0s and 1s. Out at large $x$ it keeps climbing past 1, and at small $x$ it dips below 0, predicting a 120% or a −15% chance. Probabilities are stuck between 0 and 1, so we need to stretch them onto a scale that runs all the way from $-\infty$ to $\infty$, fit the line there, and then squash back.

Odds. That stretching happens in two steps. If $p = 0.8$, the odds are $\frac{0.8}{0.2} = 4$ (4 to 1). Odds run from 0 to infinity, and taking the log stretches that to $-\infty$ to $\infty$, so a line can model log-odds. The sigmoid is just that process undone, which is why the curve has its S shape.

The 50% point is where $b_0 + b_1x = 0$, at $x = -\frac{b_0}{b_1}$. That's where the odds are 1 to 1, so the log-odds are 0.

Fitting. The coefficients are chosen to maximize the likelihood of the observed yes/no outcomes (there's no closed form, so software iterates). The model is the simplest neural network: one neuron with a sigmoid activation.

A common mistake. It's tempting to read $b_1$ like an ordinary slope, as "the probability goes up by $b_1$ per unit of $x$." It doesn't. $b_1$ is the change in the log-odds. On the odds it acts as a multiplier, $e^{b_1}$ per unit of $x$, and the change in probability itself depends on where you are on the S-curve: big in the middle, tiny near 0 or 1.

Credit scoring, spam detection, disease risk models and click prediction all use logistic regression, and it's the output layer of most classification neural networks.