Notes
🔍

Linear Predictor

When a model “adds up evidence” from different features—income, age, number of late payments—it usually does that with a weighted sum. That weighted sum is the model’s raw score before it gets turned into something like a probability.

What it is

The linear predictor is the expression that combines input features into a single number using learned weights. In logistic regression it’s typically written as:

η = β0 + β1 x1 + β2 x2 + ... + βp xp

Here, β0 is the intercept, the βj values are coefficients, and the xj values are your features. This quantity is also called the logit (or log-odds) once you connect it to probabilities: logistic regression sets log(p/(1−p)) = η, and then uses the sigmoid to convert η into a probability p.

How it behaves (intuition)

Think of the linear predictor as a “scorecard”:

  • Each feature contributes βj xj points.
  • Positive coefficients push η up; negative ones pull it down.
  • A one-unit increase in xj changes η by βj, holding other features fixed.

Because η lives on an unbounded scale, it’s easier to model additively than probabilities, which must stay between 0 and 1.

Why it matters in practice

Many key decisions happen at the linear predictor level:

  • Interpretability: coefficients tell you how features shift the log-odds (and therefore the probability).
  • Feature scaling: if features are on wildly different scales, η can be dominated by one input, affecting training and regularization.
  • Thresholding: classification ultimately depends on whether the probability derived from η crosses a cutoff (e.g., fraud vs. not fraud).

In scikit-learn, LogisticRegression computes this internally; calling decision_function(X) returns values closely tied to the linear predictor (the raw score before converting to probabilities).

A linear predictor is the weighted sum of input features plus an intercept, typically written as η = w·x + b. In logistic regression, this quantity (the log-odds) is passed through the sigmoid to produce a class probability. It matters because it is the model’s core scoring function: learning the weights that define the linear predictor determines the decision boundary and directly controls predictive performance.

Think of a linear predictor like a simple scoring recipe. You take a few ingredients, multiply each by how important it is, and add them up to get one final score. For example, a lender might score a loan application using income, debt, and payment history, each with its own weight.

In supervised learning, a linear predictor is that weighted-sum score a model builds from the input information. In logistic regression, this score doesn’t directly say “yes” or “no.” Instead, it’s the raw “confidence signal” that gets turned into a probability, like “there’s an 80% chance this email is spam.”