Notes
🔍

Regression

When you want a model to predict a number—like a house price, tomorrow’s demand, or a patient’s blood pressure—you’re in the world of regression. It’s the supervised-learning setup for “predict a quantity,” not “pick a category.”

What regression is really doing

In regression, the target (the label you train on) is a continuous value, such as 42.7 or 120, rather than a discrete class like “spam” vs “not spam.” A regression model learns a function that maps input features (square footage, location, interest rates) to a predicted number. Training means choosing model parameters that make predictions close to the true targets on the training data, according to a loss function—commonly mean squared error (MSE) or mean absolute error (MAE).

Common examples and models

  • House price prediction: predict sale price from size, neighborhood, and age.
  • Credit risk: predict expected loss amount (not just default yes/no).
  • Demand forecasting: predict units sold next week from seasonality and promotions.
  • Medical: predict length of stay or lab values from patient measurements.

Concrete tools you’ll see include scikit-learn’s LinearRegression, Ridge/Lasso (regularized linear regression), RandomForestRegressor, and gradient-boosted trees like XGBoost’s XGBRegressor.

Why it matters in a supervised pipeline

Choosing regression affects everything: which metrics you report (RMSE, MAE, R²), how you handle outliers (MAE is more robust than MSE), and even what “good” predictions look like (calibrated magnitudes, not just correct labels). Treating a numeric target as classification throws away ordering and distance information—predicting 100 vs 101 should be “less wrong” than 100 vs 1000, and regression is built to capture that.

Regression is a supervised learning problem where a model learns a mapping from input features to a continuous-valued target, producing numeric predictions (e.g., predicting house price or demand). It matters because many real-world forecasting and estimation tasks require calibrated continuous outputs, and the choice of a regression formulation determines the model family, loss function (e.g., squared error), and evaluation metrics (e.g., MAE/RMSE).

Think of trying to guess a fair price for a used car. You look at mileage, age, brand, and condition, then you come up with a number—not a category like “good” or “bad.” That kind of number-guessing is what regression means in machine learning.

In supervised learning, regression is used when the thing you want to predict is a continuous value, like a temperature, a house price, next month’s electricity bill, or how long a delivery will take. The model learns from past examples where the correct number is known, then uses those patterns to predict new numbers.