Polynomial Features

Some patterns in data are real but not straight lines. Polynomial features are a simple way to let a model capture curved relationships by creating extra input columns that represent powers and interactions of your original features.

What polynomial features are

Given an original feature like x, polynomial feature expansion adds new features such as x², x³, and so on. With multiple inputs (say x and z), it can also add interaction terms like x·z, x²·z, etc. A key idea is that this doesn’t change the model class if you’re using a linear model: linear regression is still “linear” in its coefficients, but it becomes nonlinear in the original variables because the inputs now include nonlinear transformations.

Why it matters in supervised learning

Polynomial features can dramatically improve fit when the true relationship is curved, but they also increase model capacity and the risk of overfitting. The number of generated features grows quickly with degree, which can make training unstable without regularization (like ridge regression) and good validation.

House prices: price might rise with size but flatten out; adding size² can model that curvature.
Credit risk: risk might increase sharply after a threshold; interactions like debt·income can capture combined effects.
Demand forecasting: promotions and seasonality can interact; polynomial interactions approximate those effects.

How you’ll see it in practice

In scikit-learn, this is commonly done with PolynomialFeatures, usually inside a pipeline with scaling and a regularized linear model:

from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import Ridge
from sklearn.pipeline import make_pipeline

model = make_pipeline(PolynomialFeatures(degree=2, include_bias=False), Ridge(alpha=1.0))

Polynomial features are derived input variables created by expanding original features into higher-degree terms and interaction terms (e.g., from x to x, x², x³, and x₁x₂). They let linear models represent nonlinear relationships while keeping the model linear in its parameters. This matters because they can significantly improve predictive accuracy when the target depends on curved trends or feature interactions, but they also increase dimensionality and overfitting risk.

Think of trying to predict house prices using only “size.” If you draw a straight line, it might miss the fact that prices can rise faster for bigger homes. Polynomial features are a way to give a model extra “dials” to capture these bends and curves. They do this by creating new versions of your existing inputs, like “size squared” or “size cubed,” and sometimes combinations like “size × number of rooms.”

In supervised learning, this helps simple models handle more complex patterns without changing the original data source—just by expanding the set of inputs the model can learn from.