RBF (Gaussian) Kernel

When your data isn’t cleanly separable with a straight line (or flat plane), a kernel is a way to let a “linear” model draw curved boundaries without explicitly inventing new features. The RBF (Gaussian) kernel is the most common choice because it turns “closeness” between points into a smooth measure of similarity.

What it computes

The RBF kernel between two input vectors x and x' is:

K(x, x') = exp(-γ ||x - x'||^2)

If two points are very close, K is near 1; if they’re far apart, K quickly shrinks toward 0. The parameter γ (gamma) controls the “reach” of each point’s influence: large γ makes similarity drop off fast (very local, wiggly boundaries), small γ makes it drop off slowly (smoother, more global boundaries).

How it helps SVMs learn non-linear boundaries

In an SVM, predictions depend on similarities to a subset of training points called support vectors. With an RBF kernel, the decision boundary becomes a combination of Gaussian-shaped bumps centered around those support vectors, letting the model carve out flexible regions in the original input space—without ever computing the high-dimensional mapping explicitly (the “kernel trick”).

Practical impact and examples

Spam detection: captures non-linear interactions among word-frequency features.
Fraud detection: isolates small, unusual clusters of behavior patterns.
Disease diagnosis: separates overlapping patient groups using smooth curved boundaries.

Two knobs matter most: C (how much training errors are penalized) and γ. Ignoring their balance leads to underfitting (too smooth) or overfitting (too spiky). In scikit-learn you’ll see this as SVC(kernel="rbf", C=..., gamma=...).

The RBF (Gaussian) kernel is a kernel function defined as K(x, x′) = exp(−γ‖x−x′‖²), measuring similarity that decays exponentially with squared distance. It implicitly maps inputs into an infinite-dimensional feature space, enabling linear learners such as SVM/SVR to fit highly non-linear decision boundaries. It matters because it is a strong default kernel whose performance depends critically on the γ (and regularization) setting.

Imagine you’re trying to decide whether two songs are “similar.” You might not compare every detail—just how close they feel overall. The RBF (Gaussian) kernel does something like that for data: it gives a similarity score between two examples based on how close they are. Very close points get a high score; far-apart points get a low score.

This matters in tools like Support Vector Machines, where the model can draw a flexible, curved boundary between categories (like spam vs. not spam) by leaning on these similarity scores, instead of assuming the boundary has to be a straight line.