Decision Boundary

🔀 Decision Boundary

In classification problems, a decision boundary is the surface (line in 2D) that separates different predicted classes.

It’s the point where the model is undecided — where the prediction flips from one class to another.

1. Why Do We Need It?

In regression, we predict a continuous value.
In classification, we choose a category.

To classify data, the model learns how to divide the input space into regions — one region per class.
The borderline between regions is the decision boundary.

2. Logistic Regression Example

In binary classification, logistic regression computes a probability between 0 and 1:

$$\hat{y} = \sigma(w \cdot x + b)$$

We predict class 1 when $\hat{y} \geq 0.5$, and class 0 otherwise.

So the decision boundary is where:

$$\hat{y} = 0.5 \quad \Rightarrow \quad w \cdot x + b = 0$$

This is a straight line (or hyperplane in higher dimensions) — the place where the model is exactly 50% confident.

3. Other Models

Different models produce different kinds of boundaries:

• Linear models (like logistic regression): straight lines
• k-NN: irregular, local shapes
• SVM with kernels: smooth curves
• Decision Trees: axis-aligned splits

Summary

• A decision boundary separates one class from another.
• It shows where the model switches its prediction.
• Visualizing boundaries helps us understand how the model thinks — and whether it generalizes well.