Softmax Regression

This note introduces the Softmax Regression algorithm using scikit-learn, explains the step-by-step logic behind how it works, and then demonstrates a from-scratch implementation to show that the core idea is simple and easy to build. What is Softmax Regression? Softmax Regression (also called...

Entropy and Information Gain

Overview When building a Decision Tree, we aim to split data into increasingly pure groups. But how can we measure "purity" or "impurity" mathematically? One powerful way is using entropy, a concept from information theory. Entropy measures how mixed or uncertain a group is. When we split data, we...

Regression Metrics

Quick Summary SSE: Sensitive to large errors, not same unit (squared unit). MSE: Sensitive to large errors, not same unit (squared unit). MAE: Same unit as target, treats all errors equally. RMSE: Same unit as target, sensitive to large errors. Key Points SSE (Sum of Squared Errors) $$ \text{SSE} =...

Linear Regression Direct Solution

Step 1: Define the Cost Function We want to minimize the error between predictions and true values. $$ J(w) = \frac{1}{2} | Xw - t |^2 $$ ✅ Meaning: $Xw$: predicted values $t$: true target values $Xw - t$: error vector $| \cdot |^2$: sum of squared errors $\frac{1}{2}$: for convenient derivative...

Unsupervised Learning

🧩 Introduction to Unsupervised Learning Unsupervised learning is a type of machine learning where the model learns from data without any labels. There are no answers provided — just raw input data. The goal is to discover patterns, structures, or groupings hidden inside the data. 1\. How Is It...

Supervised Learning

🧠 Introduction to Supervised Learning Supervised learning is one of the main branches of machine learning. It refers to training a model using data where the correct answers (called labels) are already known. 1\. What Does “Supervised” Mean? It’s called supervised because the model learns from...

Bias Variance Tradeoff

🎯 Bias–Variance Tradeoff In machine learning, we want our model to learn useful patterns — not just memorize the data or oversimplify it. The bias–variance tradeoff helps us understand the balance between underfitting and overfitting. 1\. What Is Bias? Bias is the error caused by using a model...

Decision Boundary

🔀 Decision Boundary In classification problems, a decision boundary is the surface (line in 2D) that separates different predicted classes. It’s the point where the model is undecided — where the prediction flips from one class to another. 1\. Why Do We Need It? In regression, we predict a...

Anomaly Detection

🚨 Introduction to Anomaly Detection Anomaly detection is about finding things that don’t belong. These could be: A fraudulent credit card transaction A faulty machine sensor reading An unusual customer behavior We want to identify rare, unusual patterns that are different from the normal data —...

Manifold Learning

🌐 Understanding Manifold Learning PCA is powerful — but it assumes that the important structure in the data lies along straight (linear) directions. But what if the data lies on a curved surface inside a high-dimensional space? This is where manifold learning comes in. 1\. Why PCA Isn't Always...

Principal Component Analysis (PCA)

This note introduces the Principal Component Analysis (PCA) technique using scikit-learn, explains the step-by-step logic behind how it works, and then demonstrates a from-scratch implementation to show that the core idea is simple and intuitive to build. What is PCA? Principal Component Analysis...

Hierarchical Clustering

This note introduces the Hierarchical Clustering algorithm using scikit-learn, explains the step-by-step logic behind how it works, and then demonstrates an intuitive from-scratch-like visualization to show that the core idea is simple and easy to understand. What is Hierarchical...

AdaBoost

This note introduces the AdaBoost algorithm using scikit-learn, explains the step-by-step logic behind how it works, and then demonstrates a from-scratch implementation to show that the core idea is intuitive and builds naturally on weak learners like Decision Stumps. What is AdaBoost? AdaBoost...

Random Forest

This note introduces the Random Forest algorithm using scikit‑learn, explains the step‑by‑step logic behind how it works, and then demonstrates a from‑scratch implementation to show that the core idea is simple and builds naturally on Decision Trees. What is a Random Forest? A Random Forest is an...

Gaussian Naive Bayes

This note introduces the Gaussian Naive Bayes algorithm using scikit‑learn, explains the step‑by‑step logic behind how it works, and then demonstrates a from‑scratch implementation to show that the core idea is simple and easy to build. What is Gaussian Naive Bayes? Gaussian Naive Bayes is a...

Multinomial Naive Bayes

This note introduces the Multinomial Naive Bayes algorithm using scikit‑learn, explains the step‑by‑step logic behind how it works, and then demonstrates a from‑scratch implementation to show that the core idea is simple and easy to build. What is Multinomial Naive Bayes? Multinomial Naive Bayes is...

Logistic Regression

This note introduces the Logistic Regression algorithm using scikit-learn, explains the step-by-step logic behind how it works, and then demonstrates a from-scratch implementation to show that the core idea is simple and easy to build. What is Logistic Regression? Logistic Regression is a method...

Lasso Regression

This note introduces the Lasso Regression algorithm using scikit-learn, explains the step-by-step logic behind how it works, and then demonstrates a from-scratch implementation to show that the core idea is simple and easy to build. What is Lasso Regression? Lasso Regression is like Linear...

Ridge Regression

This note introduces the Ridge Regression algorithm using scikit-learn, explains the step-by-step logic behind how it works, and then demonstrates a from-scratch implementation to show that the core idea is simple and easy to build. What is Ridge Regression? Ridge Regression is like Linear...

Polynomial Regression

This note introduces the Polynomial Regression algorithm using scikit-learn, explains the step-by-step logic behind how it works, and then demonstrates a from-scratch implementation to show that the core idea is simple and easy to build. What is Polynomial Regression? Polynomial Regression is like...

Perceptron

This note introduces the Perceptron algorithm using scikit-learn, explains the step-by-step logic behind how it works, and then demonstrates a from-scratch implementation to show that the core idea is simple and easy to build. What is the Perceptron? The Perceptron is one of the earliest algorithms...

DBSCAN

This note introduces the DBSCAN clustering algorithm using scikit-learn, explains the step-by-step logic behind how it works, and then demonstrates a from-scratch implementation to show that the core idea is simple and powerful. What is DBSCAN? DBSCAN (Density-Based Spatial Clustering of...

K-Means Clustering

This note introduces the K-Means Clustering algorithm using scikit-learn, explains the step-by-step logic behind how it works, and then demonstrates a from-scratch implementation to show that the core idea is simple and easy to build. What is K-Means? K-Means Clustering is like grouping similar...

Decision Tree

This note introduces the Decision Tree algorithm using scikit‑learn, explains the step‑by‑step logic behind how it works, and then demonstrates a from‑scratch implementation to show that the core idea is simple and easy to build. What is a Decision Tree? A Decision Tree is a flow‑chart full of...

Bernoulli Naive Bayes

This note introduces the Bernoulli Naive Bayes algorithm using scikit‑learn, explains the step‑by‑step logic behind how it works, and then demonstrates a from‑scratch implementation to show that the core idea is simple and easy to build. What is Bernoulli Naive Bayes? Bernoulli Naive Bayes is like...

Linear Regression

This note introduces the Linear Regression algorithm using scikit-learn, explains the step-by-step logic behind how it works, and then demonstrates a from-scratch implementation to show that the core idea is simple and easy to build. What is Linear Regression? Linear Regression is like drawing the...

K-Nearest Neighbors (KNN) Viz

K-Nearest Neighbors (KNN)

This note introduces the KNN algorithm using scikit-learn, explains the step-by-step logic behind how it works, and then demonstrates a from-scratch implementation to show that the core idea is simple and easy to build. What is KNN? K-Nearest Neighbors (KNN) is like asking your neighbors for advice...