Clustering, KNN & Advanced ML Topics
Hierarchical clustering builds a tree of clusters — agglomerative (bottom-up) is common.
Hierarchical Clustering
Hierarchical clustering builds a tree of clusters — agglomerative (bottom-up) is common.
Useful when you do not know K in advance and want a dendrogram view.
Real-life example: Organizing family photos into nested folders — siblings, then cousins, then whole family.
from sklearn.cluster import AgglomerativeClustering
X = [[1], [2], [9], [10]]
model = AgglomerativeClustering(n_clusters=2)
print(model.fit_predict(X))K-means
K-means splits data into K groups by minimizing distance to cluster centers.
Pick K with domain knowledge or elbow method.
Real-life example: K-means is seating guests at K tables so friends sit near their table center.
from sklearn.cluster import KMeans
X = [[1], [2], [10], [11]]
model = KMeans(n_clusters=2, random_state=42, n_init=10)
print(model.fit_predict(X))KNN (K-Nearest Neighbors)
KNN classifies by majority vote of K nearest training points — simple and strong baseline.
Scale features first; odd K avoids ties in binary classification.
Real-life example: KNN is asking your five nearest neighbors who they voted for — majority wins.
from sklearn.neighbors import KNeighborsClassifier
X = [[1], [2], [3], [4], [5], [6]]
y = [0, 0, 0, 1, 1, 1]
model = KNeighborsClassifier(n_neighbors=3).fit(X, y)
print(model.predict([[3.5]]))Grid Search
GridSearchCV tries hyperparameter combinations (K, depth, etc.) with cross-validation.
Finds better settings than manual guessing.
Real-life example: Grid search is tasting every spice combo on a small sample before cooking the full pot.
from sklearn.model_selection import GridSearchCV
from sklearn.neighbors import KNeighborsClassifier
X = [[1], [2], [3], [4], [5], [6]]
y = [0, 0, 0, 1, 1, 1]
grid = GridSearchCV(
KNeighborsClassifier(), {"n_neighbors": [1, 3, 5]}, cv=3
)
grid.fit(X, y)
print(grid.best_params_)Cross Validation
K-fold CV rotates train/test splits — more stable score than one split.
cross_val_score reports mean accuracy across folds.
Real-life example: Cross validation is five mock exams before boards — average score is more trustworthy than one lucky paper.
from sklearn.model_selection import cross_val_score
from sklearn.tree import DecisionTreeClassifier
X = [[1], [2], [3], [4], [5], [6]]
y = [0, 0, 0, 1, 1, 1]
scores = cross_val_score(DecisionTreeClassifier(), X, y, cv=3)
print(scores.mean())Bootstrap Aggregation
Bagging trains many models on random samples and averages votes — RandomForest is the famous example.
Reduces overfitting vs single deep trees.
Real-life example: Bootstrap aggregation is asking many doctors for diagnosis and taking the common answer.
from sklearn.ensemble import RandomForestClassifier
X = [[1], [2], [3], [4], [5], [6]]
y = [0, 0, 0, 1, 1, 1]
model = RandomForestClassifier(n_estimators=50, random_state=42)
model.fit(X, y)
print(model.predict([[2.5]]))Categorical Data
ML models need numbers. Encode categories with OneHotEncoder or pandas get_dummies().
Never feed raw strings like 'red', 'blue' directly into sklearn estimators.
Real-life example: Categorical encoding is converting city names to numbered tickets before sorting machines can process them.
import pandas as pd
df = pd.DataFrame({"color": ["red", "blue", "red"]})
encoded = pd.get_dummies(df)
print(encoded)