Regression, Decision Tree & Confusion Matrix

Linear regression fits a straight line predicting a numeric target from features.

Linear Regression

Linear regression fits a straight line predicting a numeric target from features.

Use when relationship looks roughly linear on a scatter plot.

Real-life example: Predicting electricity bill from units used — more units, higher bill, roughly straight line.

Simple linear regression

from sklearn.linear_model import LinearRegression

X = [[1], [2], [3], [4]]
y = [2, 4, 5, 4]
model = LinearRegression().fit(X, y)
print(model.predict([[5]]))

Polynomial Regression

Polynomial features let the line curve — good when growth accelerates or slows.

Risk overfitting if degree is too high for small data.

Real-life example: Plant height vs weeks — slow start, fast middle, plateau — not a straight line.

Polynomial features

import numpy as np
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression

X = np.array([[1], [2], [3], [4]])
y = [1, 4, 9, 16]
poly = PolynomialFeatures(degree=2)
X_poly = poly.fit_transform(X)
model = LinearRegression().fit(X_poly, y)
print(model.score(X_poly, y))

Multiple Regression

Multiple features predict one target — e.g. hours studied + attendance → exam score.

Same LinearRegression with multi-column X.

Real-life example: House price from size, bedrooms, and location — many inputs, one price output.

Two features

from sklearn.linear_model import LinearRegression

X = [[1, 80], [2, 85], [3, 90]]
y = [55, 65, 75]
model = LinearRegression().fit(X, y)
print(model.predict([[4, 88]]))

Logistic Regression

Logistic regression predicts classes (yes/no) using a sigmoid curve — despite the name, it classifies.

Outputs probabilities you can threshold at 0.5.

Real-life example: Email spam filter — words and sender stats predict spam (1) or ham (0).

Binary classification

from sklearn.linear_model import LogisticRegression

X = [[0.5], [1.0], [1.5], [2.0]]
y = [0, 0, 1, 1]
model = LogisticRegression().fit(X, y)
print(model.predict([[1.2]]))

Decision Tree

Decision trees split data with if/else rules — easy to visualize and explain.

Can overfit deep trees; limit max_depth for generalization.

Real-life example: Loan approval — if income > X and credit > Y then approve, else reject.

DecisionTreeClassifier

from sklearn.tree import DecisionTreeClassifier

X = [[1], [2], [3], [4]]
y = [0, 0, 1, 1]
model = DecisionTreeClassifier(max_depth=2).fit(X, y)
print(model.predict([[2.5]]))

Confusion Matrix

Confusion matrix counts true/false positives and negatives — better than accuracy alone on imbalanced data.

confusion_matrix and classification_report in sklearn summarize results.

Real-life example: Medical test — false negative misses disease; false positive causes extra stress.

Evaluate classifier

from sklearn.metrics import confusion_matrix, classification_report
from sklearn.tree import DecisionTreeClassifier

X = [[1], [2], [3], [4], [5], [6]]
y = [0, 0, 0, 1, 1, 1]
model = DecisionTreeClassifier().fit(X, y)
pred = model.predict(X)
print(confusion_matrix(y, pred))
print(classification_report(y, pred))