Supervised Fork — Regression and Classification

Loaded: fork-supervised.md — metrics helpers and fork-specific guards.

Read once the triage resolves, before Data Preparation. Both paths share the same helper shape: a function taking (model, predictors, target) and returning a one-row DataFrame, so per-model results concatenate into a comparison table without reshaping.

Regression

Metrics: RMSE, MAE, R², Adjusted R², MAPE.

Three guards, each of which produces a silently wrong number rather than an error:

MAPE and zero actuals. mean(|y - ŷ| / y) × 100 returns inf or NaN if any actual value is 0. Check (y == 0).sum() before including it. If nonzero, use sMAPE or drop MAPE and say why in one line.

Adjusted R² and wide feature sets. 1 − (1−R²)(n−1)/(n−k−1) degenerates as k approaches n. After one-hot encoding — especially of a high-cardinality categorical — check n > k + 1. On a small test set with many dummy columns this can flip negative or explode.

RMSE and deferred outliers. RMSE squares errors, so it carries outlier weight disproportionately. If outliers were deferred in preparation, say that when reporting RMSE, and read MAE alongside it — a large RMSE/MAE gap points at a few big misses rather than uniformly poor fit.

Interpretation: report train and test together and name over/underfitting explicitly rather than listing both numbers and moving on. R² near 1 on this kind of data warrants interrogation, not celebration — check the group-leakage finding from Data Overview first.

Classification

Metrics: Accuracy, Precision, Recall, F1, plus ROC-AUC where .predict_proba() is available. Import roc_auc_score only if it will actually be computed; an unused import is a code-quality flag on most rubrics.

Class balance governs how these read. State the no-information baseline before the first model: a model predicting the majority class every time achieves accuracy equal to the majority class share. Every accuracy figure afterwards is measured against that, not against zero.

Choosing the metric that drives tuning. This is a business question, not a statistical one, and it must be asked rather than defaulted:

• A false negative misses a real positive — a lost opportunity, an undetected fault, an untreated case.
• A false positive flags something that isn't — wasted effort, an unnecessary follow-up.

When the costs are asymmetric, say which and pick the metric that matches: recall when misses dominate, precision when false alarms dominate, F1 when they're comparable. Then use that metric in GridSearchCV(scoring=...), not the default. State the choice and the reasoning in a markdown cell — a grader and a stakeholder will both ask.

Expect optimising for recall to yield a smaller recall gain than expected once regularising hyperparameters (min_samples_leaf, max_depth) are also in the grid. That's not a failure — the constraint that prevents overfitting also limits how aggressively the model can chase every positive. Say so rather than treating a small recall gain as a problem.

Confusion matrix alongside feature importance for each tuned model. Interpret it in counts and business terms, not just percentages: of N actual positives, the model found X and missed Y. That sentence is what makes the metric concrete, and it's what the recommendations section will draw on.

Shared

Both paths evaluate on train and test, always, for every model. A test figure alone hides overfitting; the gap between them is the finding. A near-perfect training score with a much lower test score means memorisation, and it should be named as such rather than reported neutrally.