Test 6. Performance Metrics

$Question 1$

Define accuracy. Write its formula and explain in which scenarios accuracy is a suitable metric and when it can be misleading.

Answer 1!

Accuracy measures the proportion of correct predictions:

> Accuracy = \frac{TP + TN}{TP + TN + FP + FN}

Suitable when classes are balanced and all errors carry similar cost.
Misleading on imbalanced data: a model that always predicts the majority class can have high accuracy despite poor performance on the minority class.

$Question 2$

Define precision, recall, and F1‑score. Provide their formulas. Given a confusion matrix, explain how these metrics reflect different aspects of classification performance.

Answer 2!

Precision (positive predictive value): $> Precision = \frac{TP}{TP + FP}$
Recall (sensitivity or true positive rate): $> Recall = \frac{TP}{TP + FN}$
F1‑score (harmonic mean of precision and recall): $> F 1 = 2 \times \frac{Precision \times Recall}{Precision + Recall}$
Interpretation:
- Precision focuses on the correctness of positive predictions.
- Recall focuses on coverage of actual positives.
- F1 balances both, penalizing extreme imbalance between precision and recall.

$Question 3$

Explain the confusion matrix for a binary classifier. Label its four cells and describe how you derive accuracy, precision, recall, and specificity from it.

Answer 3!

A binary confusion matrix:

	Predicted Positive	Predicted Negative
Actual Positive	TP	FN
Actual Negative	FP	TN

Accuracy = (TP + TN) / total
Precision = TP / (TP + FP)
Recall = TP / (TP + FN)
Specificity (true negative rate) = TN / (TN + FP)

$Question 4$

Define Mean Squared Error (MSE) and Root Mean Squared Error (RMSE). Provide their formulas and explain what each metric conveys about regression performance.

Answer 4!

MSE: average squared difference between predictions and targets: $> MSE = \frac{1}{n} \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i})^{2}$
RMSE: square root of MSE, in the same units as the target: $> RMSE = \sqrt{MSE}$
Interpretation:
- MSE penalizes larger errors more heavily.
- RMSE is more interpretable, reflecting typical error magnitude.

$Question 5$

Describe the ROC curve and AUC. Define true positive rate and false positive rate. Explain how to construct the ROC curve and interpret the AUC value.

Answer 5!

True Positive Rate (TPR) = Recall = TP / (TP + FN)
False Positive Rate (FPR) = FP / (FP + TN)
ROC curve: plot TPR vs. FPR as the classification threshold varies from 0 to 1.
AUC (Area Under the Curve): probability that a randomly chosen positive ranks higher than a negative.
- AUC = 1.0: perfect separation.
- AUC = 0.5: no better than random guessing.
Use: compare classifier discrimination ability independent of threshold or class balance.