Page 302 - Ai Book - 10

EVALUATION MATRIX FOR AI MODEL

Sometimes, a confusion matrix is not enough for evaluating a model. In such a case, you need some more ways
to evaluate the model. Some commonly used methods of model evaluation are Accuracy, Precision, Recall and
F1 Score.
u Accuracy: Model Accuracy can be defined as the ratio of the correct number of predictions and the total
number of predictions. Thus, the formula of accuracy is:
Number of correct predictions
Accuracy = Total number of predictions
Sometimes, it can be written as:
TP + TN
Accuracy =
TP + TN + FP + FN
Here, TP stands for True Positive and TN stands for True Negative. Now, let us understand the term ‘Accuracy’
with the help of a flood example. Suppose, the model always predicts that there is no flood. But in reality,
there is a 10% chance of flooding.

In this case, the model predicted correctly for 90 cases but for 10 cases, the model prediction is incorrect
i.e.,no flood.

Here, True Positive = 0
True Negative = 90
Total cases = 100 .

Therefore, accuracy becomes: (0 + 90) / 100 = 0.9 which is equivalent to 90 percent.
This method is good for correct prediction, but didn’t consider the real scenario where the probability of
flooding is 10% . Thus, accuracy alone may not be enough to ensure the model’s performance on the data
which has never been used.

u Precision: The term ‘Precision’ can be defined as the ratio of True positive and the sum of True Positive and
False Positive. Thus, the formula of Precision can be written as:
True Positive
Precision =
All predicted Positives

TP
Precision =
(TP + FP)

Where, All Predicted Positive = True Positive + False Positive.
Precision is an important evaluation criteria and you should always remember that high precision indicates
the existence of more cases of True Positive as compared to False Positive.

u Recall: Likewise Precision, recall is another parameter used for evaluating model’s performance. It can be
defined as the ratio of True Positive and the sum of True Positive and False Negative.

True Positive
Recall =
True Positive + False Negative

TP
Recall =
(TP + FP)
Note that a model’s performance can be fully evaluated by determining both measures i.e.,Precision and
Recall.

176
176