ML Model Performance metrics
Classification metrics
Accuracy = (Number of correct predictions) / (Total number of predictions) for binary classification Accuracy = (TP + TN) / (TP+TN+FP+FN). Accuracy doesn’t work well with a class-imbalanced data set.
Precision = TP / (TP + FP). Tells what proportion of positive identifications was actually correct.
Recall = TP (TP + FN). Tells what proportion of actual positives was identified correctly.
Read more about precision and recall here.
Specifity = TN / (TN + FP). In the case of cancer diagnosis task, it tells what proportion of patients that did NOT have cancer, were predicted by the model as non-cancerous.
Language processing metrics
Levenshtein distance – a distance between two words in terms of the minimum number of single-character edits (insertions, deletions or substitutions) required to change one word into the other.
Perplexity – a measurement of how well a probability distribution or probability model predicts a sample. It may be used to compare probability models. A low perplexity indicates the probability distribution is good at predicting the sample.