2. Introduction

Introduction to Interpretable Models and Evaluation Systems

2.1 Interpretable Machine Learning Model

Interpretable Machine Learning is an increasingly ubiquitous topic. However, most of these accurate machine learning models remain complex black boxes, meaning that it does not reveal its internal mechanisms. Therefore, how to interpret these complex machine learning models become more and more important these days.

Taken to its full generality, to interpret data means to extract information (of some form) from them. The set of methods falling under this umbrella spans everything from designing an initial experiment to visualizing final results.

We define interpretable machine learning as the extraction of relevant knowledge from a machine-learning model concerning relationships either contained in data or learned by the model. Here, we view knowledge as being relevant if it provides insight for a particular audience into a chosen problem.

We referred to a book that helps us understand the Interpretable Machine learning Model. The methods we use here are machine learning models and model-agnostic. Here, machine learning models will be the algorithms that help us generate the models.

Ex: GBM, GLM, DRF, XGBoost, etc. While the model-agnostic will be the methods that are used to interpret these models, ex: Variable importance, Partial Dependacy Plot, ICE plot, etc.

2.2 Evaluation System of Interpretable Machine Learning Model

One of the most important factors in the evaluation of Interpretable Machine Learning is the ability to explain or to present the model in understandable terms to a human. With such a great extent of subjectiveness, the criterion of evaluation could vary from person to person. However, there is a widely accepted taxonomy of interpretability evaluation as follows: https://arxiv.org/abs/1702.08608

• Application-level evaluation (Real humans, Real tasks): Requires conducting human experiments within a real application. Imagine fracture detection software with a machine learning component that locates and marks fractures in X-rays. At the application level, radiologists would test the fracture detection software directly to evaluate the model. This requires a good experimental setup and an understanding of how to assess quality. A good baseline for this is always how good a human would be at explaining the same decision.

• Human-level evaluation (Real humans, Simplified tasks) Requires conducting simpler human-subject experiments that maintain the essence of the target application. The difference is that these experiments are not carried out with the domain experts, but with laypersons, allowing for both a bigger subject pool and fewer expenses. This makes experiments cheaper (especially if the domain experts are radiologists) and it is easier to find more testers. An example would be to show a user with pairs of explanations and let the user choose the best one.

• Function level evaluation (No humans, Proxy tasks) Requires no human experiments. This works best when the class of model used has already been evaluated by someone else in a human level evaluation. For example, it might be known that the end-users understand decision trees. In this case, a proxy for explanation quality may be the depth of the tree. Shorter trees would get a better explainability score. It would make sense to add the constraint that the predictive performance of the tree remains good and does not decrease too much compared to a larger tree.

Based on this taxonomy, we come up with our evaluation system consists of three criteria: Interpretable Models/ Model-specific perspective, Model-agnostic perspective, and human-friendly explanation perspective. Under each criterion, we have several keywords as the feature of interpretability helping us evaluate the model. And this is the structure of our evaluation system.

2.3 Customized the Interpretable Model with Specific Focus

This section is an extension of Chapter 4, aims to present the usage of the Evaluation System of the interpretable machine learning model. In general, users can select their preferred features. And based on the score table of the evaluation system, two lists of suggested methods will be presented to users. One is model-specific methods, the other is model-agnostic methods. With these two suggestion lists, the user can build their interpretable machine learning model with their specific focus. Why and How to Customized the Interpretable Model

An example is presented here to demonstrate how the evaluation and scoring system would work for the interpretable machine learning model.