🎓
An Evaluation System
  • An Evaluation System for Interpretable Machine Learning
  • 1. Abstract
  • 2. Introduction
  • 3. Interpretable Machine Learning
    • 3.1 Definition
    • 3.2 Methods
      • 3.2.1 Machine Learning Model
      • 3.2.2 Model-Agnostic
      • 3.2.3 Sample Theory
  • 4. Evaluation System of Interpretable Machine Learning
    • 4.1 Definition
    • 4.2 The Structure of the Evaluation System
      • 4.2.1 Interpretable Models(Model specific)
      • 4.2.2 Model Agnostic
      • 4.2.3 Human Explanations
    • 4.3 Reference Score Table
  • 5. Customized - Interpretable Machine Learning Model
    • 5.1 Why & How the user customize their model?
    • 5.2 Example
      • 5.2.1 Suggestion Lists
      • 5.2.2 Notebook
    • 5.3 Results & Explanation
  • 6. Discussion
  • 7. Citation and License
Powered by GitBook
On this page
  • 4.3.1 Reference Score Table - Interpretable Models/Model-Specific
  • 4.3.2 Reference Score Table - Model-Agnostic

Was this helpful?

  1. 4. Evaluation System of Interpretable Machine Learning

4.3 Reference Score Table

The below score table is a reference score table that is derived from the study that was executed. This is a generic/standard score we would like to hold for the Algorithms and the Methods for reference. It still needs examination to finalize and standardize the score to use it as a reference score table for any reader or user who would like to score their model and make their preferences.

4.3.1 Reference Score Table - Interpretable Models/Model-Specific

Algorithm

Linearity

Monotonicity

Interaction

Parametric

Transparent

Algorithmic complexity

Logistic Regression

1

1

1

1

1

1

Generalized Linear

1

1

1

1

1

0.9

Gradient Boosting

0.5

0.7

0.7

1

1

0.8

XGBoost

0.1

0.4

0.7

0.9

0

0.7

Distributed Random Forest

0.1

0.4

0.7

1

0

0.9

Deep Learning

0

0

0

0.1

0

0.2

K-means

0.1

0.8

0.1

0.7

1

0.5

4.3.2 Reference Score Table - Model-Agnostic

Explanation Method

Feature summary

Model internals

Datapoint

A surrogate intrinsically interpretable model

Expressive power

Portability

Algorithmic complexity

Detailed

Correctness

Consistency

Stability

Certainty

Importance

Novelty

Representativeness

Variable Importance / Standardized Coefficient

1

1

0.5

/

1

1

1

1

1

0.3

1

1

1

0

1

PDP

1

0.1

0.9

/

1

1

0.9

0.5

1

0.8

1

0.8

0.8

1

0.1

ICE

1

0.2

1

/

1

0.5

0.5

1

1

0.8

1

0.9

1

0.9

0.1

ALE

1

0

0.9

/

1

1

1

0.6

1

0.9

1

0

1

1

0.1

NOTE: The above can be used for references. Though having said that, this table is yet to be standardized, and socred better with mathematical inferences.

Previous4.2.3 Human ExplanationsNext5. Customized - Interpretable Machine Learning Model

Last updated 4 years ago

Was this helpful?