Interpretation of Machine Learning Results
From Blackbox to Interpretation
Types of Interpretation
Types of Interpretation
Ceteris Paribus Plot (Local and Model Agnostic)
Result
Partial Dependency Plot (Global and Model Agnostic)
Result
Variable Importance Plot — Permutations
Code
NumberOfCores=parallel::detectCores()
ModelDesignRandFor=rand_forest() %>%
set_engine("ranger", num.threads = NumberOfCores, importance="permutation") %>%
set_mode("regression")
set.seed(2021)
WfModelVax=workflow() %>%
add_model(ModelDesignRandFor)%>%
add_recipe(RecipeVax) %>%
add_case_weights(Population) %>%
fit(DataTrain)
vip(extract_fit_parsnip(WfModelVax))
Variable Importance Plot — Impurity:
Code
ModelDesignRandFor=rand_forest(trees=170, min_n=5, mtry=2) %>%
set_engine("ranger", num.threads = NumberOfCores, importance="impurity") %>%
set_mode("regression")
RecipeVax=recipe(PercVacFull~., data=DataTrain)
set.seed(2021)
WfModelVax=workflow() %>%
add_model(ModelDesignRandFor)%>%
add_recipe(RecipeVax) %>%
add_case_weights(Population) %>%
fit(DataTrain)
vip(extract_fit_parsnip(WfModelVax))
Shapley Values from a Game Theory Approach
Bruce, Carsten, and Greg are Players contributing to the profit
Shapley Values from a Game Theory Approach
Estimating Carsten’s Contribution
We do not know at which level Carsten joins:
Carsten joins last: \(\Delta P^{rofit}_{BGC}=5\)
Carsten joins second:
\(\Delta P^{rofit}_{BC}=7\) or \(\Delta ^{rofit}_{GC}=8\)Carsten joins first: \(\Delta P^{rofit}_{C}=17\)
Carsten’s average contribution (Shapley value):
\(S^{hap}_C=\frac{1}{3}17+\frac{1}{6}7+\frac{1}{6}8+\frac{1}{3}5=9.83\)
Shapley Values from a Game Theory Approach
Estimating Bruces’s Contribution
Calculate Bruce’ contribution as an exercise:
- Spoiler Alert (below is the result):
- Bruces’s average contribution (Shapley value):
\(S^{hap}_B=12.33\)
Shapley Values from a Game Theory Approach
Estimating Greg’s Contribution
Calculate Greg’ contribution as an exercise:
- Spoiler Alert (below is the result):
- Greg’s average contribution (Shapley value):
\(S^{hap}_G=13.83\)
Shapley Values from a Game Theory Approach
Starting Profit — SHAP values — Final Profit
Initial Profit: \(Profit_0=56\)
Bruce’s SHAP: \(S^{hap}_B=12.33\)
Carsten’s SHAP: \(S^{hap}_B=9.83\)
Greg’s SHAP: \(S^{hap}_G=13.83\)
Profit with 3: \(Profit_3=92\)
Shapley Values from a Game Theory Approach
Coalitions and Permutations
Number of coalitions: \(N_{coalition }= 2^k=2^3=8\)
Number of joining scenarios: \(k!=3!=1\cdot 2\cdot 3=6\)
\(BCG\),\(BGC\),
\(CBG\), \(CGB\),
\(GBC\), \(GCB\)If all 6 scenarios are calculated, one can can calculate SHAPley values for all contributors (variables).
If less than 6 scenarios are calculated, one can can estimate SHAPley values for all contributors (variables).
Local/Model Agnostic: Shap Values for Orange County
Code
if (!DraftMode){
library(DALEXtra)
DataTrainPredictorVar=DataTrain %>%
select(-PercVacFull,-Population)
ExplainerRandFor=explain_tidymodels(WfModelVax,
data=DataTrainPredictorVar,y=DataTrain$PercVacFull,
type="regression", verbose = FALSE,
label = "Vaccination Shap Values Random Forest")
set.seed(2021)
ShapValuesID141 = predict_parts(
explainer = ExplainerRandFor,
new_observation = DataVax[141,],
type = "shap",
b = 200)
}
# to plot use plot(ShapValuesID141)County: Orange, CA, Pred. Vac.: 0.72
Pred. Vac. Rate (all U.S. counties): 0.62Mean PercRep: r round(weighted.mean(DataTrain$PercRep, as.numeric(DataTrain$Population)),2)
Mean PercAsian: 0.05
Mean PercFoodSt: 0.11
Mean PercBlack: 0.11
Mean PercOld65: 0.21
Mean PercOld65: 0.19
Mean PercYoung25: 0.09
Shap Values by Variable
