nredell / shapflex Goto Github PK

The package should work as intended, but I need to go back through and tighten up the API. I also need to add tests and generally more clarity and flexibility with what can and can't be tuned.

Hi, I've tried to extract the pairwise shap interaction returned by the shap.scatter plot. I've noticed that shap_interaction(X_i, X_j) != shap_interaction(X_j, X_i), Thus if we immagine a MxM interaction matrix, it's not symmetric. Is It normal o r I've done something wrong? And if it's ok, why is that?
Thanks in Advance, Lorenzo

I used your cool function and it worked well, but for one dataset it returned me NaN.

At present, the package only works with single-outcome regression models. This isn't stated anywhere and, well, the logo seems to imply otherwise. Fix.

Within setup.py, the opening of README.md requires utf-8 encoding. Otherwise installation fails.

I dumped the adult_dataset that you mention in ReadMe into a csv and run a RandomForestClassifier with almost same settings and calculate shap values from the SHAP package library in python (as given by the author of SHAP paper). I then compare these results with the symmetric counterpart of your library.

1. As a result, I can't see the same (even approximately) set of global shapley values.
2. Also I don't understand for calculating the global shapley value, you find the mean of shapley values for every instance while the SHAP paper suggests doing an mean of absolute of those shapley values.

Pseudo Code:

import numpy as np
import pandas as pd
import shap
from sklearn.ensemble import RandomForestClassifier

df = pd.read_csv('adult_dataset.csv')
# encode categorical variables and get features and labels in X, y
X, y = preprocess(df)
model = RandomForestClassifier(max_depth=6, random_state=0, n_estimators=300)
model.fit(X, y)

shap.initjs()
explainer= shap.TreeExplainer(model, data=X)
shap_values = explainer.shap_values(X)

# Global shapley values
gsv = np.mean(np.abs(shap_values[1]), axis=0)

As a side note, TreeExplainer finds exact shap values but your results don't match with KernelExplainer even.

Thanks in advance.

I'm a big fan of the future package(s) in R. I'm going to ditch mclapply if favor of futures.

Show how the package works with multiple datasets, not simply two models in an ensemble. Also, be more explicit about the dataset weighting when I do this.

The sampling method(s) in the package need to be more clearly spelled out. There are a of couple related methods in the literature that I'd like to incorporate. Namely, there should be a clear trade-off that the user can make between sampling instances vs. features. Right now, the stochastic-ness in the algorithm is to sample a random instance and shuffle its features in one go...but there might be benefit to sampling one instance and shuffling its features multiple times. Seems like both approaches would converge in the limit but the whole point of the Monte Carlo approach is that we're nowhere near "the limit". Also, the impact of feature dependence needs to be worked out. I've done some reading here but I'm not confident about what the best approach is.

Hello,
thanks a lot for implementing the idea of decomposing the R2 Shapley decomposition in R.
I've tried to understand and implement all steps you mention in your paper.
The main important thing for me is the calculation of the sigma unique (due to feature correlations). However, by applying the formula in the paper I receive values higher than one.
So I was wondering whether I have to replace the real y by the predicted y in the counter of the formula?

Hello. I am trying to use your package to assess healthcare attrition on a large scale, However, I am unable to install this package.

install.packages("shapFlex")
Warning in install.packages :
package ‘shapFlex’ is not available for this version of R

What should I do?