About the training of the Pet

[Smiling-Weeping-zhr]

Hello authors, I have been paying attention to your paper recently, but I followed the steps of your scripts to reproduce the experiment of the Pet part, but found that the effect is not good, the visualization is shown in the figure below, can you please tell me how you train Pet.
Thank you very much for your reply.

[chanwkimlab]

Hi, just to clarify, the training process for the Pet dataset is essentially the same as for the ImageNette dataset since both are multi-class classification tasks. The only differences are the number of classes and the specific dataset class used for loading the data. The MURA dataset, on the other hand, is a bit different, as it is for binary classification, which uses a sigmoid function instead of softmax at the final layer.

For debugging your results, it is challenging to pinpoint the issue based on the provided figure only. However, here are a few quick things to check: Ensure that the classifier's training was successful by verifying the accuracy. Additionally, check if the sum of Shapley values for each class, plus the null value, matches the prediction of the surrogate model for the classifier. From the image provided, the sum values for the correct classes do not seem high. If the Shapley value calculation/plotting was done correctly, this might mean that the performance of classifier was bad.

[Smiling-Weeping-zhr]

Is the accuracy of this pet base model normal?

[chanwkimlab]

Sorry for the late response. The accuracy looks good to me. This might suggest that something about your explainer code is off. The explainer is supposed to provide output that satisfies the efficiency property due to the additive efficiency normalization.

https://github.com/suinleelab/vit-shapley/blob/master/vit_shapley/modules/explainer.py#L218
Also please refer to the equation 16 in our paper https://arxiv.org/abs/2206.05282