xai 3

• for LIME, the neighbouring values are randomly sampled, so it might differ from run to run

• so how do we explain a model with 50000 feat instead of 20?

1. KernelSHAP
   • shapley is not scalable so kernelshap scales to many features by fitting a wt linear surrogate
   • kernelSHAP- regression setup

• instead of calculating all 2^p coalitions, we only check for a smaller number and then use weighted least squares

2. Deep Learning explanations

   • motivation

     • heatmaps reveal what experts overlook!
     • new patterns immerge that can explain a scenario better
     • saliency maps - scores each region in the input by how strongly it impacts model pred
     • high acc may be wrong sometimes - clever hans effect or shortcut learning

   • deep nets are differentiable, one backward pass gives saliency value per pixel

     • 3 families of pixel attribution
       • gradient based - compute df/dx at the input, fast (1 bwd pass) ex- vanilla gradient
       • activation based - computes for intermediate feat map activations (coarse but class discriminative) ex- CAM, Grad-CAM
       • removal based - how f’ changes when feat are removed (slow) ex- LIME, KernelSHAP

   • Vanilla Gradient

     

   • Integrated gradients

     • fixed flaw of vanilla gradient - gradient saturation or dead ReLUs
     • since deep nn use non linear activation, its local derivative might be 0 because ReLU along that path might be saturated → leading to blank or incomplete heatmaps

     

   • class activation mapping - using classifier’s own weights as localiser

     

   • Grad-CAM

• Grad-CAM uses the idea that by using backpropagation we can calculate gradients, we can derive the importance of each feature map dynamically, regardless of what layers lie between the feature map and the final output score.

• why this is better than using input feat?
  • Semantic Abstraction: Deep channels at the final conv layer capture high-level semantic concepts (e.g., “dog snout”, “wheel”), whereas raw input pixels $x$ only capture low-level color and edge information.

Links:

202608150116