Wavelet Attribution Method

Abstract

Feature attribution methods aim to improve the transparency of deep neural networks by identifying the input features that influence a model's decision. Pixel-based heatmaps have become the standard for attributing features to high-dimensional inputs, such as images, audio representations, and volumes. While intuitive and convenient, these pixel-based attributions fail to capture the underlying structure of the data. Moreover, the choice of domain for computing attributions has often been overlooked. This work demonstrates that the wavelet domain allows for informative and meaningful attributions. It handles any input dimension and offers a unified approach to feature attribution. Our method, the Wavelet Attribution Method (WAM), leverages the spatial and scale-localized properties of wavelet coefficients to provide explanations that capture both the where and what of a model's decision-making process. We show that WAM quantitatively matches or outperforms existing gradient-based methods across multiple modalities, including audio, images, and volumes. Additionally, we discuss how WAM bridges attribution with broader aspects of model robustness and transparency.

Motivation and proposed approach

Feature attribution in the wavelet domain. WAM explains any input modality by decomposing the model’s decision in the wavelet domain. It computes the gradient of the model’s prediction with respect to the wavelet coefficients of the input modality (audio, images, volumes). Unlike pixels, wavelet coefficients preserve structural information about the input signal, offering deeper insights into the model’s behavior and going beyond where it focuses.

Explainable AI (XAI) methods, particularly feature attribution methods, are crucial for understanding the decision-making processes of deep neural networks.

Current feature attribution methods rely on a feature space - namely the pixel domain - that overlooks the inherent temporal, spatial, or geometric relationships within the data.

Wavelets offer a hierarchical decomposition that retains both spatial and frequency information, unlike pixel-based methods that lose structural context. This makes wavelets a stronger foundation for interpreting model decisions across diverse modalities (or signals), as wavelets are inherently low-level features defined across various signal dimensions.

Click here to learn more about the method.

Explanation of different modalities

Image Explanation. WAM decomposes important components at multiple scales, revealing what the model focuses on. The same spatial location encodes different structural elements at different scales: the model predicts the elephant because of its trunk and ears, but needs both the fine texture details, the intermediate-scale contours and the coarse edges.

Audio Explanation. The audio of the target class ('Crow') is mixed with a corrupting audio ('Chirping birds') to form the input to the classifier. Interpretation audio reconstructed with important wavelet coefficients virtually eliminate signal from the corrupting audio, and also clearly emphasize parts of the target class audio .

Volume Explanation. Multi-scale decomposition of feature importance on voxels. Coarse scales highlight the edges of the number, capturing its global structure. Fine scales focus on the digit's center, capturing high-frequency details and localized variations.

WAM provides explanations for various input modalities, including images, audio, and volumes. The following carousel illustrates how WAM decomposes the model's decision into wavelet coefficients at multiple scales, revealing both the where and what of the model's focus. Click here to get more details on the quantitative evaluation of WAM on these modalities and here to see more examples

WAM bridges the gap between feature attribution and broader aspects of model robustness and transparency. Click here visualize application examples.

Poster

Click on the image to enlarge

BibTeX

@inproceedings{
        kasmi2025WAM,
        title={One Wave To Explain Them All: A Unifying Perspective On Feature Attribution},
        author={Kasmi, Gabriel and Brunetto, Amandine and Fel, Thomas and Parekh, Jayneel},
        booktitle={Forty-second International Conference on Machine Learning},
        year={2025},
        }