Feature attributions for sequence labelling neural networks – Josef Lindman Hörnlund, Modulai

I will discuss some recent axiomatic attribution methods and their application to sequence labeling neural networks in an attempt to explain the predictions of these models. Some of the important axioms for attribution methods are discussed and what they imply for the explanations we get. The difficulty of staying true to the model or true to the data is highlighted.

Key Takeaways

  • A lot is happening with respect to feature attributions right now, but it is still an open research question.
  • Feature dependence in the data is tricky to handle.
  • Never trust a method unless you know how it works.

Add comment