MILLET Interpretable Time Series Classifier — Poster

Paper context

Paper title: Inherently Interpretable Time Series Classification via Multiple Instance Learning Abstract: This poster introduces MILLET, a method for interpretable time series classification using Multiple Instance Learning. It details the Conjunctive Pooling mechanism, evaluates performance on a synthetic WebTraffic dataset and UCR benchmarks, and demonstrates improved interpretability without sacrificing predictive accuracy. Paper body (method & results): Published as a conference paper at ICLR 2024 INHERENTLY INTERPRETABLE TIME SERIES CLASSIFICATION VIA MULTIPLE INSTANCE LEARNING Joseph Early∗, Gavin KC Cheung†, Kurt Cutajar,† Hanting Xie,† Jas Kandola,† & Niall Twomey† Corresponding authors: J.A.Early@soton.ac.uk; njtwomey@amazon.co.uk ABSTRACT Conventional Time Series Classification (TSC) methods are often black boxes that obscure inherent interpretation of their decision-making processes. In this work, we leverage Multiple Instance Learning (MIL) to overcome this issue, and pro- pose a new framework called MILLET: Multiple Instance Learning for Locally Explainable Time series classification. We apply MILLET to existing deep learn- ing TSC models and show how they become inherently interpretable without com- promising (and in some cases, even improving) predictive performance. We eval- uate MILLET on 85 UCR TSC datasets and also present a novel synthetic dataset that is specially designed to facilitate interpretability evaluation. On these datasets, we show MILLET produces sparse explanations quickly that are of higher quality than other well-known interpretability methods. To the best of our knowledge, our work with MILLET, which is available on GitHub1, is the first to develop general MIL methods for TSC and apply them to an extensive variety of domains. Figure 1: Conventional TSC techniques (left) usually only provide class-level predictive probabili- ties. In addition, our proposed method (MILLET, right) also highlights class-conditional discrimi- natory motifs that influence the predicted class. In the heatmap, green regions indicate support for the predicted class, red regions refute the predicted class, and darker regions are more influential. 1 INTRODUCTION Time Series Classification (TSC) is the process of assigning labels to sequences of data, and oc- curs in a wide range of settings – examples from the popular UCR collection of datasets include predicting heart failure from electrocardiogram data, and identifying household electric appliance usage from electricity data (Dau et al., 2019). Each of these domains have their own set of class- conditional discriminatory motifs (the signatures that determine the class of a time series). Deep Learning (DL) methods have emerged as a popular family of approaches for solving TSC problems. However, we identify two drawbacks with these conventional supervised learning approaches: 1) representations are learnt for each time point in a time series, but these representations are then lost through an aggregation process that weights all time points equally, and 2) these methods are black boxes that provide no inherent explanations for their decision making, i.e. they cannot localise the class-conditional discriminatory motifs. These drawbacks not only limit predictive performance, but also introduce barriers to their adoption in practice as the models are not transparent. To mitigate these shortcomings, we take an alternative view of DL for TSC, approaching it as a Multiple Instance Learning (MIL) problem. MIL is a weakly supervised learning paradigm in which a collection (bag) of elements (MIL instances) all share the same label. In the context of TSC, a bag is a time series of data over a contiguous interval. In the MIL setting, the learning objective is to assign class labels to unlabelled bags of time series data whilst also discovering the salient motifs ∗University of Southampton, UK – work completed during an internship at Amazon Prime Video, UK. †Amazon Prime Video, UK 1https://github.com/JAEarly/MILTimeSeriesClassification 1 arXiv:2311.10049v3 [cs.LG] 16 Mar 2024 Published as a conference paper at ICLR 2024 within the time series that explain the reasons for the predicted class. As we explore in this work, MIL is well-suited to overcome the drawbacks identified above, leading to inherent interpretability without compromising predictive performance (even improving it in some cases). We propose a new general framework applying MIL to TSC called MILLET: Multiple Instance Learning for Locally Explainable Time series classification. Demonstrative MILLET model outputs are depicted in Fig. 1. MIL is well-suited to this problem setting since it was developed for weakly supervised contexts, can be learnt in an end-to-end framework, and boasts many successes across several domains. Fur- thermore, MIL has the same label specificity as TSC: labels are given at the bag level, but not at the MIL instance level. To explore the intersection of these two areas, we propose plug-and-play con- cepts that are adapted from MIL and applied to existing TSC approaches (in this work, DL models2). Furthermore, to aid in our evaluation of the interpretability of these new methods, we introduce a new synthetic TSC dataset, WebTraffic, where the location of the class-conditional discriminatory motifs within time series are known. The time series shown in Fig. 1 is sampled from this dataset. Our key contributions are as follows: 1. We propose MILLET, a TSC framework that utilises MIL to provide inherent interpretabil- ity without compromising predictive performance (even improving it in some cases). 2. We design plug-and-play MIL methods for TSC within MILLET. 3. We propose a new method of MIL aggregation, Conjunctive pooling, that outper- forms existing pooling methods in our TSC experiments. 4. We propose and evaluate 12 novel MILLET models on 85 univariate datasets from the UCR TSC Archive (Dau et al., 2019), as well as a novel synthetic dataset that facilitates better evaluation of TSC interpretability. 2 BACKGROUND AND RELATED WORK Time Series Classification While a range of TSC methods exist, in this work we apply MIL to DL TSC approaches. Methods in this family are effective and widely used (Ismail Fawaz et al., 2019; Foumani et al., 2023); popular methods include Fully Convolutional Networks (FCN), Residual Networks (ResNet), and InceptionTime (Wang et al., 2017; Ismail Fawaz et al., 2020). In- deed, a recent TSC survey, Bake Off Redux (Middlehurst et al., 2023), found InceptionTime to be competitive with SOTA approaches such as the ensemble method HIVE-COTE 2 (HC2; Middle- hurst et al., 2021) and the hybrid dictionary-convolutional method Hydra-MultiRocket (Hydra-MR; Dempster et al., 2023). Although the application of Matrix Profile for TSC also yields inherent in- terpretability (Yeh et al., 2017; Guidotti & D’Onofrio, 2021), we choose to focus on DL approaches due to their popularity, strong performance, and scope for improvement (Middlehurst et al., 2023). Multiple Instance Learning In its standard assumption, MIL is a binary classification problem: a bag is positive if and only if at least one of its instances is positive (Dietterich et al., 1997). As we are designing MILLET to be a general and widely applicable TSC approach, we do not constrain it to any specific MIL assumption except that there are temporal relationships, i.e. the order of instances within bags matters (Early et al., 2022; Wang et al., 2020). As we explore in Sec. 3.4, this allows us to use positional encodings in our MILLET methods. Although the application of MIL to TSC has been explored prior to this study, earlier work focused on domain-specific problems such as intensive care in medicine and human activity recognition (Dennis et al., 2018; Janakiraman, 2018; Poyiadzi et al., 2018; Poyiadzis et al., 2019; Shanmugam et al., 2019). Furthermore, existing work considers MIL as its own unique approach separate from existing TSC methods. The work most closely related to ours is Zhu et al. (2021), which proposes an uncertainty-aware MIL TSC framework specifically designed for long time series (marine vessel tracking), but without the generality and plug-and-play nature of MILLET. Therefore, to the best of our knowledge, our work with MILLET is the first to apply MIL to TSC in a more general sense and to do so across an extensive variety of domains. Interpretability TSC interpretability methods can be grouped into several categories (Theissler et al., 2022) – in this work we focus on class-wise time point attribution (saliency maps), i.e. identi- fying the discriminatory time points in a time series that support and refute different classes. This is a form of local interpretation, where model decision-making is explained for individual time series 2While we focus on DL TSC in this work, we envision that our MILLET framework can be applied to other TSC approaches in the future, such as the ROCKET family of methods (Dempster et al., 2020; 2023). 2 Published as a conference paper at ICLR 2024 (Molnar, 2022). It also aligns with MIL interpretability as proposed by Early et al. (2021): which are the key MIL instances in a bag, and what outcomes do they support/refute? MILLET facilitates interpretability by inherently enhancing existing TSC approaches such that they provide interpre- tations alongside their predictions with a single forward pass of the model. This is in contrast to perturbation methods such as LIME (Ribeiro et al., 2016), SHAP (Lundberg & Lee, 2017), Oc- clusion Sensitivity (Zeiler & Fergus, 2014), and MILLI (Early et al., 2021), which are much more expensive to run (often requiring 100+ forward passes per interpretation). An interpretability ap- proach that can be run with a single forward pass is Class Activation Mapping (CAM) (Zhou et al., 2016; Wang et al., 2017). It uses the model’s weights to identify discriminatory time points, and serves as a benchmark in this work. For more details on existing TSC interpretability methods and their evaluation metrics, see App. A.1, Theissler et al. (2022), and ˇSimi´c et al. (2021). 3 METHODOLOGY To apply MIL to TSC, we propose the broad framework MILLET: Multiple Instance Learning for Locally Explainable Time series classification. We advocate for the use of MIL in TSC as it is a natural fit that provides inherent interpretability (explanations for free) without requiring any additional labelling beyond that already provided by existing TSC datasets (we further discuss our motivation for using MIL in App. A). 3.1 THE MILLET FRAMEWORK A TSC model within our MILLET framework has to satisfy three requirements: Requirement 1: Time Series as MIL Bags Input data consists of time series, Xi, where a time series is formed of t > 1 time points: Xi = {x1 i , x2 i , . . . , xt i} and i is the sample index.3 Each time step is a c-dimensional vector, where c is the number of channels in the time series – in this work we focus on univariate time series (c = 1) and assume all time series in a dataset have the same length. We consider each time series as a MIL bag, meaning each time point is a MIL instance.4 A time series bag can be denoted as Xi ∈Rt×c, and each bag has an associated bag-level label Yi which is the original time series label. There is also the concept of MIL instance labels {y1 i , y2 i , . . . , yt i}, but these are not provided for most MIL datasets (like the absence of time