VAAL: Variational Adversarial Active Learning

论文上下文

Paper title: Learning with not Enough Data Part 2: Active Learning | Lil'Log Abstract: The figure illustrates the architecture and data flow of the Variational Adversarial Active Learning (VAAL) framework. Source body: Table of Contents Notations What is Active Learning? Acquisition Function Uncertainty Sampling Diversity Sampling Expected Model Change Hybrid Strategy Deep Acquisition Function Measuring Uncertainty Ensemble and Approximated Ensemble Uncertainty in Parameter Space Loss Prediction Adversarial Setup Measuring Representativeness Core-sets Approach Diverse Gradient Embedding Measuring Training Effects Quantify Model Changes Forgetting Events Hybrid Citation References This is part 2 of what to do when facing a limited amount of labeled data for supervised learning tasks. This time we will get some amount of human labeling work involved, but within a budget limit, and therefore we need to be smart when selecting which samples to label. Notations # Symbol Meaning $K$ Number of unique class labels. $(\mathbf{x}^l, y) \sim \mathcal{X}, y \in \{0, 1\}^K$ Labeled dataset. $y$ is a one-hot representation of the true label. $\mathbf{u} \sim \mathcal{U}$ Unlabeled dataset. $\mathcal{D} = \mathcal{X} \cup \mathcal{U}$ The entire dataset, including both labeled and unlabeled examples. $\mathbf{x}$ Any sample which can be either labeled or unlabeled. $\mathbf{x}_i$ The $i$-th sample. $U(\mathbf{x})$ Scoring function for active learning selection. $P_\theta(y \vert \mathbf{x})$ A softmax classifier parameterized by $\theta$. $\hat{y} = \arg\max_{y \in \mathcal{Y}} P_\theta(y \vert \mathbf{x})$ The most confident prediction by the classifier. $B$ Labeling budget (the maximum number of samples to label). $b$ Batch size. What is Active Learning? # Given an unlabeled dataset $\mathcal{U}$ and a fixed amount of labeling cost $B$, active learning aims to select a subset of $B$ examples from $\mathcal{U}$ to be labeled such that they can result in maximized improvement in model performance. This is an effective way of learning especially when data labeling is difficult and costly, e.g. medical images. This classical survey paper in 2010 lists many key concepts. While some conventional approaches may not apply to deep learning, discussion in this post mainly focuses on deep neural models and training in batch mode. Illustration of a cyclic workflow of active learning, producing better models more efficiently by smartly choosing which samples to label. To simplify the discussion, we assume that the task is a $K$-class classification problem in all the following sections. The model with parameters $\theta$ outputs a probability distribution over the label candidates, which may or may not be calibrated, $P_\theta(y \vert \mathbf{x})$ and the most likely prediction is $\hat{y} = \arg\max_{y \in \mathcal{Y}} P_\theta(y \vert \mathbf{x})$. Acquisition Function # The process of identifying the most valuable examples to label next is referred to as “sampling strategy” or “query strategy”. The scoring function in the sampling process is named “acquisition function”, denoted as $U(\mathbf{x})$. Data points with higher scores are expected to produce higher value for model training if they get labeled. Here is a list of basic sampling strategies. Uncertainty Sampling # Uncertainty sampling selects examples for which the model produces most uncertain predictions. Given a single model, uncertainty can be estimated by the predicted probabilities, although one common complaint is that deep learning model predictions are often not calibrated and not correlated with true uncertainty well. In fact, deep learning models are often overconfident. Least confident score , also known as variation ratio : $U(\mathbf{x}) = 1 - P_\theta(\hat{y} \vert \mathbf{x})$. Margin score : $U(\mathbf{x}) = P_\theta(\hat{y}_1 \vert \mathbf{x}) - P_\theta(\hat{y}_2 \vert \mathbf{x})$, where $\hat{y}_1$ and $\hat{y}_2$ are the most likely and the second likely predicted labels. Entropy : $U(\mathbf{x}) = \mathcal{H}(P_\theta(y \vert \mathbf{x})) = - \sum_{y \in \mathcal{Y}} P_\theta(y \vert \mathbf{x}) \log P_\theta(y \vert \mathbf{x})$. Another way to quantify uncertainty is to rely on a committee of expert models, known as Query-By-Committee (QBC). QBC measures uncertainty based on a pool of opinions and thus it is critical to keep a level of disagreement among committee members. Given $C$ models in the committee pool, each parameterized by $\theta_1, \dots, \theta_C$. Voter entropy : $U(\mathbf{x}) = \mathcal{H}(\frac{V(y)}{C})$, where $V(y)$ counts the number of votes from the committee on the label $y$. Consensus entropy : $U(\mathbf{x}) = \mathcal{H}(P_\mathcal{C})$, where $P_\mathcal{C}$ is the prediction averaging across the committee. KL divergence : $U(\mathbf{x}) = \frac{1}{C} \sum_{c=1}^C D_\text{KL} (P_{\theta_c} | P_\mathcal{C})$ Diversity Sampling # Diversity sampling intend to find a collection of samples that can well represent the entire data distribution. Diversity is important because the model is expected to work well on any data in the wild, just not on a narrow subset. Selected samples should be representative of the underlying distribution. Common approaches often rely on quantifying the similarity between samples. Expected Model Change # Expected model change refers to the impact that a sample brings onto the model training. The impact can be the influence on the model weights or the improvement over the training loss. A later section reviews several works on how to measure model impact triggered by selected data samples. Hybrid Strategy # Many methods above are not mutually exclusive. A hybrid sampling strategy values different attributes of data points, combining different sampling preferences into one. Often we want to select uncertain but also highly representative samples. Deep Acquisition Function # Measuring Uncertainty # The model uncertainty is commonly categorized into two buckets ( Der Kiureghian & Ditlevsen 2009 , Kendall & Gal 2017 ): Aleatoric uncertainty is introduced by noise in the data (e.g. sensor data, noise in the measurement process) and it can be input-dependent