DFA Class Bias in Cross-Attention Layers

论文上下文

Paper title: Learning Class Difficulty in Imbalanced Histopathology Segmentation via Dynamic Focal Attention Abstract: Semantic segmentation of histopathology images under class imbalance is typically addressed through frequency-based loss reweighting, which implicitly assumes that rare classes are difficult. However, true difficulty also arises from morphological variability, boundary ambiguity, and contextual similarity-factors that frequency cannot capture. We propose Dynamic Focal Attention (DFA), a simple and efficient mechanism that learns class-specific difficulty directly within the cross-attention of query-based mask decoders. DFA introduces a learnable per-class bias to attention logits, enabling representation-level reweighting prior to prediction rather than gradient-level reweighting after prediction. Initialised from a log-frequency prior to prevent gradient starvation, the bias is optimised end-to-end, allowing the model to adaptively capture difficulty signals through training, effectively unifying frequency-based and difficulty-aware approaches under a common attention-bias framework. On three histopathology benchmarks (BDSA, BCSS, CRAG), DFA consistently improves Dice and IoU, matching or exceeding a difficulty-aware baseline without a separate estimator or additional training stage. These results demonstrate that encoding class difficulty at the representation level provides a principled alternative to conventional loss reweighting for imbalanced segmentation. Passages referencing this figure: thology [ 3 , 16 , 12 ] . Transformer-based architectures [ 7 , 6 ] achieve strong overall performance yet consistently underperform on clinically critical classes—a gap widely attributed to class imbalance. However, imbalance alone is incomplete: difficulty is also governed by morphological variability, boundary ambiguity, and contextual similarity, factors largely orthogonal to class frequency. Figure 1 : DFA integrated into cross-attention. A learnable class-specific bias b c b_{c} is added to attention logits before softmax, initialised from a log-frequency prior and optimised end-to-end via ℒ \mathcal{L} (Eq. 7 ). Limitations of existing approaches. Data-level strategies [ 2 , 14 ] alter training distributions without introducing semantic information, while loss-level methods [ 15 , 2