MOMENTA Architecture: Pipeline and Optimization
Figure 1: MOMENTA architecture: full pipeline and optimization strategy. The framework integrates pretrained encoders (XLM-RoBERTa and CLIP), intra-modal self-attention, and modality-specific Mixture-of-Experts (MoE). Bidirectional co-attention aligns text and image representations, followed by gated fusion and a discrepancy branch. Temporal aggregation captures narrative evolution using time-decayed attention, drift, and momentum. The model is trained using a multi-objective loss combining classification, alignment, contrastive learning, temporal consistency, domain-adversarial training, and
Paper context
Paper title: MOMENTA: Mixture-of-Experts Over Multimodal Embeddings with Neural Temporal Aggregation for Misinformation Detection Abstract: The widespread dissemination of multimodal content on social media has made misinformation detection increasingly challenging, as misleading narratives often arise not only from textual or visual content alone, but also from semantic inconsistencies between modalities and their evolution over time. Existing multimodal misinformation detection methods typically model cross-modal interactions statically and often show limited robustness across heterogeneous datasets, domains, and narrative settings. To address these challenges, we propose MOMENTA, a unified framework for multimodal misinformation detection that captures modality heterogeneity, cross-modal inconsistency, temporal dynamics, and cross-domain generalization within a single architecture. MOMENTA employs modality-specific mixture-of-experts modules to model diverse misinformation patterns, bidirectional co-attention to align textual and visual representations in a shared semantic space, and a discrepancy-aware branch to explicitly capture semantic disagreement between modalities. To model narrative evolution, we introduce an attention-based temporal aggregation mechanism with drift and momentum encoding over overlapping time windows, enabling the framework to capture both short-term fluctuations and longer-term trends in misinformation propagation. In addition, domain-adversarial learning and a prototype memory bank improve domain invariance and stabilize representation learning across datasets. The model is trained Passages referencing this figure: nin et al. [ 2016 ] ; and (8) an EMA prototype memory bank that maintains per-dataset, per-class prototypes and aligns embeddings to dataset-invariant global class centroids. Optionally, a timestamp-aware Transformer over the temporal sequence and a Transformer temporal consistency loss refine the temporal modeling. The following subsections detail each component and its mathematical formulation. Figure 1: MOMENTA architecture: full pipeline and optimization strategy. The framework integrates pretrained encoders (XLM-RoBERTa and CLIP), intra-modal self-attention, and modality-specific Mixture-of-Experts (MoE). Bidirectional co-attention aligns text and image representations, followed by gated fusion and a discrepancy branch. Temporal aggregation captures narrative evolution using time-deca co-attention aligns text and image representations, followed by gated fusion and a discrepancy branch. Temporal aggregation captures narrative evolution using time-decayed attention, drift, and momentum. The model is trained using a multi-objective loss combining classification, alignment, contrastive learning, temporal consistency, domain-adversarial training, and prototype-based regularization. Figure 1 illustrates the overall architecture of the MOMENTA framework. The model processes multimodal inputs through parallel text and image encoders, followed by intra-modal refinement and expert-based specialization. Cross-modal relationships are modeled through bidirectional co-attention and a discrepancy-aware fusion mechanism, allowing the model to capture both semantic alignment and inconsi
