CAMP-VQA Quality-Aware Captioning for Video Quality

논문 컨텍스트

Paper title: CAMP-VQA: Caption-Embedded Multimodal Perception for No-Reference Quality Assessment of Compressed Video Abstract: The prevalence of user-generated content (UGC) on platforms such as YouTube and TikTok has rendered no-reference (NR) perceptual video quality assessment (VQA) vital for optimizing video delivery. Nonetheless, the characteristics of non-professional acquisition and the subsequent transcoding of UGC video on sharing platforms present significant challenges for NR-VQA. Although NR-VQA models attempt to infer mean opinion scores (MOS), their modeling of subjective scores for compressed content remains limited due to the absence of fine-grained perceptual annotations of artifact types. To address these challenges, we propose CAMP-VQA, a novel NR-VQA framework that exploits the semantic understanding capabilities of large vision-language models. Our approach introduces a quality-aware prompting mechanism that integrates video metadata (e.g., resolution, frame rate, bitrate) with key fragments extracted from inter-frame variations to guide the BLIP-2 pretraining approach in generating fine-grained quality captions. A unified architecture has been designed to model perceptual quality across three dimensions: semantic alignment, temporal characteristics, and spatial characteristics. These multimodal features are extracted and fused, then regressed to video quality scores. Extensive experiments on a wide variety of UGC datasets demonstrate that our model consistently outperforms existing NR-VQA methods, achieving improved accuracy without the need for costly manual fine-grained annota Passages referencing this figure: Figure 1 : CAMP-VQA automatically generates quality-aware captions and fuses spatial, temporal, and semantic features to predict video quality. The CAMP-VQA model, as illustrated in Fig. 1 , comprises three feature components: the Semantic artifact Embedding Extractor (SEE), which extracts semantic artifact embeddings aligned with visual information; the Temporal Motion Extractor (TME), built on SlowFast [ feichtenhofer2019slowfast ] as the backbone; and the Spatial Vision Extractor (SVE), built on Swin-Large [ liu2021swin ] . 3 Proposed Framework The basic concepts and components of the CAMP-VQA framework (see Fig. 1 ) are introduced in this section.