Implicit Variant Architecture with InternVL3 and LoRA
Figure 2: Architecture of the implicit variant. The frozen InternVL3-8B-Instruct backbone carries three stage-specific LoRA adapters. After each stage, a gated context projector extracts the hidden state at the final prompt token and injects a normalized, gated projection into the input embeddings of the next stage. The full three-stage chain executes in a single inference pipeline.
论文上下文
Paper title: Cross-Stage Coherence in Hierarchical Driving VQA: Explicit Baselines and Learned Gated Context Projectors Abstract: Graph Visual Question Answering (GVQA) for autonomous driving organizes reasoning into ordered stages, namely Perception, Prediction, and Planning, where planning decisions should remain consistent with the model's own perception. We present a comparative study of cross-stage context passing on DriveLM-nuScenes using two complementary mechanisms. The explicit variant evaluates three prompt-based conditioning strategies on a domain-adapted 4B VLM (Mini-InternVL2-4B-DA-DriveLM) without additional training, reducing NLI contradiction by up to 42.6% and establishing a strong zero-training baseline. The implicit variant introduces gated context projectors, which extract a hidden-state vector from one stage and inject a normalized, gated projection into the next stage's input embeddings. These projectors are jointly trained with stage-specific QLoRA adapters on a general-purpose 8B VLM (InternVL3-8B-Instruct) while updating only approximately 0.5% of parameters. The implicit variant achieves a statistically significant 34% reduction in planning-stage NLI contradiction (bootstrap 95% CIs, p < 0.05) and increases cross-stage entailment by 50%, evaluated with a multilingual NLI classifier to account for mixed-language outputs. Planning language quality also improves (CIDEr +30.3%), but lexical overlap and structural consistency degrade due to the absence of driving-domain pretraining. Since the two variants use different base models, we present them as complementary case studies: expl Passages referencing this figure: e autoregressive context window, which is not trainable as a routing mechanism. Stage-specific adapters Jia et al. ( 2023 ) train each adapter independently, making cross-stage coherence accidental. What is missing is a trainable mechanism that propagates a compact semantic state from one reasoning stage to the next. We approach this through two complementary forms of cross-stage context passing (Figure 1 ). In the explicit form, context passes between stages as generated text through one of three prompt-based conditions; no training is involved. In the implicit form, a compact vector from the VLM’s last-layer hidden states is projected and gated into the next stage’s input embeddings through a learned linear module, within a single inference pipeline. The explicit variant uses Mini-Intern rence pipeline. The explicit variant uses Mini-InternVL2-4B-DA-DriveLM OpenGVLab ( 2024 ); Gao et al. ( 2024 ) , a domain-adapted model; the implicit variant uses InternVL3-8B-Instruct OpenGVLab ( 2025 ); Zhu et al. ( 2025 ) , a general-purpose VLM without driving pretraining. Results across variants are not directly comparable; we report them as complementary case studies. Our contributions are: Figure 1: Context passing mechanisms for hierarchical driving VQA. (a) Shared setup: six surround-view cameras and three stage-specific questions. (b) Flat: each stage processes image and question independently. (c) History-chain: prior answers flow as conversational history. (d) Injection-chain: prior answers are prepended as structured text prefixes. (e) Proposed framework: learned context vecto ters. The vision encoder and all non-adapter parameters remain frozen. Dataset. Both variants use DriveLM-nuScenes Sima et al. ( 2023 ); Caesar et al. ( 2019 ) (v1.1, ∼ 3 , 200 {\sim}3{,}200 keyfram
