Document Agentic RAG Workflow Overview
Figure 3: Document agentic workflow. A planning agent uses a metadata schema to generate sub-queries, an end-to-end RAG system answers single-document queries, then responses are normalized to JSON and analyzed by code to obtain the final answer.
Paper context
Paper title: Navigating Large-Scale Document Collections: MuDABench for Multi-Document Analytical QA Abstract: This paper introduces the task of analytical question answering over large, semi-structured document collections. We present MuDABench, a benchmark for multi-document analytical QA, where questions require extracting and synthesizing information across numerous documents to perform quantitative analysis. Unlike existing multi-document QA benchmarks that typically require information from only a few documents with limited cross-document reasoning, MuDABench demands extensive inter-document analysis and aggregation. Constructed via distant supervision by leveraging document-level metadata and annotated financial databases, MuDABench comprises over 80,000 pages and 332 analytical QA instances. We also propose an evaluation protocol that measures final answer accuracy and uses intermediate-fact coverage as an auxiliary diagnostic signal for the reasoning process. Experiments reveal that standard RAG systems, which treat all documents as a flat retrieval pool, perform poorly. To address these limitations, we propose a multi-agent workflow that orchestrates planning, extraction, and code generation modules. While this approach substantially improves both process and outcome metrics, a significant gap remains compared to human expert performance. Our analysis identifies two primary bottlenecks: single-document information extraction accuracy and insufficient domain-specific knowledge in current systems. MuDABench is available at https://github.com/Zhanli-Li/MuDABench. Passages referencing this figure: e context window and then answer the query in one or a few model calls. Wikipedia-style multi-hop datasets such as HotpotQA and its successors Yang et al. ( 2018b ); Ho et al. ( 2020 ); Trivedi et al. ( 2022 ); Zhu et al. ( 2024 ); Levy et al. ( 2025 ) instantiate this view, and recent work on long-context benchmarks extends it to longer inputs without changing the underlying interaction pattern. Figure 1: An example of multi-doc analytical QA. The collection of documents behind a question is organized into a semi-structured database through metadata, and answering the question involves first identifying which documents are useful and then targeting the information extraction for the final aggregated answer. Dataset Doc / Q Pages / Doc Multihop Metadata Multilingual Realistic Publicly rele ate announcements of listed companies to detect abnormal changes in accounting firms or risk indicators; researchers survey hundreds of papers to construct performance tables over datasets and tasks; and public-sector agencies aggregate heterogeneous reports to audit policy outcomes. In these settings, missing one relevant document or misinterpreting one table can invalidate the final conclusion. Figure 1 illustrates an example of analytical QA that is of critical concern to financial regulators. The underlying data consists of annual reports from multiple companies over several years, and the question asks which companies changed their accounting firms in 2024, as this may signal significant financial changes. To answer this question, the required steps include: filtering all company annu
