MVFA Visual Feature Adaptation for Medical Anomaly Detection
A conference poster presenting MVFA, a multi-level visual feature adaptation framework for anomaly detection in medical images using pre-trained visual-language models like CLIP, demonstrating superior performance in few-shot and zero-shot settings across diverse modalities.
Paper context
Paper title: Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images Abstract: A conference poster presenting MVFA, a multi-level visual feature adaptation framework for anomaly detection in medical images using pre-trained visual-language models like CLIP, demonstrating superior performance in few-shot and zero-shot settings across diverse modalities. Paper body (method & results): <!DOCTYPE html> <html lang="en"> <head> <meta content="text/html; charset=utf-8" http-equiv="content-type"/> <title>Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images</title> <!--Generated on Tue Mar 19 09:27:28 2024 by LaTeXML (version 0.8.7) http://dlmf.nist.gov/LaTeXML/.--> <meta content="width=device-width, initial-scale=1, shrink-to-fit=no" name="viewport"/> <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/css/bootstrap.min.css" rel="stylesheet" type="text/css"/> <link href="/static/browse/0.3.4/css/ar5iv_0.7.4.min.css" rel="stylesheet" type="text/css"/> <link href="/static/browse/0.3.4/css/latexml_styles.css" rel="stylesheet" type="text/css"/> <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/js/bootstrap.bundle.min.js"></script> <script src="https://cdnjs.cloudflare.com/ajax/libs/html2canvas/1.3.3/html2canvas.min.js"></script> <script src="/static/browse/0.3.4/js/addons.js"></script> <script src="/static/browse/0.3.4/js/feedbackOverlay.js"></script> <base href="/html/2403.12570v1/"/></head> <body> <nav class="ltx_page_navbar"> <nav class="ltx_TOC"> <ol class="ltx_toclist"> <li class="ltx_tocentry ltx_tocentry_section"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#S1" title="1 Introduction ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">1 </span>Introduction</span></a></li> <li class="ltx_tocentry ltx_tocentry_section"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#S2" title="2 Related Works ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">2 </span>Related Works</span></a></li> <li class="ltx_tocentry ltx_tocentry_section"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#S3" title="3 Problem Formulation ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">3 </span>Problem Formulation</span></a></li> <li class="ltx_tocentry ltx_tocentry_section"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#S4" title="4 Train: Multi-Level Feature Adaptation ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">4 </span>Train: Multi-Level Feature Adaptation</span></a></li> <li class="ltx_tocentry ltx_tocentry_section"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#S5" title="5 Test: Multi-Level Feature Comparison ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">5 </span>Test: Multi-Level Feature Comparison</span></a></li> <li class="ltx_tocentry ltx_tocentry_section"> <a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#S6" title="6 Experiments ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">6 </span>Experiments</span></a> <ol class="ltx_toclist ltx_toclist_section"> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#S6.SS1" title="6.1 Experimental Setups ‣ 6 Experiments ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">6.1 </span>Experimental Setups</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#S6.SS2" title="6.2 Comparison with State-of-the-art Methods ‣ 6 Experiments ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">6.2 </span>Comparison with State-of-the-art Methods</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#S6.SS3" title="6.3 Ablation Studies ‣ 6 Experiments ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">6.3 </span>Ablation Studies</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#S6.SS4" title="6.4 Visualization Analysis ‣ 6 Experiments ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">6.4 </span>Visualization Analysis</span></a></li> </ol> </li> <li class="ltx_tocentry ltx_tocentry_section"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#S7" title="7 Conclusion ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">7 </span>Conclusion</span></a></li> <li class="ltx_tocentry ltx_tocentry_appendix"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#A1" title="Appendix A Medical Anomaly Detection Benchmark ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">A </span>Medical Anomaly Detection Benchmark</span></a></li> <li class="ltx_tocentry ltx_tocentry_appendix"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#A2" title="Appendix B Text Prompt Formatting ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">B </span>Text Prompt Formatting</span></a></li> <li class="ltx_tocentry ltx_tocentry_appendix"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#A3" title="Appendix C Additional Quantitative Results ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">C </span>Additional Quantitative Results</span></a></li> <li class="ltx_tocentry ltx_tocentry_appendix"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#A4" title="Appendix D Additional Qualitative Results ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">D </span>Additional Qualitative Results</span></a></li> <li class="ltx_tocentry ltx_tocentry_appendix"><a class="ltx_ref" href="https://arxiv.org/html/2403.12570v1#A5" title="Appendix E Ablation Model Structure ‣ Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">E </span>Ablation Model Structure</span></a></li> </ol></nav> </nav> <div class="ltx_page_main"> <div class="ltx_page_content"> <div aria-label="Conversion errors have been found" class="package-alerts ltx_document" role="status"> <button aria-label="Dismiss alert" onclick="closePopup()"> <span aria-hidden="true"><svg aria-hidden="true" focusable="false" height="20" role="presentation" viewbox="0 0 44 44" width="20"> <path d="M0.549989 4.44999L4.44999 0.549988L43.45 39.55L39.55 43.45L0.549989 4.44999Z"></path> <path d="M39.55 0.549988L43.45 4.44999L4.44999 43.45L0.549988 39.55L39.55 0.549988Z"></path> </svg></span> </button> <p>HTML conversions <a href="https://info.dev.arxiv.org/about/accessibility_html_error_messages.html" target="_blank">sometimes display errors</a> due to content that did not convert correctly from the source. This paper uses the following packages that are not yet supported by the HTML conversion tool. Feedback on these issues are not necessary; they are known and are being worked on.</p> <ul arial-label="Unsupported packages used in this paper"> <li>failed: epic</li> </ul> <p>Authors: achieve the best HTML results from your LaTeX submissions by following these <a href="https://info.arxiv.org/help/submit_latex_best_practices.html" target="_blank">best practices</a>.</p> </div><div class="section" id="target-section"><div id="license-tr">License: arXiv.org perpetual non-exclusive license</div><div id="watermark-tr">arXiv:2403.12570v1 [cs.CV] 19 Mar 2024</div></div> <script> function closePopup() { document.querySelector('.package-alerts').style.display = 'none'; } </script> <article class="ltx_document ltx_authors_1line"> <h1 class="ltx_title ltx_title_document">Adapting Visual-Language Models for <br class="ltx_break"/>Generalizable Anomaly Detection in Medical Images</h1> <div class="ltx_authors"> <span class="ltx_creator ltx_role_author"> <span class="ltx_personname">Chaoqin Huang<sup class="ltx_sup" id="id2.2.id1">1,2,3</sup>, Aofan Jiang<sup class="ltx_sup" id="id3.3.id2">1,3</sup><math alttext="{}^{*}" class="ltx_Math" display="inline" id="id1.1.m1.1"><semantics id="id1.1.m1.1a"><msup id="id1.1.m1.1.1" xref="id1.1.m1.1.1.cmml"><mi id="id1.1.m1.1.1a" xref="id1.1.m1.1.1.cmml"></mi><mo id="id1.1.m1.1.1.1" xref="id1.1.m1.1.1.1.cmml">*</mo></msup><annotation-xml encoding="MathML-Content" id="id1.1.m1.1b"><apply id="id1.1.m1.1.1.cmml" xref="id1.1.m1.1.1"><times id="id1.1.m1.1.1.1.cmml" xref="id1.1.m1.1.1.1"></times></apply></annotation-xml><annotation encoding="application/x-tex" id="id1.1.m1.1c">{}^{*}</annotation><annotation encoding="application/x-llamapun" id="id1.1.m1.1d">start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPT</annotation></semantics></math>, Jinghao Feng<sup class="ltx_sup" id="id4.4.id3">1,3</sup>, Ya Zhang<sup class="ltx_sup" id="id5.5.id4">1,3</sup>, Xinchao Wang<sup class="ltx_sup" id="id6.6.id5">2,</sup>, Yanfeng Wang<sup class="ltx_sup" id="id7.7.id6">1,3,<span class="ltx_note ltx_role_footnotemark" id="footnotex1"><sup class="ltx_note_mark">†</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">†</sup><span class="ltx_note_type">footnotemark: </span></span></span></span></sup> <br class="ltx_break"/><sup class="ltx_sup" id="id8.8.id7">1</sup> Shanghai Jiao Tong University, <sup class="ltx_sup" id="id9.9.id8">2</sup> National University of Singapore <br class="ltx_break"/><sup class="ltx_sup" id="id10.10.id9">3</sup> Shanghai Artificial Intelligence Laboratory <br class="ltx_break"/><span class="ltx_text ltx_font_typewriter" id="id11.11.id10" style="font-size:70%;">{huangchaoqin,stillunnamed,fjh1345528968,ya_zhang,wangyanfeng622}@sjtu.edu.cn; {xinchao}@nus.edu.sg </span> </span><span class="ltx_author_notes">Equal ContributionCorresponding authors: Yanfeng Wang and Xinchao Wang</span></span> </div> <div class="ltx_abstract"> <h6 class="ltx_title ltx_title_abstract">Abstract</h6> <p class="ltx_p" id="id12.id1">Rece
