A conference poster presenting DSPO, a score-matching-based alignment algorithm for diffusion models. It compares DSPO against baselines like Diffusion-DPO using win-rate charts and performance tables on Stable Diffusion 1.5 and XL across datasets like PickV2 and HPSV2.
Paper title: DSPO: Direct Score Preference Optimization for Diffusion Model Alignment Abstract: A conference poster presenting DSPO, a score-matching-based alignment algorithm for diffusion models. It compares DSPO against baselines like Diffusion-DPO using win-rate charts and performance tables on Stable Diffusion 1.5 and XL across datasets like PickV2 and HPSV2. Paper body (method & results): <!DOCTYPE html> <html lang="en"> <head> <meta content="text/html; charset=utf-8" http-equiv="content-type"/> <title>Divergence Minimization Preference Optimization for Diffusion Model Alignment</title> <!--Generated on Thu Jul 10 07:55:38 2025 by LaTeXML (version 0.8.8) http://dlmf.nist.gov/LaTeXML/.--> <meta content="width=device-width, initial-scale=1, shrink-to-fit=no" name="viewport"/> <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/css/bootstrap.min.css" rel="stylesheet" type="text/css"/> <link href="/static/browse/0.3.4/css/ar5iv.0.7.9.min.css" rel="stylesheet" type="text/css"/> <link href="/static/browse/0.3.4/css/ar5iv-fonts.0.7.9.min.css" rel="stylesheet" type="text/css"/> <link href="/static/browse/0.3.4/css/latexml_styles.css" rel="stylesheet" type="text/css"/> <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/js/bootstrap.bundle.min.js"></script> <script src="https://cdnjs.cloudflare.com/ajax/libs/html2canvas/1.3.3/html2canvas.min.js"></script> <script src="/static/browse/0.3.4/js/addons_new.js"></script> <script src="/static/browse/0.3.4/js/feedbackOverlay.js"></script> <base href="/html/2507.07510v1/"/></head> <body> <nav class="ltx_page_navbar"> <nav class="ltx_TOC"> <ol class="ltx_toclist"> <li class="ltx_tocentry ltx_tocentry_section"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S1" title="In Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">1 </span>Introduction</span></a></li> <li class="ltx_tocentry ltx_tocentry_section"> <a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S2" title="In Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">2 </span>Background</span></a> <ol class="ltx_toclist ltx_toclist_section"> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S2.SS1" title="In 2 Background ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">2.1 </span>Diffusion Models</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S2.SS2" title="In 2 Background ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">2.2 </span>Reinforcement Learning from Human Feedback (RLHF)</span></a></li> </ol> </li> <li class="ltx_tocentry ltx_tocentry_section"> <a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S3" title="In Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">3 </span>Method</span></a> <ol class="ltx_toclist ltx_toclist_section"> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S3.SS1" title="In 3 Method ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">3.1 </span>Revisiting Diffusion-DPO</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S3.SS2" title="In 3 Method ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">3.2 </span>Our Method</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S3.SS3" title="In 3 Method ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">3.3 </span>Theoretical Analysis</span></a></li> </ol> </li> <li class="ltx_tocentry ltx_tocentry_section"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S4" title="In Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">4 </span>Related Work</span></a></li> <li class="ltx_tocentry ltx_tocentry_section"> <a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S5" title="In Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">5 </span>Experiments</span></a> <ol class="ltx_toclist ltx_toclist_section"> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S5.SS1" title="In 5 Experiments ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">5.1 </span>Experimental Setting</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S5.SS2" title="In 5 Experiments ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">5.2 </span>Alignment Result</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S5.SS3" title="In 5 Experiments ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">5.3 </span>Image Editing Results</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S5.SS4" title="In 5 Experiments ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">5.4 </span>Ablation Study</span></a></li> </ol> </li> <li class="ltx_tocentry ltx_tocentry_section"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#S6" title="In Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">6 </span>Conclusion</span></a></li> <li class="ltx_tocentry ltx_tocentry_appendix"> <a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#A1" title="In Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">A </span>Proofs and Derivations</span></a> <ol class="ltx_toclist ltx_toclist_appendix"> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#A1.SS1" title="In Appendix A Proofs and Derivations ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">A.1 </span>Proof of <span class="ltx_text ltx_ref_tag">Theorem</span> <span class="ltx_text ltx_ref_tag">1</span></span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#A1.SS2" title="In Appendix A Proofs and Derivations ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">A.2 </span>Convexity of <span class="ltx_text ltx_ref_tag">Equation</span> <span class="ltx_text ltx_ref_tag">12</span></span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#A1.SS3" title="In Appendix A Proofs and Derivations ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">A.3 </span>Proof of <span class="ltx_text ltx_ref_tag">Theorem</span> <span class="ltx_text ltx_ref_tag">2</span></span></a></li> </ol> </li> <li class="ltx_tocentry ltx_tocentry_appendix"> <a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#A2" title="In Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">B </span>Experimental Details</span></a> <ol class="ltx_toclist ltx_toclist_appendix"> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#A2.SS1" title="In Appendix B Experimental Details ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">B.1 </span>Details of Evaluation</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#A2.SS2" title="In Appendix B Experimental Details ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">B.2 </span>Details of Datasets</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#A2.SS3" title="In Appendix B Experimental Details ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">B.3 </span>Pseudo code of DMPO</span></a></li> </ol> </li> <li class="ltx_tocentry ltx_tocentry_appendix"> <a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#A3" title="In Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">C </span>More Experimental Results</span></a> <ol class="ltx_toclist ltx_toclist_appendix"> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#A3.SS1" title="In Appendix C More Experimental Results ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">C.1 </span>More Qualitative Results</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="https://arxiv.org/html/2507.07510v1#A3.SS2" title="In Appendix C More Experimental Results ‣ Divergence Minimization Preference Optimization for Diffusion Model Alignment"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">C.2 </span>More Quantitative Results</span></a></li> </ol> </li> </ol></nav> </nav> <div class="ltx_page_main"> <div class="ltx_page_content"> <article class="ltx_document ltx_authors_1line"><span class="ltx_note ltx_role_footnotetext" id="footnotex1"><sup class="ltx_note_mark">1</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">1</sup><span class="ltx_note_type">footnotetext: </span>Equal contribution; junior auth