Long-Term 3D Dynamics Parsing Pipeline — Poster
A conference poster presenting a method for parsing long-term dynamics in 3D environments. It details a unified pipeline for instance matching, registration, and reconstruction using equivariant embeddings, alongside experimental results on synthetic and real-world datasets showing state-of-the-art performance.
Paper context
Paper title: Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments Abstract: A conference poster presenting a method for parsing long-term dynamics in 3D environments. It details a unified pipeline for instance matching, registration, and reconstruction using equivariant embeddings, alongside experimental results on synthetic and real-world datasets showing state-of-the-art performance. Paper body (method & results): <!DOCTYPE html> <html lang="en"> <head> <meta content="text/html; charset=utf-8" http-equiv="content-type"/> <title>Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments</title> <!--Generated on Thu Dec 14 17:08:54 2023 by LaTeXML (version 0.8.7) http://dlmf.nist.gov/LaTeXML/.--> <meta content="width=device-width, initial-scale=1, shrink-to-fit=no" name="viewport"/> <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/css/bootstrap.min.css" rel="stylesheet" type="text/css"/> <link href="/static/browse/0.3.4/css/ar5iv_0.7.4.min.css" rel="stylesheet" type="text/css"/> <link href="/static/browse/0.3.4/css/latexml_styles.css" rel="stylesheet" type="text/css"/> <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/js/bootstrap.bundle.min.js"></script> <script src="https://cdnjs.cloudflare.com/ajax/libs/html2canvas/1.3.3/html2canvas.min.js"></script> <script src="/static/browse/0.3.4/js/addons.js"></script> <script src="/static/browse/0.3.4/js/feedbackOverlay.js"></script> <base href="/html/2312.09138v1/"/></head> <body> <nav class="ltx_page_navbar"> <nav class="ltx_TOC"> <ol class="ltx_toclist"> <li class="ltx_tocentry ltx_tocentry_section"><a class="ltx_ref" href="#S1" title="1 Introduction ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">1 </span>Introduction</span></a></li> <li class="ltx_tocentry ltx_tocentry_section"> <a class="ltx_ref" href="#S2" title="2 Related Work ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">2 </span>Related Work</span></a> <ol class="ltx_toclist ltx_toclist_section"> <li class="ltx_tocentry ltx_tocentry_paragraph"><a class="ltx_ref" href="#S2.SS0.SSS0.Px1" title="Dynamic point cloud understanding. ‣ 2 Related Work ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title">Dynamic point cloud understanding.</span></a></li> <li class="ltx_tocentry ltx_tocentry_paragraph"><a class="ltx_ref" href="#S2.SS0.SSS0.Px2" title="𝐒𝐎(𝟑)-equivariant networks. ‣ 2 Related Work ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><math alttext="\mathbf{SO(3)}" class="ltx_Math" display="inline"><semantics><mrow><mi>𝐒𝐎</mi><mo></mo><mrow><mo stretchy="false">(</mo><mn>𝟑</mn><mo stretchy="false">)</mo></mrow></mrow><annotation-xml encoding="MathML-Content"><apply><times></times><ci>𝐒𝐎</ci><cn type="integer">3</cn></apply></annotation-xml><annotation encoding="application/x-tex">\mathbf{SO(3)}</annotation><annotation encoding="application/x-llamapun">bold_SO ( bold_3 )</annotation></semantics></math>-equivariant networks.</span></a></li> <li class="ltx_tocentry ltx_tocentry_paragraph"><a class="ltx_ref" href="#S2.SS0.SSS0.Px3" title="Point cloud registration. ‣ 2 Related Work ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title">Point cloud registration.</span></a></li> <li class="ltx_tocentry ltx_tocentry_paragraph"><a class="ltx_ref" href="#S2.SS0.SSS0.Px4" title="Multi-object reconstruction. ‣ 2 Related Work ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title">Multi-object reconstruction.</span></a></li> </ol> </li> <li class="ltx_tocentry ltx_tocentry_section"> <a class="ltx_ref" href="#S3" title="3 Living Scenes ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">3 </span>Living Scenes</span></a> <ol class="ltx_toclist ltx_toclist_section"> <li class="ltx_tocentry ltx_tocentry_paragraph"><a class="ltx_ref" href="#S3.SS0.SSS0.Px1" title="Problem Setting. ‣ 3 Living Scenes ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title">Problem Setting.</span></a></li> <li class="ltx_tocentry ltx_tocentry_paragraph"><a class="ltx_ref" href="#S3.SS0.SSS0.Px2" title="Method Overview. ‣ 3 Living Scenes ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title">Method Overview.</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"> <a class="ltx_ref" href="#S3.SS1" title="3.1 Encoder-decoder Network ‣ 3 Living Scenes ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">3.1 </span>Encoder-decoder Network</span></a> <ol class="ltx_toclist ltx_toclist_subsection"> <li class="ltx_tocentry ltx_tocentry_paragraph"><a class="ltx_ref" href="#S3.SS1.SSS0.Px1" title="Vector Neuron Encoder (VN). ‣ 3.1 Encoder-decoder Network ‣ 3 Living Scenes ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title">Vector Neuron Encoder (VN).</span></a></li> <li class="ltx_tocentry ltx_tocentry_paragraph"><a class="ltx_ref" href="#S3.SS1.SSS0.Px2" title="Neural Implicit Decoder. ‣ 3.1 Encoder-decoder Network ‣ 3 Living Scenes ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title">Neural Implicit Decoder.</span></a></li> <li class="ltx_tocentry ltx_tocentry_paragraph"><a class="ltx_ref" href="#S3.SS1.SSS0.Px3" title="Training. ‣ 3.1 Encoder-decoder Network ‣ 3 Living Scenes ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title">Training.</span></a></li> </ol> </li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="#S3.SS2" title="3.2 Instance Matching ‣ 3 Living Scenes ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">3.2 </span>Instance Matching</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="#S3.SS3" title="3.3 Instance Registration ‣ 3 Living Scenes ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">3.3 </span>Instance Registration</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="#S3.SS4" title="3.4 Instance Reconstruction ‣ 3 Living Scenes ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">3.4 </span>Instance Reconstruction</span></a></li> <li class="ltx_tocentry ltx_tocentry_subsection"> <a class="ltx_ref" href="#S3.SS5" title="3.5 Joint Optimization for Accumulation ‣ 3 Living Scenes ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">3.5 </span>Joint Optimization for Accumulation</span></a> <ol class="ltx_toclist ltx_toclist_subsection"> <li class="ltx_tocentry ltx_tocentry_paragraph"><a class="ltx_ref" href="#S3.SS5.SSS0.Px1" title="Initialization. ‣ 3.5 Joint Optimization for Accumulation ‣ 3 Living Scenes ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title">Initialization.</span></a></li> <li class="ltx_tocentry ltx_tocentry_paragraph"><a class="ltx_ref" href="#S3.SS5.SSS0.Px2" title="Optimization Objectives. ‣ 3.5 Joint Optimization for Accumulation ‣ 3 Living Scenes ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title">Optimization Objectives.</span></a></li> </ol> </li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="#S3.SS6" title="3.6 Implementation Details ‣ 3 Living Scenes ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">3.6 </span>Implementation Details</span></a></li> </ol> </li> <li class="ltx_tocentry ltx_tocentry_section"> <a class="ltx_ref" href="#S4" title="4 Experiments ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">4 </span>Experiments</span></a> <ol class="ltx_toclist ltx_toclist_section"> <li class="ltx_tocentry ltx_tocentry_subsection"> <a class="ltx_ref" href="#S4.SS1" title="4.1 Datasets ‣ 4 Experiments ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">4.1 </span>Datasets</span></a> <ol class="ltx_toclist ltx_toclist_subsection"> <li class="ltx_tocentry ltx_tocentry_paragraph"><a class="ltx_ref" href="#S4.SS1.SSS0.Px1" title="FlyingShape. ‣ 4.1 Datasets ‣ 4 Experiments ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><em class="ltx_emph ltx_font_italic">FlyingShape</em>.</span></a></li> <li class="ltx_tocentry ltx_tocentry_paragraph"><a class="ltx_ref" href="#S4.SS1.SSS0.Px2" title="3RScan [63]. ‣ 4.1 Datasets ‣ 4 Experiments ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><em class="ltx_emph ltx_font_italic">3RScan</em> <cite class="ltx_cite ltx_citemacro_cite">[<span class="ltx_ref"><span class="ltx_text" style="font-size:90%;">63</span></span>]</cite>.</span></a></li> </ol> </li> <li class="ltx_tocentry ltx_tocentry_subsection"> <a class="ltx_ref" href="#S4.SS2" title="4.2 Evaluation Metrics ‣ 4 Experiments ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">4.2 </span>Evaluation Metrics</span></a> <ol class="ltx_toclist ltx_toclist_subsection"> <li class="ltx_tocentry ltx_tocentry_paragraph"><a class="ltx_ref" href="#S4.SS2.SSS0.Px1" title="End-to-end Metrics. ‣ 4.2 Evaluation Metrics ‣ 4 Experiments ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title">End-to-end Metrics.</span></a></li> </ol> </li> <li class="ltx_tocentry ltx_tocentry_subsection"><a class="ltx_ref" href="#S4.SS3" title="4.3 End-to-end Performance ‣ 4 Experiments ‣ Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments"><span class="ltx_text ltx_ref_title"><span class="ltx_tag ltx_tag_ref">4.3 </span>End-to-end
