INR-Based STEM Tomographic Reconstruction Overview
Figure 1: Overview of the INR-based tomographic reconstruction algorithm. (a) Schematic of a tomographic experiment using STEM. An image is recorded at each tilt step as the sample is rotated around the tilt axis. (b) A neural network receives a 3-D position vector, r → = { x , y , z } \vec{r}=\{x,y,z\} as input, and produces an intensity value, ϕ \phi , at that position. This way of representing the volume is called an implicit neural representation (INR). (c) The reconstruction volume is sampled by rays that terminate at individual pixels of the experimental data. Each ray also incorporates and optimizes the pose for each tilt image in the dataset. (d) Optimization loop schematic. Coordinates sampled along each ray are fed to the INR, whose outputs are integrated to predict the pixel intensity. This prediction is compared to the experimental measurement with a pixel-wise loss, which is back-propagated to update both the INR weights and the pose. (e) Schematic showing how the experimental tilt images are reshaped into a 1D array of pixels that are compared to individual pixel predictions made using the INR.
Paper context
Paper title: Missing Wedge Inpainting and Joint Alignment in Electron Tomography through Implicit Neural Representations Abstract: Electron tomography is a powerful tool for understanding the morphology of materials in three dimensions, but conventional reconstruction algorithms typically suffer from missing-wedge artifacts and data misalignment imposed by experimental constraints. Recently proposed supervised machine-learning-enabled reconstruction methods to address these challenges rely on training data and are therefore difficult to generalize across materials systems. We propose a fully self-supervised implicit neural representation (INR) approach using a neural network as a regularizer. Our approach enables fast inline alignment through pose optimization, missing wedge inpainting, and denoising of low dose datasets via model regularization using only a single dataset. We apply our method to simulated and experimental data and show that it produces high-quality tomograms from diverse and information limited datasets. Our results show that INR-based self-supervised reconstructions offer high fidelity reconstructions with minimal user input and preprocessing, and can be readily applied to a wide variety of materials samples and experimental parameters. Passages referencing this figure: 2 Results Figure 1: Overview of the INR-based tomographic reconstruction algorithm. 1 INR Tomography Workflow A schematic of our reconstruction workflow is shown in Fig. 1 . (Fig. 1 a). Specifically, we use a fully-connected neural network with periodic activation functions [ gao_h-siren_2024 ] , which takes as input an arbitrary coordinate vector and returns an intensity value (Fig. 1 b). We first compute rays along the beam direction with discrete sampling points (Fig. 1 c) which correspond to each pose.
