3D Ptychographic Inverse Imaging with Generative Diffusion Models

Le résumé fourni par la source

Artificial intelligence and machine learning are poised to revolutionize electron microscopy, enabling advances in instrument automation, parameter optimization, image classification, and defect analysis [1-5]. These applications primarily leverage the predictive power of machine learning, focusing on tasks such as feature extraction and regression-based modeling . On the other end of the spectrum, generative diffusion models have unlocked new possibilities beyond conventional image processing, particularly in solving inverse problems such as image inpainting, denoising, and super-resolution [6-8]. Many microscopy techniques, such as tomography and ptychography, are inherently inverse problems, making them well-suited for generative approaches that integrate data-driven priors with physics-based models. In this work, we improve the depth resolution of multislice electron ptychography (MEP) by integrating a generative diffusion model into the iterative reconstruction process, which recovers 3D atomic structure using 4D scanning transmission electron microscopy (STEM) diffraction patterns [9]. Figure 1 illustrates the workflow of our physics-guided diffusion framework. Unlike conventional optimization-based approaches that rely solely on experimental data, our method integrates a pre-trained generative prior within the iterative reconstruction process, ensuring physically consistent and statistically plausible solutions. Following a Bayesian inversion framework, we leverage prior knowledge from a large crystal structure database to sample from the posterior distribution, incorporating both the likelihood (data fidelity) and the prior (structural constraints). We train our diffusion model on more than 200,000 crystal structures acquired from the ICSD database [10], with atomic potentials rendered as volumetric data using abTEM [11]. For evaluation, we generate independent validation and test sets by simulating 4D-STEM diffraction patterns for an additional 5,000 crystal structures at a dose of 106 e-/ Å2, which are not included in our training set. Notably, the diffusion model is trained exclusively on crystal volume data without incorporating any knowledge of experimental conditions or specific imaging modalities. As a result, our model serves as a purely data-driven prior for crystal structures, making it broadly applicable to other 3D inverse imaging techniques such as tilt-coupled MEP or joint ptycho-tomography [12-13]. Figure 2 presents a comparative analysis of MEP reconstructions using different methods, including PtychoShelves (LSQ-ML) [14], gradient descent (Adam), and our physics-guided diffusion. The standard approaches struggle with depth resolution, producing reconstructions where individual slices resemble depth-averaged projections. In contrast, our physics-guided diffusion framework successfully reconstructs distinct depth-dependent features, closely matching the ground truth atomic structure of this test data. However, it is important to note that prior-informed reconstructions are not deterministic and do not guarantee retrieval of the exact underlying structure. Instead, the generative prior helps constrain solutions to be more physically meaningful, effectively complementing conventional methods and enable a better measurement of reconstruction uncertainty. Our work demonstrates the power of generative priors for inverse problem solving in electron microscopy [15]. Schematic illustration of the workflows for (a) physics-guided diffusion, (b) conventional iterative ptychographic reconstruction, and (c) a standard generative diffusion model trained on a crystal structure database. Our physics-guided diffusion approach integrates a pre-trained generative prior into the conventional iterative algorithm for multislice ptychographic reconstruction. By leveraging a generative prior trained on over 200,000 crystal structures, this method produces high-quality reconstructions that remain consistent with the physical measurements. Comparison of multislice electron ptychographic reconstruction methods, including PtychoShelves (LSQ-ML), gradient descent (Adam), and Physics-Guided Diffusion, against the ground truth. Diffraction patterns are simulated from ground truth crystal structures and reconstructed using each method. While both PtychoShelves and gradient descent (Adam) suffer from limited depth resolution—producing slice reconstructions that resemble depth-averaged projections—Physics-Guided Diffusion successfully recovers most of the depth structure, achieving an excellent match with the ground truth atomic structure. Note that the test crystal is not included in the training set of our diffusion prior. The slice thickness is 1.6 Å.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.