An artificial intelligence’s interpretation of complex high-resolution in situ transmission electron microscopy data

<title>Abstract</title> <italic>In situ</italic> transmission electron microscopy (TEM) has enabled researchers to visualize complicated nano- and atomic-scale processes with sub-Angstrom spatial resolution and millisecond time resolution. These processes are often highly dynamical and can be time-consuming to analyze and interpret. Here, we report how variational autoencoders (VAEs), a deep learning algorithm, can provide an artificial intelligence’s interpretation of high-resolution <italic>in situ</italic> TEM data by condensing and deconvoluting complicated atomic-scale dynamics into a latent space with reduced dimensionality. In this work, we designed a VAEs model with high latent dimensions capable of deconvoluting information from complex high-resolution TEM data. We demonstrate how VAEs with high latent dimensions trained on atomically resolved TEM images of lead sulfide (PbS) nanocrystals is able to capture movements and perturbations of periodic lattices in both simulated and real <italic>in situ</italic> TEM data. The VAEs model shows capability of detecting and deconvoluting dynamical nanoscale physical processes, such as the rotation of crystal lattices and intraparticle ripening during the annealing of semiconductor nanocrystals. With the help of the VAEs model, we can identify an <italic>in situ</italic> observation that can serve as a direct experimental evidence of the existence of intraparticle ripening, a process so far has not been recorded under atomically-resolved <italic>in situ</italic> TEM for colloidal semiconductor nanocrystals. The VAEs model provides a potent tool for facilitating the analysis and interpretation of complex <italic>in situ</italic> TEM data as a part of an autonomous experimental workflow. It also shows potential to identify unexpected phenomena during the data analysis.