J.P. Horwath, D.J. Groom, P.J. Ferreira, E.A. Stach
University of Pennsylvania,
United States
Keywords: imaging, machine learning, Electron Microscopy
Summary:
Advances in both methods and instrumentation for in situ Transmission Electron Microscopy (TEM) enable the observation of nanoscale materials with unprecedented spatial and temporal resolution. However, the accuracy and speed with which immense volumes of in situ TEM data can be analyzed limits the amount of information that can be obtained from them. Though simple observations of dynamic phenomena improve our qualitative knowledge of how materials interact with their environment, quantitative information must be extracted from images to develop a physical understanding of how processes occur. Over the past decade the computer science and computer vision communities have made large strides in rapid image segmentation and analysis using machine learning, particularly using convolutional neural networks (CNN), yet translation of these cutting-edge techniques to applications in physical sciences can be difficult. This talk will examine the use of conventional computer vision and machine learning techniques for the automated segmentation of TEM images. As a first step, the performance of deep learning image segmentation will be compared to that of conventional and hybrid approaches with regards to processing speed, pixelwise segmentation accuracy, and generalization across datasets. Recent research on the interpretability of machine learning models for image segmentation will be discussed, along with the limitations of both supervised and unsupervised machine learning models for practical application to segmenting images of nanoscale materials undergoing dynamic transitions. Finally, we will consider future directions for automated analysis of scientific images and how this can enable autonomous experiments.