In the past decade, there has been a significant progress in machine learning technologies, and there are already many tools available for a wide variety of image analysis fields. One interesting question is, how do they compete against the conventional approaches? For example, anatomical segmentation techniques are maturing, especially with the multi-atlas approaches. The recent conventional approaches can also be considered as a type of machine learning based on multiple atlas libraries (teaching sets), but they are substantially different from techniques such as DLDeep Learning. We are developing and testing hybrid approaches, which could give us the best balance of efficiency and accuracy. The team is uniquely positioned to work on these questions because of our accumulated knowledge, data, and tools. As we already have gold standard material (atlases that define structures or lesions) and existing tools, we can readily measure the efficiency (required time) and accuracy and compare with conventional approaches. We also have a large amount of data to test the tools from variable sources.
We are also developing a framework for diffeomorphic image registration that can incorporate extracted features (i.e. labels and lesions) to perform brain mapping of cases with severe pathological conditions. Figure 1 below shows an example of the use of deep learning in multi-scale image analysis (1,2).
(1) Tward, D. and M. Miller, EM-LDDMM for 3D to 2D registration. bioRxiv, 2019: p. 604405
(2) Tward, D., T. Brown, Y. Kageyama, J. Patel, Z. Hou, S. Mori, M. Albert, J. Troncoso, and M. Miller, Diffeomorphic Registration With Intensity Transformation and Missing Data: Application to 3D Digital Pathology of Alzheimer's Disease. Front Neurosci, 2020. 14: p. 52.
Fig. 1: Starting from tau tangles segmented from histology images (a) with a deep convolutional network (2) we formed local statistical ensembles to visualize (b) layer specific distributions in tau tangle size (1). In parallel work, we are developing scattering transform-based methods to enhance Nissl with texture information for multi-modality registration and tissue classification. At whole brain scale, structures with the same average gray level in (c, image from Allen Brain Institute) have very different textures (d).