Machine Learning for Tomographic Imaging. Professor Ge Wang

and prototyping of human intelligence so that we not only demystify the ultimate secret of life but also let machines work for us intelligently (figure 0.2).

      Figure 0.2. Curiosity and needs drive scientific pursuits through the industrial, information, and intelligent revolutions.

      Over only the past few years, AI/ML techniques have achieved impressive successes in computer vision, image analysis, speech recognition, language processing, and many other areas. A major feature behind these successes is that they use deep artificial neural networks trained with big data. An artificial neural network consists of many artificial neurons. Such neurons are basic data processing units performing a linear (weighted sum) operation followed by a simple nonlinear (thresholding) operation. This was inspired by how a biological neuron works. A biological neuron accumulates multiple stimuli, and when the overall stimulation is over a threshold, the neuron will become excited and respond by sending an electrical signal to other neurons or cells. There are a huge number of biological neurons in our brain, and it is the biological neural network that gives us intelligence, and the unmistaken example showing that intelligence is feasible. Similar to this biological/neurological system, an artificial neural network can behave, to a good degree, like a brain, if the number of artificial neurons is high enough, organized deeply (i.e. with many layers of artificial neurons), and trained well with big data. This resembles the learning process in our childhoods, where our neurological connections are formed adaptively, and we become increasingly intelligent.

      The importance and potential of AI/ML has now been well recognized. The AI Executive Order was issued by the White House in February 2019 (https://www.whitehouse.gov/articles/accelerating-americas-leadership-in-artificial-intelligence/), and the response from NIST is also inspiring to read (https://www.nist.gov/topics/artificial-intelligence). International competition is remarkable in advancing AI/ML theory and technologies (figure 0.3).

      Figure 0.3. Web of Knowledge results with ‘deep learning’ as the topic term (data collected on 11 July 2019).

      0.2 Image analysis versus image reconstruction

      As the most famous examples of AI/ML applications, computer vision and image analysis deal with existing images and produce features of these images thanks to the great efforts of many talented researchers. We are researchers in the field of tomographic imaging, and our products are tomographic images reconstructed from externally measured, complicated data that look totally different from the underlying images, and are actually various features (attenuated/non-attenuated line integrals, Fourier/harmonic components, and so on) of the underlying images. Currently, machine learning (especially deep learning) techniques are being actively developed worldwide for tomographic image reconstruction, which is a new area of research, with low hanging fruit in terms of data-driven post-processing and high hanging fruit in terms of end-to-end mapping via transfer, adversarial, ensemble, and other forms of machine learning.

      This first-of-its-kind book is dedicated to machine learning for tomographic image reconstruction, or tomographic imaging, primarily targeting image reconstruction (from data to images) with some mentions of relevant image analysis (from images to images/features) and end-to-end mapping (from data to features). Tomography is a Greek word, meaning reconstruction of cross-sectional images. It is the emphasis on tomography that sets our book apart from other AI/ML or deep learning books (figure 0.4).

      Figure 0.4. Uniqueness of this book, dedicated to tomographic image reconstruction in the AI/ML framework, in contrast to deep learning for image analysis or computer vision that takes images as the input.

      0.3 Analytic/iterative/deep learning algorithms for tomographic reconstruction

      Traditionally, there are two kinds of algorithms for tomographic reconstruction—analytic and iterative. When tomographic data are of high quality and sufficiently collected, the relationship from an underlying image to the tomographic measures can be expressed as a forward model, which can be mathematically and computationally inverted. The inverse transform will transform the data back to the underlying image, which is what image reconstruction means. When such an inverse transform is in a closed form (for example, an inverse Fourier transform), a reconstruction algorithm can be directly obtained, and implemented on a computer.

      When tomographic data are compromised, incomplete, or the forward model is too complicated to be analytically inverted, an iterative algorithm can be used for image reconstruction. An iterative algorithm does not solve the problem in one shot. From an initial estimate of the underlying image, which can be as simple as an all zero or all one image or another form when specific prior knowledge is available, the algorithm refines an intermediate solution iteratively (the first one is the initially guessed image). The refinement is guided by two preferences. First, the discrepancy should be small between the measured data and the data computed according to the forward model based on an intermediate image. Second, the characteristics of a reconstructed image should look reasonable or consistent with our prior knowledge, such as non-negative pixel values and no severely oscillating features. These requirements are summarized into an overall objective function, and the reconstruction problem becomes an optimization task. In other words, the iterative algorithm is just for this optimization. In most cases of tomographic imaging, measured data are not enough to determine the underlying image uniquely, and prior knowledge is instrumental for satisfactory image reconstruction. The stronger the prior knowledge is, the better the reconstructed image quality will be. Various kinds of prior knowledge are in use for iterative reconstruction, including non-negativity, maximum entropy, roughness penalty, total variation minimization, low rank, dictionary, and low-dimensional manifold learning. Normally, an iterative algorithm is very time-consuming.

      It is clear now that deep learning networks form the third category of image reconstruction algorithms. In contrast to the aforementioned kinds of prior knowledge, each of which can be formulated as one mathematical term in one or two lines, an unprecedented source of prior knowledge is big data. For example, millions of CT scans contain overwhelming information on underlying anatomical and pathological information, and a new scan should be very much correlated to or consistent with the existing scans. If these data can be utilized for image reconstruction, superior image quality is expected. Fortunately, deep neural networks can be trained with big data on a high-performance computing platform so that prior knowledge can be represented by the trained neural network that serves as a mapping from data to images. Because the prior knowledge used by the neural network is task-specific and yet comprehensive, in principle the network may produce better image quality than an iterative algorithm when it falls short of clinical satisfaction. Although training the network is still time-consuming, the trained network only involves forward operations and is computationally efficient.

      The analytic, iterative, and deep learning algorithms for tomographic image reconstruction can be compared and contrasted (table 0.1). Briefly speaking, an analytic algorithm can be formulated in the following form, f(x,y)=Op(θ,t), where f represents an image in a 2D case without loss of generality, p denotes data as a function of projection viewing angle and detector position, and O is an analytic operation in the closed form, such as an inverse Fourier transform. An iterative algorithm, on the other hand, is expressed as f(k)(x,y)=Op(θ,t),f(k−1)(x,y), where the index k goes from 0, 1, 2, to a sufficiently large number K for the iterative process to converge. The image for k = 0 is the initial guess as the starting point of the iterative process. Different from either analytic or iterative algorithms, a deep learning based tomographic reconstruction algorithm is written as f(x,y)=OθN…Oθ1p(θ,t),