Deep Learning for Physical Scientists. Edward O. Pyzer-Knapp

the organization, website, or product may provide or recommendations it may make. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

      For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at [phone], outside the United States at [phone] or fax [phone].

      Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at ww.wiley.com.

       Library of Congress Cataloging‐in‐Publication Data

      Names: Pyzer-Knapp, Edward O., author. | Benatan, Matthew, author.

      Title: Deep learning for physical scientists : accelerating research with machine learning / Edward O. Pyzer-Knapp, IBM Research UK, Data Centric Cognitive Systems, Daresbury Laboratory, Warrington UK, Matthew Benatan, IBM Research UK, Data Centric Cognitive Systems, Daresbury Laboratory, Warrington UK.

      Description: Hoboken, NJ : Wiley, 2022. | Includes index.

      Identifiers: LCCN 2021036996 (print) | LCCN 2021036997 (ebook) | ISBN 9781119408338 (hardback) | ISBN 9781119408321 (adobe pdf) | ISBN 9781119408352 (epub)

      Subjects: LCSH: Physical sciences–Data processing. | Machine learning.

      Classification: LCC Q183.9 .P99 2022 (print) | LCC Q183.9 (ebook) | DDC 500.20285/631–dc23

      LC record available at https://lccn.loc.gov/2021036996 LC ebook record available at https://lccn.loc.gov/2021036997

      Cover Design: Wiley

      Cover Image: © Anatolyi Deryenko/Alamy Stock Photo

About the Authors

      Dr Edward O. Pyzer‐Knapp is the worldwide lead for AI Enriched Modelling and Simulation at IBM Research. Previously, he obtained his PhD from the University of Cambridge using state of the art computational techniques to accelerate materials design then moving to Harvard where he was in charge of the day‐to‐day running of the Harvard Clean Energy Project ‐ a collaboration with IBM which combined massive distributed computing, quantum‐mechanical simulations, and machine‐learning to accelerate discovery of the next generation of organic photovoltaic materials. He is also the Visiting Professor of Industrially Applied AI at the University of Liverpool, and the Editor in Chief for Applied AI Letters, a journal with a focus on real‐world application and validation of AI.

      Dr Matt Benatan received his PhD in Audio‐Visual Speech Processing from the University of Leeds, after which he went on to pursue a career in AI research within industry. His work to date has involved the research and development of AI techniques for a broad variety of domains, from applications in audio processing through to materials discovery. His research interests include Computer Vision, Signal Processing, Bayesian Optimization, and Scalable Bayesian Inference.

      Acknowledgements

      EPK: This book would not have been possible without the support of my wonderful wife, Imogen.

      MB: Thanks to my wife Rebecca and parents Dan & Debby for their continuing support.

1 Prefix – Learning to “Think Deep”

      Paradigm shifts in the way we do science occur when the stars align. For this to occur we must have three key ingredients:

      1 A fundamental problem, which is impeding progress;

      2 A solution to that problem (often theoretical); and crucially

      3 The capability to fully realise that solution.

      Whilst this may seem obvious, the lack of (3) can have dramatic consequences. Imperfectly realised solutions, especially if coupled with overzealous marketing (or hype) can set back a research field significantly – sometimes resulting in decades in the wilderness.

      Machine learning has suffered this fate not once, but twice – entering the so‐called AI‐winters where only a few brave souls continued to work. The struggle was not in vain, however, and breakthroughs in the theory – especially the rapid improvement and scaling of deep learning – coupled with strides in computational capability have meant that the machine learning age seems to be upon us with a vengeance.

      This “era” of AI feels more permanent, as we are finally seeing successful applications of AI in areas where it had previously struggled. Part of this is due to the wide range of tools which are at the fingertips of the everyday researcher, and part of it is the willingness of people to change the way they think about problems so as to use these new capabilities to their full potential.

      The aims of this book are twofold:

      1 Introduce you to the prevalent techniques in deep learning, so that you can make educated decisions about what the best tools are for the job.

      2 Teach you to think in such a way that you can come up with creative solutions to problems using the tools that you learn throughout this book.

1.1 So What Do I Mean by Changing the Way You Think?

      Typically in the sciences, particularly the physical sciences, we focus on the model rather than on the task that it is providing. This can lead to siloing of techniques in which (relatively) minor differences become seemingly major differentiating factors.

      As a data‐driven researcher it becomes much more important to understand broad categorisations and find the right tools for the job, to be skilled at creating analogies between tasks, and to be OK with losing some explainability of the model (this is particularly true with deep learning) at the gain of enabling new capabilities. More than any other area I have been involved with, deep learning is more a philosophy than any single technique; you must learn to “think deep.” Will deep learning always be the right approach? Of course not, but with growing experience you will be able to intuit when to use these tools, and also be able to conceive of new ways in which these tools can be applied. As we progress through this journey I will show you particularly innovative applications of deep learning, such as the neural fingerprint, and task you to stretch your imagination through the use of case studies. Who knows, you might even improve upon the state of the art!

      Key Features of Thinking Deep

      1 Never get tied into domain specific language, instead use broad terms which describe the process you are trying to achieve.

      2 Look for analogies between tasks. By forcing yourself to “translate” what is going on in a particular process you often come to understand more deeply what you are trying to achieve.

      3 Understand the difference between data and information. A few well chosen pieces of data can be more useful than a ton of similar data points.

      4 Finally, never lose sight of your true goal – at the end of the day this is more likely to be “provide insight into process X” than “develop a super fancy method for predicting X.” The fancy method may be necessary, but it is not the goal.

Thinking back to the three ingredients of a paradigm shift, we remember that one of the major blockers to achieving this is the lack of a capability to