Optimization and Machine Learning. Patrick Siarry

187

      191 188

      192  189

      193 190

      194  191

      195  192

      196  193

      197  194

      198  195

      199  196

      200  197

      201  198

      202  199

      203  201

      204 202

      205  203

      206  204

      207  205

      208  206

      209  207

      210  208

      211  209

      212  210

      213  211

      214  212

      215  213

      216  214

      217 215

      218  216

      219  217

      220  218

      221  219

      222  220

      223  221

      224  222

      225  223

      226  224

      227  225

      228  226

      229  227

      230  228

      231  229

      232  230

      233  231

      234  232

      235  233

      236  234

      237  235

      238 236

      239  237

      SCIENCES

      Computer Science,

      Field Directors – Valérie Berthé and Jean-Charles Pomerol

      Operational Research and Decision, Subject Head – Patrick Siarry

      Optimization and Machine Learning

       Optimization for Machine Learning and Machine Learning for Optimization

       Coordinated by

      Rachid Chelouah

      Patrick Siarry

      First published 2022 in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc.

      Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the under mentioned address:

      ISTE Ltd

      27-37 St George’s Road

      London SW19 4EU

      UK

       www.iste.co.uk

      John Wiley & Sons, Inc

      111 River Street

      Hoboken, NJ 07030

      USA

       www.wiley.com

      © ISTE Ltd 2022

      The rights of Rachid Chelouah and Patrick Siarry to be identified as the authors of this work have been asserted by them in accordance with the Copyright, Designs and Patents Act 1988.

      Library of Congress Control Number: 2021949293

      British Library Cataloguing-in-Publication Data

      A CIP record for this book is available from the British Library

      ISBN 978-1-78945-071-2

      ERC code:

      PE1 Mathematics

       PE1_19 Control theory and optimization

      PE6 Computer Science and Informatics

       PE6_11 Machine learning, statistical data processing and applications using signal processing (e.g. speech, image, video)

      Introduction

       Rachid CHELOUAH

       CY Cergy Paris University, France

      Machine learning is revolutionizing our world. It is difficult to conceive of any other information technology that has developed so rapidly in recent years, in terms of real impact.

      The fields of machine learning and optimization are highly interwoven. Optimization problems form the core of machine learning methods and modern optimization algorithms are using machine learning more and more to improve their efficiency.

      Machine learning has applications in all areas of science. There are many learning methods, each of which uses a different algorithmic structure to optimize predictions, based on the data received. Hence, the first objective of this book is to shed light on key principles and methods that are common within both fields.

      Machine learning and optimization share three components: representation, evaluation and iterative search. Yet while optimization solvers are generally designed to be fast and accurate on implicit models, machine learning methods need to be generic and trained offline on datasets. Machine learning problems present new challenges for optimization researchers, and machine learning practitioners seek simpler, generic optimization algorithms.

Quite recently, modern approaches to machine learning have also been applied to the design of optimization algorithms themselves, taking advantage of their ability to capture valuable information from complex structures in large spaces. Those capacities appear to be useful, especially for the representation and evaluation components. As large, complex structures are ubiquitous in optimization problems, and can be used as huge implicit datasets, the use of machine learning enabled the efficiency and genericity