Rank-Based Methods for Shrinkage and Selection. A. K. Md. Ehsanes Saleh

estimates and selects variables, it does not possess “oracle properties” in general. To overcome this problem Fan and Li (2001) proposed the “smoothly clipped absolute deviation” (SCAD) penalty function. Following Fan and Li (2001), Zou (2006) modified LASSO using a weighted L₁-penalty function. Zou (2006) called this estimator an adaptive LASSO (aLASSO). Later, Zhang (2010) suggested a minimax concave penalty (MCP) estimator. All results found in the above literature are based on penalized least squares criterion.

      This book contains a thorough study of rank-based estimation with three basic penalty estimators, namely, ridge regression, LASSO and “elastic net”. It also includes preliminary test and Stein-type R-estimators for completeness. Efforts are made to present a clear and balanced introduction of rank-based estimators with mathematical comparisons of the properties of various estimators considered. The book is directed towards graduate students, researchers of statistics, economics, bio-statistical biologists and for all applied statisticians, economists and computer scientists and data scientists, among others. The literature is very limited in the area of robust penalty and other shrinkage estimators in the context of rank-based estimation. Here, we provide both theoretical and practical aspects of the subject matter.

The book is spread over twelve chapters. Chapter 1 begins with an introductory examination of the median, outliers and robust rank-based methods, along with a brief look at penalty estimators. Chapter 2 continues with the characteristics of rank-based penalty estimators and demonstrates their enormous value in machine learning. Chapter 3 provides the preliminaries of rank-based theory and various aspects of it, along with a description of penalty estimators, which are then applied to location and simple linear models. Chapters 4 deals with ANOVA and Chapter 5 with seemingly unrelated simple linear models. Chapter 6 considers the multiple linear model and Chapter 7 expands on the “partially linear regression model” (PLM). The Liu regression estimator is discussed in Chapter 8. Chapter 9 introduces the AR(p) model. Chapter 10 covers selection and shrinkage of variables in high-dimensional data analysis. Chapter 11 deals with multivariate rank-based logistic regression models. Finally, Chapter 12 concludes with applications of rank-based neural networks.

      To our knowledge, this is one of the first books to combine advanced statistical analysis with advanced machine learning. Each chapter is self-contained but those interested in machine learning may consider Chapters 1 and 2, and 11 and 12, while those interested in statistics may consider Chapters 3–10. A good mix of the two would be derived from Chapters 1–4 and 11 and 12. It is our hope that readers in both fields will find something of value, and that it will lead to many areas of future research.

      The authors wish to thank the developers of Rfit (Kloke and McKean, 2012) and glmnet (Stanford University) which are extremely useful packages for R-estimation and penalized maximum likelihood estimation, respectively. We also thank Professor Brent Johnson (University of Rochester) for the rank-based LASSO and aLASSO code (Johnson and Peng, 2008) provided on his website.

      Professor A.K. Md. E. Saleh is grateful to NSERC for supporting his research for more than four decades and is appreciative of Professors P.K. Sen (U. of North Carolina), J. Jurečková (Charles U., Prague), M. Ghosh (Gainsville, Florida), H. L. Koul (Michigan State U.), T. Kubokawa (Tokyo, Japan), T. Shiraishi (Tsukuba, Japan) and C. Robert (Université Paris-Dauphine) for their active participation during the time of these grants. He is grateful to his loving wife, Shahidara Saleh, for her support over 70 years of marriage. He also appreciates his grandchildren Jasmine Alam, Sarah Alam, Migel Saleh and Malique Saleh for their loving care.

      Professor M. Arashi wishes to thank his family in Iran, specifically his wife, Reihaneh Arashi (maiden name: Soleimani) for her everlasting love and support and his daughter, Elena Arashi. This research is supported by Ferdowsi University of Mashhad, Iran and in part by the National Research Foundation (NRF) SARChI Research Chair UID: 71199, and DST-NRF Centre of Excellence in Mathematical and Statistical Sciences (CoE-MaSS) of South Africa.

      Professor R. Saleh wishes to thank his parents, Dr. Ehsanes Saleh and Mrs. Shadidara Saleh, for their love and support. He also wishes to acknowledge valuable discussions with Victor Aken’Ova and Sohaib Majzoub regarding logistic regression and neural networks, and the encouragement and support from Lynn Saleh, Isme Alam, Raihan Saleh and Jody Fast during the writing of this book.

Dr. M. Norouzirad deeply thanks her parents, Abbas Ali Norouzirad and Fereshteh Arefian for their unconditional trust, timely encouragement, and endless patience. She also thanks her sister, Mehrnoosh Norouzirad, for heart-warming kindness. She also wishes to acknowledge funding provided by National Funds through the FCT - FundaÇão para a Ciência e a Tecnologia, I.P., under the scope of the project UIDB/00297/2020 (Center for Mathematics and Applications).

       A.K. Md. Ehsanes Saleh

       Mohammad Arashi

       Resve A. Saleh

       Mina Norouzirad