The Pocket Guide to Critical Appraisal. Iain K. Crombie

      Iain K. Crombie

      University of Dundee Dundee, UK

      This second edition first published 2022

      © 2022 John Wiley & Sons Ltd

      All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.

      The right of Iain K. Crombie to be identified as the author of this work has been asserted in accordance with law.

       Registered Offices

      John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA

      John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK

       Editorial Office

      9600 Garsington Road, Oxford, OX4 2DQ, UK

      For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.

      Wiley also publishes its books in a variety of electronic formats and by print‐on‐demand. Some content that appears in standard print versions of this book may not be available in other formats.

      Limit of Liability/Disclaimer of Warranty: The contents of this work are intended to further general scientific research, understanding, and discussion only and are not intended and should not be relied upon as recommending or promoting scientific method, diagnosis, or treatment by physicians for any particular patient. In view of ongoing research, equipment modifications, changes in governmental regulations, and the constant flow of information relating to the use of medicines, equipment, and devices, the reader is urged to review and evaluate the information provided in the package insert or instructions for each medicine, equipment, or device for, among other things, any changes in the instructions or indication of usage and for added warnings and precautions. While the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

       Library of Congress Cataloging‐in‐Publication Data Applied for

      Paperback ISBN: 9781119835240

      Cover Design: Wiley

      Cover Image: © Govindanmarudhai/Getty Images

Acknowledgements

      I am grateful to the University of Dundee for providing the facilities to research and write the second edition of this book. The success of the first edition was due in no small part to my colleagues for their support and encouragement. As the style and much of the content of this new edition follows the original, I am pleased to be able to thank them again for their help. Special thanks are due to Geraldine Fardon whose constructive comments highlighted imprecise, vague, and confusing text. I take full responsibility for any remaining flaws.

Preface to the First Edition

      This book was written to meet the needs of health professionals as medicine moves to be evidence‐based. The initial idea arose during discussion with younger colleagues and students on difficulties of interpreting the medical literature. It quickly became apparent that their needs would be best met by a short book detailing criteria for critical appraisal.

      The book is organised in two parts. The first five chapters provide an introduction to critical appraisal of quantitative research, indicating how papers can be read and how the results can be interpreted. Experienced researchers could easily omit these chapters. The final six chapters provide annotated checklists for critical appraisal. The first of these contains the general questions which can be asked of any study using a quantitative methodology. The succeeding five chapters review in turn the questions which are specific for each quantitative method. For convenience each of these five chapters concludes with a combined list of general and specific questions.

      The book has been written to be simple and quick to use. Technical terms are avoided where possible, and the assessment criteria are explained but not justified. A larger and less accessible text would have been needed to give a proper rationale for each the checklists. To keep this a pocket guide, it was also decided to omit evaluations of other topics such as qualitative methods, health economics, clinical audit, decision analysis, and screening tests. An argument could be made for the inclusion of each, but to include them all would nearly double the size of the book. I hope the checklists prove useful.

Preface to the Second Edition

      Several developments have highlighted the need for a second edition of this book. Recent work, particularly from the new discipline of meta‐research, has provided important evidence for critical appraisal. By examining and comparing large numbers of studies, this research has clarified the nature of and impact of many sources of bias. A separate development is the recognition that critical appraisal should distinguish between the risk of bias and the value of the findings to individuals, health care systems, and the wider society. Critical appraisal checklists should provide separate sets of questions for the two issues. Finally, many years of teaching critical appraisal skills have provided the author with an understanding of the difficulties that students and health professionals encounter when evaluating papers. These developments have led to improvements in the guidance given in this book.

      This new edition follows the format and the spirit of the first: it is written to be easy to use and, where possible, technical terms are avoided. However, substantial modifications have been made. A new Introduction clarifies the aims of critical appraisal. The second chapter, entitled ‘Do Not Read the Paper’, is also new; it provides a simple method for eliciting the information needed for critical appraisal from published studies. This is supported by Chapter 5, ‘The In‐Depth Interrogation’, which is a much‐amended version of the original ‘Standard Appraisal Questions’. In addition, Chapter 4 on ‘Interpreting the Results’ has been updated in line with current views on the nature of statistical significance and the interpretation of confidence intervals.

All of the chapters that provide checklists for critical appraisal of specific research designs have been revised and updated, particularly Chapter