The Pocket Guide to Critical Appraisal. Iain K. Crombie

information into a mental framework. This simplifies the task of understanding the paper and helps you organise the information it contains. The process uses four questions.

      What does the title reveal about the study?

      The interrogation begins with the title. Good titles should indicate the patient group being investigated, the research design used, and why the study is being done. Sometimes titles are written to be catchy rather than informative, but even then, they should provide one or two facts about the study. The title provides the first indication of what the study is about.

      Does the abstract help in constructing the mental map?

      The Abstract usually gives key information about the four main sections of the paper: the Introduction, Methods, Results, and Discussion. Ideally it will clarify the study aim, the research design that was used, the main findings and their implications. Together with the title, the Abstract should enable you to construct a provisional map of the paper. However, this is not guaranteed. Information in the Abstracts may be presented to impress rather than to be informative. The provisional map should be tested by comparing what it says with the information presented in the Introduction and the Methods sections.

      Does the introduction confirm the aims?

The aims of the study are usually presented towards the end of the Introduction. They may be phrased as hypotheses to be tested or as questions to be answered. The absence of a clear statement of aims could mean that the authors had no real idea of what they were trying to find out. Research studies with unclearly stated aims are often of poor quality.

      Does the methods section explain how the study aims will be achieved?

      The Methods section should explain how all aspects of the study were carried out. The descriptions in this section are often brief and difficult to follow. This section should clarify how participants were recruited, the data that were collected, and the statistical techniques that were used. Often, the research design, which should explain how the study aims will be achieved, is not clearly stated. To assist with this limitation, Chapter 3 provides guidance on identifying the research design. It outlines the main features and the key terms indicative of the six most common research designs.

The in‐depth interrogation

      The initial interrogation should provide a provisional map of the paper into which information about the sources of bias and the value of the study findings can be fitted. This process requires an in‐depth interrogation using detailed questions which probe the quality of the study. These questions are described in Chapters 5–11. Before reading these chapters, it may be helpful to review Chapter 3 ‘Identifying the Research Design’ and Chapter 4 ‘Interpreting the Results’.

CHAPTER 3 Identifying the Research Design

      The first step in critical appraisal is to identify the research design used in a study so the appropriate appraisal checklist can be selected. This chapter provides an outline of the common research designs to aid their identification. It does not provide a full description of each method, but instead gives sufficient detail to confirm which design was used.

Surveys

Surveys are often used to estimate how common something is: how many people have high blood pressure, or how many suffer from chronic pain. They can also investigate associations between factors: is it more common in men or women; does the frequency of high blood pressure vary with age? Surveys can study whole populations; for example, to establish the proportion of people who currently smoke cigarettes. Or they can investigate specific groups; for example, they could explore the health beliefs of pregnant women, or the frequency of loneliness in persons aged between 65 and 90 years.

      Essential features

      Surveys take samples from a target group or population. The idea behind this is that a well‐taken sample contains almost as much information as would come from studying the whole population. In principle, surveys obtain a complete list of the group or population of interest from which a sample of individuals is selected for further study. The selection is carried out randomly (not haphazardly), such that each individual has an equal chance of being chosen. In practice, a complete list may not be available and alternative approaches such as cluster sampling can be used (these more complicated designs are not described here). The important points are that selection of individuals uses random sampling and that the sample obtained is representative of a target population.

      Complications

      Surveys usually do not have a separate control or comparison group, so studies that have them are not surveys. However, in the analysis of surveys one subgroup in the sample may be compared against another (e.g. men versus women, or old versus young). Comparisons are being made, but there is no sense in which one group is acting as a control to another group. All the individuals have been selected at the same time and an internal comparison is made.

      Terms of identification

Use of the term survey in a paper should identify the method, but sometimes the term is used for what is really a cohort study. Cross‐sectional is a helpful term because it is seldom used with any other research method. Prevalence, the frequency of something in a sample, also suggests that a study is a survey. The terms sample and random sample are unhelpful because they often appear in the description of the other research designs. The terms simple, cluster, or systematic can be used with the word sampling to describe different ways of drawing a sample, e.g. cluster sampling. These terms are seldom used with the other research designs. The phrase stratified sampling is also used in surveys, but the word stratified can also be used in randomised controlled trials (RCTs).

Cohort studies

      Cohort studies are used to find out what happens to study participants over time. It is the method of choice for studying disease prognosis, or for investigating the consequences of exposure to potentially harmful agents. For example, studies could investigate how long patients with acute low back pain take to recover, or how many people who smoke subsequently develop lung cancer. Whatever the topic, a group of individuals is identified and followed up to see what events befall them. Most commonly the aim is to determine whether exposure to a potentially noxious substance leads to an increased risk of a disease. Sometimes the interest is in whether potentially beneficial substances, such as dietary vitamins or fish oils, reduce the risk of disease.

      Essential features

      The defining characteristic of cohort studies is the element of time: in cohort studies time flows forwards. A set of individuals is identified at one point in time and followed up to a later time to ascertain what has happened. These studies are called prospective cohort studies. When studying the impact of some exposure, cohort studies often have a comparison or control group. The controls are usually identified at around the same time and followed for the same length of time. This type of cohort study is called a concurrent cohort study.

Many cohort studies do not have a control group; instead they can make internal comparisons. For example, a cohort could recruit a large sample from the general population and identify who smoked and who did not. The whole sample would