The Handy Psychology Answer Book. Lisa J. Cohen

of central tendency, it is also important to know how much the individual members of the sample vary from each other and from the group mean. The variance of a sample is a measure of the average distance of any sample member from the group mean. The standard deviation is the square root of the variance. The standard deviation is an important component of many statistical tests.

      What does it mean to say that a finding is statistically significant?

      When a finding is described as statistically significant, it means that there is a very low probability that the finding is due to chance. If the same analysis were performed at another time or with another sample, we can be very sure (although not absolutely sure) that we will get the same result. Most research sets the criterion for statistical significance, known as the alpha coefficient, at 5 percent. The results are therefore statistically significant if the p value is less than .05. The p value refers to the probability that the test results are due to chance.

      Importantly, statistical significance does not determine the magnitude of a finding, only the reliability of a finding. In large samples, a group difference may be very small, but still highly significant; the finding may be reliable even if it is essentially trivial.

      How do we compare mean values across different groups?

      Many studies compare one or more variables across different groups. For example, does Drug A reduce depression more than Drug B? Do children from bilingual homes learn languages more quickly than children from monolingual (single language) homes? Is there more crime during the week of the full moon than during the week before? Several statistical tests help determine whether the means of two or more groups differ from each other and whether the difference found is statistically significant. Such tests include the Student’s t-test, the analysis of variance (ANOVA), and several variations on the ANOVA test (e.g., analysis of repeated measures (ANOVAR) and multivariate analysis of variance (MANOVA)). All of these tests compare the mean score between two or more groups, taking into account the variation within the groups (i.e., the standard deviation). The standard deviation is important because the greater the variation within the two groups, the more likely that the difference between the two groups is due to chance and would not be repeated if the same test were performed on a different sample.

      What does correlation mean?

      Correlation is another common means of evaluating the relationship between two variables. If one variable increases at the same time another one increases, the two variables are positively correlated. For example, gregariousness and number of friends are well correlated. The more gregarious a person is the more friends he or she is likely to have. Less gregarious people are likely to have fewer friends. If one variable increases while another decreases, the two variables are negatively correlated. Age and impulsivity are negatively correlated. The older someone gets, the less likely he or she is to engage in impulsive behavior. Likewise, younger people are more likely to engage in impulsive behavior. If there is no relationship between variables, they have no correlation. Month of birth and mathematical skills likely have no relationship. We do not anticipate that the month of birth would have any impact on a person’s mathematical ability.

      What does a study confound mean?

      A confound is something that biases the results of a study. It is a third, extraneous variable that accounts for the relationship between the two variables of interest. For example, much of the early literature on intelligence tests found that Americans of northern European descent had greater intelligence than immigrants from southern or eastern Europe. These results were confounded by language fluency as the immigrants were not fluent in English. We cannot conclude that the difference in test scores across ethnic groups is due to intelligence if it is confounded by language ability. There are statistical techniques to control for confounds, but they are not appropriate in all cases and it is always better, if possible, to avoid confounds in the first place.

      Why do we covary for certain variables?

      Although it is better to avoid potentially confounding variables, it is not always possible to do so. In these cases, it is possible to remove the effect of the confounding variable statistically. The relationship between variable A and variable B will be assessed, covarying for the effect of variable C. Let’s say we want to study the impact of child abuse on personality functioning in adulthood. We could correlate a measure of child abuse with a measure of adult personality functioning. However, some of the subjects had alcoholic parents. In order to make sure that we’re studying the impact of child abuse and not parental alcoholism, we would correlate childhood abuse and adult personality, covarying for the effect of parental alcoholism.

      What are the advantages of multivariate analysis?

      Psychology is the study of human behavior and as we all know, human behavior can be very complicated. Therefore it is often desirable to study many variables at once and to examine how the different variables relate to each other. Analyses that look at many variables at once are called multivariate analyses. Imagine you want to determine whether gym class improves academic performance in grade school children. You test a large sample of middle school children but you also want to account for many other factors that might impact a child’s academic performance, such as parental education, parental marital status, diet, sibling order, family income, teacher, school, child age, etc. A multivariate analysis would allow you to test the effect of each of these variables, while covarying for the effect of all other variables at the same time.

      What does it mean when a study is generalizable?

      If the results of a study can be applied to a larger population, we say the study is generalizable. Another term for generalizability is external validity.

      What are the critical concepts one needs to know when interpreting the results of a study?

      We trust the scientific method to give us reliable knowledge. Nevertheless, research should never be taken at face value. There are many ways a study can be biased and it is extremely important to be able to interpret the results of a study critically. The issue of validity is of particular importance. Are the results valid or is the study flawed to the extent that the conclusions are not supported by the data? Internal validity refers to the integrity of the study methods. Is there a fatal flaw that is intrinsic to the design of the study? For example, a study comparing the effectiveness of two drugs used one drug that had passed its expiration date. In this case, drug B may be less effective than drug A simply because it passed its expiration date. External validity refers to the extent to which the results can be applied to a larger population. A study of attitudes toward religion that only includes atheists will have limited external validity. It may be an accurate measure of the subjects’ religious beliefs but the study would not tell us much about non-atheists. In general, internal validity is more important than external validity.

      PSYCHOLOGICAL TESTS

      What role do psychological tests play in the science of psychology?

      Psychological tests are the bread and butter—the currency—of psychological science. Research in psychology depends upon the measurement of psychological traits, which can only be accomplished with psychological tests. Nonetheless, psychological traits are inherently difficult to assess. They are not concrete objects that are obviously measured, like the number of green peas or the height of a giraffe. They are abstract and intangible traits like love or happiness or self-esteem that can neither be seen, touched, nor counted and may be interpreted differently by different people. Therefore a critical part of psychological research involves the construction of tests that can measure psychological traits in a systematic and reliable way.

      What are the different kinds of tests?

      There are many forms of psychological tests, all of which offer both advantages and disadvantages. Perhaps the most common form