Population Genetics. Matthew B. Hamilton

2pq – F2pq d2pq – dF2pq aa −a q² + Fpq −aq² – aFpq

      population mean: ap² + d2pq – dF2pq ‐ aq² = a(p‐q) + d2pq(1‐F)

      Heterosis: The increase in performance, survival, and ability to reproduce of individuals possessing heterozygous loci (hybrid vigor); increase in the population average phenotype associated with increased heterozygosity.

      Inbreeding depression: The reduction in performance, survival, and ability to reproduce of individuals possessing homozygous genotypes; decrease in the population average phenotype associated with mating among relatives that increases homozygosity.

      There is evidence that humans possess homozygosity because of mating among relatives and also experience inbreeding depression. Genome‐scale genetic marker data in humans has revealed stretches of the genome where both chromosomes possess identical alleles called runs of homozygosity that are explained by both recent family members being related as well as by the history of population size and mixing (reviewed by Ceballos et al. 2018). There is also evidence for inbreeding depression in humans based on observed phenotypes in the offspring of couples with known consanguinity. For example, mortality among children of first‐cousin marriages was about 3.5% greater than for marriages between unrelated individuals measured in a range of human populations (Bittles and Black 2010). Human studies have utilized existing parental pairs with relatively low levels of inbreeding, such as uncle/niece, first cousins, or second cousins, in contrast to animal and plant studies where both very high levels and a broad range of coancestry coefficients are achieved intentionally. Drawing conclusions about the causes of variation in phenotypes from such observational studies requires caution, since the prevalence of consanguineous mating in humans is also correlated with social and economic variables such as illiteracy, age at marriage, duration of marriage, and income. These latter variables are therefore not independent of consanguinity and can themselves contribute to variation in phenotypes such as fertility and infant mortality (see Bittles and Black 2010).

The Mendelian genetic causes of inbreeding depression have been a topic of population genetics research for more than a century. None other than Charles Darwin carried out experimental pollinations in numerous plant species and observed that progeny of self‐fertilization were shorter and produced fewer seeds than outcrossed progeny (Darwin 1876). There are two classical hypotheses to explain inbreeding depression and changes in fitness as the fixation index increases (Charlesworth and Charlesworth 1999; Carr et al. 2003). Both hypotheses predict that levels of inbreeding depression will increase along with consanguineous mating that increases homozygosity, although for different reasons (Table 2.11). The first hypothesis, often called the dominance hypothesis, is that increasing homozygosity increases the phenotypic expression of fully and partly recessive alleles with deleterious effects. The second hypothesis is that inbreeding depression is the result of the decrease in the frequency of heterozygotes that occurs with consanguineous mating. This explanation supposes that heterozygotes have higher fitness than homozygotes (heterosis) and is called the overdominance hypothesis. In addition, the fitness interactions of alleles at different loci (epistasis; see Chapter 9) may also cause inbreeding depression, a hypothesis that is particularly difficult to test (see Carr and Dudash 2003). These causes of inbreeding depression may all operate simultaneously.

Table 2.11 A summary of the Mendelian basis of inbreeding depression under the dominance and overdominance hypotheses along with predicted patterns of inbreeding depression with continued consanguineous mating.

Hypothesis	Mendelian basis	Low fitness genotypes	Changes in inbreeding depression w/ continued consanguineous mating
Dominance	recessive and partly	only homozygotes for	purging of deleterious alleles that
	recessive deleterious alleles	deleterious recessive	is increasingly effective as degree of
		alleles	recessiveness increases
Overdominance	heterozygote advantage	all homozygotes	no changes as long as consanguineous mating keeps heterozygosity low
	or heterosis

      These dominance and overdominance hypotheses make different testable predictions about how inbreeding depression (measured as the average phenotype of a population) will change over time with continued consanguineous mating. Under the dominance hypothesis, recessive alleles that cause lowered fitness are more frequently found in homozygous genotypes under consanguineous mating. This exposes the deleterious phenotype and the genotype will decrease in frequency in a population by natural selection (individuals homozygous for such alleles have lower survivorship and reproduction). This reduction in the frequency of deleterious alleles by natural selection is referred to as purging of genetic load. Purging increases the frequency of alleles that do not have deleterious effects when homozygous, so that the average phenotype in a population then returns to the initial average it had before the onset of consanguineous mating. In contrast, the overdominance hypothesis does not predict a purging effect with consanguineous mating. With consanguineous mating, the frequency of heterozygotes will decrease and not recover until mating patterns change (see Figure 2.12). Even if heterozygotes are frequent and have a fitness advantage, each generation of mating and Mendelian segregation will reconstitute the two homozygous genotypes so purging cannot occur. These predictions highlight the major difference between the hypotheses. Inbreeding depression with overdominance arises from genotype frequencies in a population, while inbreeding depression with dominance is caused by the frequency of deleterious recessive alleles in a population. Models of natural selection that are relevant to inbreeding depression on population genotype and allele frequencies receive detailed coverage in Chapter 6.

Inbreeding depression in many animals and plants appears to be caused, at least in part, by deleterious recessive alleles consistent with the dominance hypothesis (Byers and Waller 1999; Charlesworth and Charlesworth 1999; Crnokrak and Barrett 2002). A classic example of inbreeding depression and recovery of the population mean for litter size in mice is shown in Figure 2.17. Model research organisms such as mice, rats, and Drosophila, intentionally bred by schemes such as full sibling mating for 10s or 100s of generations to create highly homozygous, so‐called pure‐breeding lines, are also not immune to inbreeding depression. Such inbred lines are often founded from multiple families, and many of these family lines go extinct from low viability or reproductive failure with habitual inbreeding. This is another type of purging effect due to natural selection that leaves only those lines that exhibit less inbreeding depression, which could be due to dominance, overdominance, or epistasis.