Response to selection from new mutation and effective size of partially inbred populations. I. Theoretical results

Summary The effects of partial inbreeding on effective population size and rates of fixation of mutant genes are investigated in selected populations. Truncation selection and an infinitesimal model of gene effects for the selected trait are assumed. Predictions of effective size under this model are given for partial selfing and partial full-sib mating and an extension to a more general model is outlined. The joint effect of selection and partial inbreeding causes a large reduction in the effective size relative to the case of random mating. This effect is especially remarkable for small amounts of selected genetic variation. For example, for initial heritability 0·1 and proportion selected 1/6, the ratio of effective size to population size is 0·10 in populations with about 90% selfing while it is 0·85 in random mating populations. The consequence is a reduction in the fixation probability of favourable genes and, therefore, a reduction in the final response to selection. Stochastic simulations are used to investigate the effects of partial inbreeding and selection on fixation and extinction rates of genes of large effect and of recessive lethals with effects on the selected trait. For genes of very large effect, the effective size is not a critical factor and it is expected that partial inbreeding will be efficient in increasing fixation rates of recessive mutants. Lethal recessives are eliminated more frequently and their equilibrium frequency is lower under partial inbreeding, but only when their effects on the heterozygote are not very large.