Modes of Speciation

The most common definition for a species we use is the biological species concept: a population that is reproductively isolated from another population is a different species.

Reproductive isolation can be prezygotic, i.e. from before copulation. For example, grasshoppers with different mating calls will not attract each other and mating will not take place. Alternatively, reproductive isolation can be postzygotic, i.e. it’s the offspring that causes the problems by having lower fitness. What that means is that the offspring will reproduce less, due to either environmental factors (e.g. higher predation) or due to intrinsic factors (genetically-mediated sterility, professionally called Dobzhansky-Muller (DM) incompatibilities).

Those are the very basics of what you need to know to understand this post (and speciation in general), where I will go into how a new species can originate.

Most commonly, species arise by allopatry. A barrier (geographic, climatic, anthropogenic) arises and splits one population in two, with very litte to no gene flow between them (in other words, no dispersal). These two population then evolve their own way, accumulating mutations. This genetic divergence eventually leads to them becoming reproductively isolated, i.e. you get two new species from an ancestral one.

Allopatry comes in two flavours (Wiley & Mayden, 1985): vicariance is when a barrier arises naturally (river course change, mountain-building, road), and dispersal is when migration of a segment of the population over a geographical barrier leads to isolation; see the diagram above (Stigall, 2012). We can differentiate between them by taking a historical view with the help of phylogenetics and biogeography (Lieberman, 2000), and several algorithms are available for the analysis; I recommend LBPA (Lieberman, 2000) or PACT (Wojcicki & Brooks, 2005). In vicariance, the range of both daughter species is the same as the original species. In dispersal, one of the daughter species covers a geographical area not covered by the original species due to the migration (Lieberman & Eldredge, 1996). In general, vicariance is much more common than dispersal (Congreve & Lieberman, 2010), and where vicariance hasn’t played a role, speciation rates are much lower (Stigall, 2010).

Allopatry is probably the most documented, and also the easiest one to demonstrate, because barriers tend to be noticeable; complexity only arises if there is secondary remixing of the populations.

Related to dispersal is peripatric speciation, when the migrated segment of the population encounters a different ecology and thus adapts to it, causing eventual speciation. An example of this can be found in some Nephila golden orb spider sister species (Su et al., 2011).

However, there are also other ways in which species can form that don’t require actual physical barriers. Sympatric speciation refers to speciation that occurs in a barrier-less geographical area; parapatric speciation is when the distribution of the populations don’t overlap except at the boundaries. In both these cases, gene flow between the two populations is still possible – there are no abiotic factors in the way, at least. Of course, many non-interbreeding species always exist in sympatry (a farm has 50+ insect species, and several birds and mammal species); we refer to each population’s genetic neighbourhood (Wright, 1946), i.e. the area from which parent individuals can come from, and it’s this area that is continuous with no abiotic barriers.

The steps leading to new species in this way are more complex than in allopatric cases, obviously, mostly because they require the strong action of natural selection, something not implicit in allopatric speciation. To make it easier to imagine, picture our populations inhabiting a large geographical area, perhaps with several habitat types. Despite the constant high possibility of gene flow, those organisms in the same habitat will tend to reproduce with each other – it’s safer than travelling a distance. We’ll call these ones Population A; the rest of the population will be Population B. This means that there will still be a small bit of isolation, allowing mutations to accumulate within Pop. A. Eventually, some of these mutations will have functional effects, in that they will cause DM incompatibilities in the offspring of Pop. A and Pop. B, or they will somehow negatively affect the fitness of the offspring.

These alleles become fixed by natural selection, since the net effect is that it’s AxA offspring that are reproductively fit. AxB offspring have less chance of producing offspring of their own. This is basically the start of a slippery slope. Eventually, these genetic incompatibilities will be so great that reproduction between Pop. A and Pop. B is futile. Speciation has thus occurred.

Sympatry can also occur under the action of sexual selection, where mating preferences and mate attraction provide the reproductive barrier (this happens in grasshoppers who might be coinhabiting, but have different mating calls, for example).

Needless to say, the greater complexity of these sympatric scenarios means that it is quite difficult to actually find it in nature; most of the time, what may appear to be sympatry is just as likely to be allopatry with secondary remixing (see e.g. Goodman et al., 1999). In fact, for a long time, sympatry was discounted out of mere theoretical grounds – divergence couldn’t come about in the face of gene flow (Wright, 1951).

But since then evidence supporting sympatry and parapatry has come in from theory (e.g. Navarro & Barton, 2003), where disruptive selection, sexual selection strongly affected by genes, and a large gene pool were identified as critical prerequisites for sympatic speciation. But it also has been seen, in unambiguous form, in nature (Coyne, 2007). An example is the study by Mattern & McLennan (2000) about speciation in felids, where the conclusion is that sympatric speciation is responsible for 51.8% of all felid speciation events. For example, the ranges of the the lion and the jaguar (sister groups) overlap in Western Asia, meaning that there is potential gene flow between the population, however the different ecologies of the cats means that they didn’t reproduce together.

The conclusions of the above study rely on the common sense inference that sympatric speciation will lead to sympatric sister species, i.e. the new species will overlap in their ranges at least temporarily. Bolnick & Fitzpatrick (2007) did a quantitative analysis of the occurrence of sympatric sister species in vertebrates and found that most (70+%) species are completely allopatric, with less than 10% being highly sympatric. However, it should be cautioned that this is biased towards lumbering idiot vertebrates; I would wager that if done on insects, nematodes, and other speciose invertebrates, the results would be different; but I know of no study to cite in support of that statement, it’s just an intuitive hunch.

This ubiquity of allopatry and rarity of sympatry led Coyne & Orr (2004) to propose allopatry as the default null hypothesis to use in speciation research.

One big area of research in speciation right now is the search for so-called speciation genes, ones that enable or enhance reproductive isolation in the face of high gene flow. While I was skeptical of their existence when I first heard of the concept, it would be silly of me to deny them: they exist and have been shown in all organismal groups from yeast (Lee et al., 2008) to insects (Presgraves et al., 2003) to vertebrates (Greene-Till et al., 2000). We can recognise them in two ways: they have the negative fitness effect, and they show signs of positive selection.

I already mentioned that sympatric speciation requires natural selection; allopatry might involve selection as well, but it’s not an integral part of the process. When this selection is divergent, we refer to the speciation that occurred as ecological speciation. In these cases, natural selection leads to phenotypic differences, which result in isolation, thus reinforcing the divergences.

However, speciation can also occur without divergent selection. Populations facing similar pressures might experience different mutations. These will lead to DM incompatibilities, isolating the compatible populations together and fixing the mutations. In these cases, the nature of selection is different: instead of driving divergence, it merely fixes the mutations. The selection is uniform across all the populations, without any “preference” shown to any mutation. Of course,t he mutations can also be fixed and/or reinforced by genetic drift with natural selection playing no role at all.

Another mode of speciation is polyploidy, a process commonly found in plants: 2-4% of angiosperms and 7% of ferns arose this way (Otto & Whitton, 2000). Polyploidy is unique in that the reproductive isolation is exclusively postzygotic, caused by genetic incompatibility between the mating individuals because of the wrong chromosome numbers – mating and fertilisation will occur, but the offspring will have odd-numbered ploidies (triploid, pentaploid, etc.) and produce sterile gametes with uneven chromosome numbers (aneuploids).

All these different modes of speciation should make one thing clear. The classical splitting of speciation into allopatric, sympatric and paraptric, while practical, is not very realistic. As with all things in nature, speciation patterns come on a wide spectrum, and all we do is erect arbitrary lines on that spectrum to make our science easier to do (Mallet et al., 2009); but it’s critical to realise the artificiality of these lines so we can recognise nuances when they arise (Schluter, 2001).

If you want more information on speciation, two leading textbooks are recommended. On the theoretical side, Gavrilets (2004); on the empirical side, Coyne & Orr (2004).


Bolnick DI & Fitzpatrick BM. 2007. Sympatric Speciation: Models and Empirical Evidence. Annual Review of Ecology, Evolution, and Systematics 38, 459-487.

Congreve CR & Lieberman BS. 2010. PHYLOGENETIC AND BIOGEOGRAPHIC ANALYSIS OF DEIPHONINE TRILOBITES. Journal of Paleontology 84, 128-136.

Coyne JA. 2007. Sympatric speciation. Current Biology 17, R787-R788.

Coyne JA & Orr HA. 2004. Speciation.

Gavrilets S. 2004. Fitness Landscapes and the Origin of Species.

Goodman SJ, Barton NH, Swanson G, Abernethy K & Pemberton JM. 1999. Introgression Through Rare Hybridization: A Genetic Study of a Hybrid Zone Between Red and Sika Deer (Genus Cervus) in Argyll, Scotland. Genetics 152, 355-371.

Greene-Till H, Zjao Y & Hardies SC. 2000. Gene flow of unique sequences between Mus musculus domesticus and Mus spretus. Mammalian Genome 11, 225-230.

Lee H-Y, Chou J-Y, Cheong L, Chang N-H, Yang S-H & Leu J-Y. 2008. Incompatibility of Nuclear and Mitochondrial Genomes Causes Hybrid Sterility between Two Yeast Species. Cell 135, 1065-1073.

Lieberman BS. 2000. Paleobiogeography: Using fossils to study global change, plate tectonics, and evolution.

Lieberman BS & Eldredge N. 1996. Trilobite Biogeography in the Middle Devonian: Geological Processes and Analytical Methods. Paleobiology 22, 66-79.

Mallet J, Meyer A, Nosil P & Feder L. 2009. Space, sympatry and speciation. Journal of Evolutionary Biology 22, 2332-2341.

Mattern MY & McLennan DA. 2000. Phylogeny and Speciation of Felids. Cladistics 16, 232-253.

Navarro A & Barton NH. 2003. Accumulating postzygotic isolation genes in parapatry: A new twist on chromosomal speciation. Evolution 57, 447-459.

Otto SP & Whitton J. 2000. POLYPLOID INCIDENCE AND EVOLUTION. Annual Review of Genetics 34, 401-437.

Presgraves DC, Balagopalan L, Abmayr SM & Orr HA. 2003. Adaptive evolution drives divergence of a hybrid inviability gene between two species of Drosophila. Nature 423, 715-719.

Schluter D. 2001. Ecology and the origin of species. TrEE 16, 372-380.

Stigall AL. 2010. Invasive Species and Biodiversity Crises: Testing the Link in the Late Devonian. PLoS ONE 5, e15584.

Stigall AL. 2012. Speciation collapse and invasive species dynamics during the Late Devonian “Mass Extinction”. GSA Today 22, 4-9.

Su Y-C, Chang Y-H, Smith D, Zhu M-S, Kuntner M & Tso I-M. 2011. Biogeography and Speciation Patterns of the Golden Orb Spider Genus Nephila (Araneae: Nephilidae) in Asia. Zoological Science 28, 47-55.

Wiley EO & Mayden RL. 1985. Species and Speciation in Phylogenetic Systematics, with Examples from the North American Fish Fauna. Annals of the Missouri Botanical Garden 72, 596-635.

Wojcicki M & Brooks DR. 2005. PACT: an efficient and powerful algorithm for generating area cladograms. Journal of Biogeography 32, 755-774.

Wright S. 1946. Isolation by Distance under Diverse Systems of Mating. Genetics 31, 39-59.

Wright S. 1951. THE GENETICAL STRUCTURE OF POPULATIONS. Annals of Eugenics 15, 323-354.

Research Blogging necessities :)

Bolnick, D., & Fitzpatrick, B. (2007). Sympatric Speciation: Models and Empirical Evidence Annual Review of Ecology, Evolution, and Systematics, 38 (1), 459-487 DOI: 10.1146/annurev.ecolsys.38.091206.095804
Wiley, E., & Mayden, R. (1985). Species and Speciation in Phylogenetic Systematics, with Examples from the North American Fish Fauna Annals of the Missouri Botanical Garden, 72 (4) DOI: 10.2307/2399217

One Comment

  1. Pingback: Why ancient lakes are hotspots for evolutionary biology research | Teaching Biology

Leave a Reply