Speciation | Varsity Tutors

SECTION 1

Historical Context & Motivation

The question of how new species originate has captivated naturalists for centuries, but it was not until the mid-nineteenth century that a mechanistic framework began to take shape. Before Darwin, most Western scholars adhered to a fixist view of species — the idea that each species was independently created and immutable. Observations of geographic variation in island fauna, fossil succession, and selective breeding of domesticated organisms gradually eroded this view and opened the door to evolutionary explanations. The intellectual journey from recognizing that species change to understanding how one species splits into two has driven some of the most productive research programs in modern biology.

1859

Darwin's Origin of Species

Charles Darwin publishes On the Origin of Species, articulating natural selection as the engine of evolutionary change. Although Darwin discussed divergence of character, his treatment of the actual splitting event — speciation — remained relatively vague.

1937

Dobzhansky's Genetics and the Origin of Species

Theodosius Dobzhansky merges Mendelian genetics with Darwinian evolution, formally defining species through the lens of reproductive isolation. His work establishes the conceptual bridge between population genetics and speciation.

1942

Mayr's Biological Species Concept

Ernst Mayr formalizes the biological species concept (BSC), defining species as groups of actually or potentially interbreeding populations reproductively isolated from other such groups. The BSC becomes the dominant framework for studying speciation in animals.

1963–1981

Mayr's Peripatric Model & Punctuated Equilibrium

Mayr elaborates peripatric speciation (founder-effect speciation), and Eldredge & Gould propose punctuated equilibrium in 1972, arguing that speciation events are geologically rapid and followed by long periods of stasis, reshaping how paleontologists interpret the fossil record.

1990s–present

Genomic Era of Speciation Research

Advances in molecular phylogenetics, genome-wide association studies, and population genomics allow researchers to identify specific speciation genes (e.g., Dobzhansky-Muller incompatibility loci) and trace gene flow during divergence, revealing that speciation is often a protracted, leaky process rather than a single, clean event.

The central question that speciation research addresses is deceptively simple: how does one interbreeding population become two reproductively isolated lineages? Answering this question requires integrating ecology, genetics, geography, and behavior — and the answers differ depending on the geographic, genetic, and selective context in which divergence occurs.

SECTION 2

Core Principles & Definitions

Before examining the mechanisms that drive speciation, it is essential to establish the foundational concepts. Speciation is the evolutionary process by which populations evolve to become distinct species, typically through the accumulation of reproductive isolating mechanisms — barriers that prevent formerly interbreeding populations from exchanging genes. These barriers may be ecological, behavioral, mechanical, gametic, or genetic in nature, and they can act before fertilization (prezygotic barriers) or after (postzygotic barriers). The interplay of geography, natural selection, genetic drift, and mutation determines which mode of speciation predominates in any given lineage.

Reproductive Isolation

The hallmark of speciation under the biological species concept. Reproductive isolation occurs when populations can no longer produce viable, fertile offspring — whether due to geographic separation, ecological divergence, or intrinsic genetic incompatibilities.

Gene Flow

The transfer of alleles between populations through migration and interbreeding. Gene flow homogenizes populations; its reduction or cessation is a prerequisite for divergence in most speciation models.

Genetic Divergence

Once gene flow is curtailed, populations accumulate distinct mutations through drift and selection. Divergence at enough loci — particularly those involved in mate recognition or hybrid viability — eventually leads to irreversible reproductive isolation.

Prezygotic vs. Postzygotic Barriers

Prezygotic barriers (habitat isolation, temporal isolation, behavioral isolation, mechanical isolation, gametic isolation) prevent the formation of a zygote. Postzygotic barriers (hybrid inviability, hybrid sterility, hybrid breakdown) reduce hybrid fitness after fertilization.

Reinforcement

When incipient species come into secondary contact and hybrids are less fit, natural selection favors individuals that preferentially mate with their own type. This process, called reinforcement (or the Wallace effect), strengthens prezygotic barriers and accelerates the completion of speciation.

✦ KEY TAKEAWAY

Think of gene flow as a rope tethering two boats (populations) together. As long as the rope holds, the boats drift in roughly the same direction. Speciation is the process of fraying and severing that rope — through geography, selection, or both — so each boat charts an independent evolutionary course. Once the rope is cut and the boats have drifted far enough apart, even retying them (secondary contact) cannot merge their trajectories because the accumulated differences in hull shape, rigging, and crew (genes) make them fundamentally incompatible.

SECTION 3

Modes of Speciation — Visual Overview

Speciation events are traditionally classified by the geographic context in which divergence occurs. The three primary modes — allopatric, parapatric, and sympatric — differ fundamentally in the spatial relationship between diverging populations and, consequently, in the relative roles of drift, selection, and gene flow during the speciation process. The following diagram illustrates these three geographic modes alongside a fourth mechanism, polyploidy, which can produce instantaneous reproductive isolation through chromosomal duplication.

The four panels illustrate the major modes of speciation. Allopatric speciation involves complete geographic separation. Parapatric speciation occurs across a gradient with a narrow hybrid zone. Sympatric speciation proceeds without any geographic barrier via disruptive selection and assortative mating. Polyploidy produces instant isolation through chromosomal incompatibility. The bottom bar shows that the difficulty of evolving reproductive isolation increases as gene flow during divergence increases.

The diagram underscores a fundamental principle: the likelihood and speed of speciation are inversely related to the amount of gene flow between diverging populations. Allopatric speciation, in which gene flow is eliminated by a physical barrier such as a mountain range or ocean strait, is widely considered the most common mode in animals. Sympatric speciation, by contrast, requires very strong disruptive selection and assortative mating to overcome the homogenizing effect of ongoing gene flow, making it theoretically possible but empirically rarer — the classic example being host-race formation in Rhagoletis pomonella (the apple maggot fly). Parapatric speciation occupies an intermediate position, occurring along environmental gradients where selection across the cline is strong enough to counteract gene flow in the contact zone. Polyploidy stands apart from these geographic models because whole-genome duplication can create a new species in a single generation, a mechanism particularly important in plants (an estimated 15% of angiosperm speciation events involve polyploidy).

SECTION 4

Genetic Mechanisms of Reproductive Isolation

While geographic context sets the stage for speciation, the genetic mechanisms that produce reproductive isolation are what ultimately create new species. Two interrelated genetic models — the Dobzhansky-Muller incompatibility (DMI) model and reinforcement — provide the theoretical backbone for understanding how intrinsic postzygotic and prezygotic barriers evolve, respectively.

Dobzhansky-Muller Incompatibility Model

The DMI model elegantly explains how postzygotic isolation can evolve without requiring any population to pass through a maladaptive intermediate. Consider an ancestral population with genotype AABB at two independently assorting loci. After geographic separation, Population 1 fixes a new allele a at locus A (becoming aaBB), while Population 2 independently fixes allele b at locus B (becoming AAbb). Within each population, each new allele is compatible with its genetic background; a has been tested alongside B, and b alongside A. However, alleles a and b have never been tested together. In hybrids (genotype AaBb), the novel a–b interaction may cause developmental failure, sterility, or inviability — a genetic incompatibility that neither parental population ever experienced.

SNOWBALL EFFECT — DMI ACCUMULATION

Number of potential DMIs ∝ n(n − 1)/2 ≈ n²/2

Where n = number of substitutions that have accumulated between two lineages. Because each new substitution can potentially be incompatible with every previous substitution in the other lineage, the number of potential pairwise incompatibilities grows approximately as n squared — the so-called snowball effect (Orr 1995). This means that reproductive isolation accelerates over time as divergence accumulates.

Reinforcement and Character Displacement

When partially isolated populations come into secondary contact, hybrids may exhibit reduced fitness due to DMIs. Under these circumstances, natural selection favors individuals that avoid hybridizing — for example, through divergence in mating calls, coloration, or flowering time. This selective strengthening of prezygotic barriers is reinforcement. A hallmark prediction of reinforcement is reproductive character displacement: mating traits (e.g., song frequency, body size used in mate choice) diverge more in sympatry than in allopatry. This pattern has been documented in Drosophila species pairs, chorus frogs, and many other taxa.

HARDY-WEINBERG DEVIATION AS EVIDENCE OF ASSORTATIVE MATING

F_IS = 1 − (H_obs / H_exp)

Where F_IS is the inbreeding coefficient within subpopulations, H_obs is observed heterozygosity, and H_exp is expected heterozygosity under random mating. Positive F_IS values indicate a deficit of heterozygotes consistent with assortative mating or a Wahlund effect, both of which are relevant during the early stages of speciation.

Haldane's Rule and the Genetics of Hybrid Breakdown

Haldane's rule (1922) states that when one sex of an interspecific hybrid is inviable or sterile, it is more often the heterogametic sex (XY in mammals, ZW in birds). This pattern is one of the most robust generalizations in speciation biology and arises because recessive incompatibility alleles on the X (or Z) chromosome are exposed in the heterogametic sex, which lacks a second copy to mask them. Haldane's rule has been confirmed across Drosophila, mammals, birds, Lepidoptera, and other taxa, and it provides a powerful framework for predicting which crosses will fail and at what developmental stage.

SECTION 5

Reproductive Isolating Barriers — Detailed Classification

Reproductive isolation is rarely the product of a single barrier; instead, multiple prezygotic and postzygotic mechanisms typically act in concert to prevent gene exchange between incipient species. Understanding these barriers in detail is critical because their relative strength determines where in the mating-to-offspring pipeline reproductive isolation is most effective. The following diagram and table provide a comprehensive classification.

This diagram illustrates the sequential nature of reproductive barriers. An individual must navigate each barrier stage — from encounter to offspring fitness — for gene flow to occur. Prezygotic barriers (left of the red line) are generally considered more effective at reducing gene flow because they act earlier in the reproductive pipeline, preventing wasted gametic investment.

Major reproductive isolating barriers with representative examples

Barrier Type	Stage	Mechanism	Classic Example
Habitat isolation	Prezygotic	Populations occupy different microhabitats within the same area	Thamnophis garter snakes: aquatic vs. terrestrial ecotypes
Temporal isolation	Prezygotic	Species breed at different times of day, season, or year	Eastern spotted skunk (spring) vs. western spotted skunk (fall)
Behavioral isolation	Prezygotic	Differences in courtship signals or mate preferences	Blue-footed vs. red-footed boobies — foot color in display
Mechanical isolation	Prezygotic	Incompatible reproductive structures prevent copulation or pollination	Sage species pollinated by different bee/hummingbird species
Gametic isolation	Prezygotic	Molecular incompatibility between egg and sperm surface proteins	Sea urchin bindin protein specificity
Hybrid inviability	Postzygotic	Hybrid embryos fail to develop properly due to genetic incompatibilities	Sheep × goat crosses rarely survive past early embryogenesis
Hybrid sterility	Postzygotic	Hybrids are viable but fail to produce functional gametes	Mule (horse × donkey): vigorous but sterile due to chromosome mispairing
Hybrid breakdown	Postzygotic	F₁ hybrids are fertile, but F₂ or later generations show reduced fitness	Some rice cultivar crosses produce vigorous F₁ but weak F₂ offspring

SECTION 6

Worked Example — Analyzing Speciation in Darwin's Finches

The Galápagos finches (genus Geospiza, Camarhynchus, etc.) represent one of the most thoroughly studied adaptive radiations in evolutionary biology. The following worked example walks through the reasoning a biologist would use to identify the mode of speciation, the operating barriers, and the genetic signature of divergence in this system.

Identifying Mode of Speciation in Galápagos Finches

Step 1 — Establish Geographic ContextThe Galápagos archipelago consists of isolated volcanic islands separated by ocean channels. A mainland ancestor colonized one island, and subsequent dispersal events carried finch populations to other islands. Because each island population was geographically separated from the others — with ocean serving as a barrier to routine gene flow — the initial divergence most likely followed the allopatric model. Each island effectively functions as a replicate natural experiment in independent evolution.

Mode: allopatric speciation (vicariance across island barriers)

Step 2 — Identify Selective Pressures Driving DivergenceDifferent islands offer different food resources: hard seeds, soft seeds, insects, cactus nectar, and fruits. Natural selection on each island favored beak morphologies optimized for the locally available diet. The classic research by Peter and Rosemary Grant documented natural selection on beak depth in Geospiza fortis during drought years, directly demonstrating that beak size is under strong ecological selection. Over many generations, such differential selection across islands produces divergent natural selection — the driver of adaptive divergence.

Selective agent: ecological divergence driven by food resource partitioning

Step 3 — Determine Which Reproductive Barriers OperateWhen formerly allopatric finch species come into secondary contact on the same island, what prevents them from interbreeding? Studies show that behavioral isolation through song type is a major prezygotic barrier: females preferentially mate with males that sing their species-specific song, which is culturally learned from fathers. Additionally, beak morphology itself serves as a visual signal in mate assessment, creating a link between ecological adaptation and assortative mating — a phenomenon termed magic trait speciation, where the same trait is simultaneously under natural selection and sexual selection.

Primary barriers: behavioral isolation (song) and ecological character displacement (beak size divergence in sympatry)

Step 4 — Evaluate Evidence for ReinforcementIf reinforcement is occurring, we expect greater divergence in mating signals in sympatry than in allopatry. The Grants' data show that G. fortis and G. fuliginosa have more divergent beak sizes on islands where both species coexist than on islands where only one is present. This pattern of character displacement is consistent with reinforcement: natural selection disfavors hybrids with intermediate beak sizes (poorly suited to either seed type), strengthening prezygotic barriers.

Reinforcement signature detected: reproductive character displacement in beak size in sympatry

Step 5 — Identify Genetic BasisMolecular studies have identified the ALX1 gene (a transcription factor) as a major contributor to variation in beak shape, and HMGA2 as influencing beak size. Genome-wide analyses reveal that divergence is concentrated in relatively few genomic regions — 'islands of divergence' — while much of the genome remains similar between species, consistent with recent speciation with incomplete lineage sorting.

Key genes: ALX1 (beak shape), HMGA2 (beak size) — genomic islands of divergence in a background of shared ancestry

SECTION 7

Comparing Modes of Speciation — Strengths & Limitations

Each mode of speciation has distinct theoretical requirements, empirical support, and limitations. Recognizing these differences is important not only for academic understanding but also for predicting how speciation may proceed in different taxa under different ecological and geographic circumstances.

Comparison of the four major modes of speciation

Mode	Gene Flow During Divergence	Strengths / Evidence	Limitations / Criticisms
Allopatric	None (complete barrier)	Strongest empirical support; observed in island biogeography, isthmus closures (e.g., Panama), mountain uplift. Compatible with all genetic models of divergence.	Difficult to rule out brief periods of contact. Vicariance events may be hard to date. Does not predict divergence rate well without additional ecological information.
Parapatric	Restricted (across cline)	Hybrid zones provide natural laboratories. Cline theory offers quantitative predictions. Documented in ring species (e.g., Ensatina salamanders).	Hybrid zones may be stable for long periods without completing speciation. Hard to distinguish from secondary contact after allopatric divergence.
Sympatric	Ongoing (same area)	Compelling cases in cichlid fishes, Rhagoletis apple maggot fly, palm species on Lord Howe Island. Demonstrates that geography is not always necessary.	Very restrictive theoretical conditions (strong disruptive selection + assortative mating). Hard to exclude past allopatry. Few unambiguous examples.
Polyploidy	N/A (instant isolation)	Creates immediate reproductive isolation. Very common in plants (angiosperms, ferns). Allopolyploidy can generate novel phenotypes.	Rare in animals (except some fish, amphibians). New polyploid faces minority disadvantage — must find other polyploid mates or self-fertilize.

✦ KEY TAKEAWAY

The boundaries between speciation modes are not as sharp as textbook categories suggest. Think of speciation modes as a continuum rather than discrete bins — analogous to the electromagnetic spectrum, where radio waves blend into microwaves without a sharp boundary. In practice, many speciation events involve periods of geographic isolation interspersed with episodes of gene flow (so-called speciation with gene flow), making it difficult to assign real-world cases to a single geographic category. Modern genomic data consistently reveal that speciation is often a messy, protracted process.

SECTION 8

Connections to Advanced Theory — Species Concepts and Genomic Speciation

The study of speciation is intimately linked to how we define species, and the choice of species concept has profound implications for how speciation events are detected and counted. The biological species concept (BSC) remains the most widely used framework for sexually reproducing organisms, but it fails for asexual lineages, fossils, and many groups with extensive hybridization. Alternative concepts — the morphological, phylogenetic, ecological, and cohesion species concepts — each capture different aspects of species identity and highlight different criteria for what constitutes a speciation event.

Major species concepts and their implications for speciation research

Species Concept	Criterion for Species Status	Relationship to Speciation
Biological (BSC)	Reproductive isolation from other groups	Speciation = evolution of reproductive barriers
Phylogenetic (PSC)	Smallest diagnosable monophyletic group	Speciation = lineage splitting; may detect 'species' before reproductive isolation is complete
Ecological	Adaptive zone or ecological niche distinctness	Speciation = ecological divergence; emphasizes the role of natural selection
Unified / General Lineage (de Queiroz)	Independently evolving metapopulation lineage	Speciation = lineage divergence; all other criteria (reproductive isolation, monophyly, ecological distinction) are contingent properties that accumulate over time

Genomic Islands of Speciation

Advances in population genomics have revealed that genetic divergence during speciation is often heterogeneous across the genome. Rather than the entire genome diverging uniformly, small genomic regions — termed genomic islands of speciation — show elevated F_ST (genetic differentiation) while the rest of the genome retains high gene flow. These islands often harbor genes under divergent selection or genes contributing to reproductive isolation, and they may be located in regions of reduced recombination (near centromeres or within inversions) that resist introgression. This view of heterogeneous genomic divergence challenges the classical model of genome-wide barriers and suggests that speciation can proceed gene by gene, with isolation at certain loci preceding isolation at others.

🔬 Looking Ahead

In advanced evolutionary genetics courses, you will encounter mathematical models of speciation continuum theory, the genomic basis of reproductive isolation (speciation genes such as Hmr and Lhr in Drosophila), and the application of coalescent theory to distinguish between incomplete lineage sorting and introgression. Understanding the foundational concepts presented in this lesson will be essential for engaging with these more sophisticated frameworks.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

A river changes course and permanently divides a population of salamanders into two groups. Over many generations, the two populations evolve different skin coloration, body size, and courtship behaviors. When researchers bring individuals from the two populations together in the lab, they show no interest in mating with each other. Identify the mode of speciation and classify the primary reproductive barrier that prevents interbreeding.

PROBLEM 2 — BASIC CALCULATION

According to the Dobzhansky-Muller incompatibility model, the number of potential pairwise incompatibilities grows as n(n − 1)/2, where n is the number of substitutions that differ between two lineages. If Lineage A has accumulated 8 unique substitutions and Lineage B has accumulated 12, calculate the maximum number of potential pairwise incompatibilities between the two lineages' derived alleles.

PROBLEM 3 — INTERMEDIATE

Two closely related bird species hybridize in a narrow zone where their ranges overlap. Hybrids have intermediate plumage coloration and suffer approximately 40% reduced survival compared to parentals. Over the past 50 years, researchers have documented that female birds in the overlap zone have become significantly more choosy about male plumage color, preferring males with species-typical coloration. Meanwhile, no such increase in female choosiness has been observed in populations far from the hybrid zone. (a) What evolutionary process does this pattern suggest? (b) What specific prediction can you make about prezygotic isolation in sympatry versus allopatry? (c) What condition is necessary for this process to work?

PROBLEM 4 — APPLIED

Conservation biologists discover that a dam constructed 60 years ago has divided a population of freshwater fish into upstream and downstream groups. Genetic analysis reveals an F_ST of 0.05 between the two groups, with elevated divergence at a few loci associated with water temperature tolerance (the reservoir above the dam is warmer). Crosses between upstream and downstream fish in the lab produce viable, fertile offspring. (a) Are these populations separate species under the BSC? (b) What mode of speciation might be initiated here? (c) What additional data would you want to determine whether speciation is likely to be completed?

PROBLEM 5 — CRITICAL THINKING

A research group studies two sympatric species of cichlid fish in an African lake. Species A feeds on algae scraped from rocks in shallow water, while Species B feeds on zooplankton in open water. The two species have distinct male coloration (Species A is blue, Species B is yellow), and females strongly prefer males of their own color morph. When the researchers experimentally increase water turbidity using controlled lighting, female mate preference breaks down, hybridization rate increases dramatically, and the resulting hybrids show intermediate coloration and feeding morphology. Using your knowledge of speciation, (a) explain why turbidity causes the collapse of reproductive isolation, (b) discuss what this experiment reveals about the nature of the isolating barrier, (c) evaluate whether this system provides evidence for sympatric speciation, and (d) predict the evolutionary outcome if lake turbidity increases permanently due to eutrophication.

SUMMARY

Speciation — Key Concepts Review

Speciation is the evolutionary process by which one lineage splits into two or more reproductively isolated species. The geographic context of divergence — allopatric (complete geographic separation), parapatric (adjacent ranges with a hybrid zone), sympatric (same area, no barrier), or polyploidy (instantaneous chromosomal duplication) — determines the relative contributions of genetic drift, natural selection, and gene flow to the speciation process. Reproductive isolating barriers are classified as prezygotic (habitat, temporal, behavioral, mechanical, gametic) or postzygotic (hybrid inviability, sterility, breakdown), and multiple barriers typically act in concert to restrict gene flow.

The Dobzhansky-Muller incompatibility model explains how postzygotic isolation evolves without maladaptive intermediates, with incompatibilities accumulating according to the snowball effect (proportional to n²). Reinforcement strengthens prezygotic barriers when incipient species come into secondary contact and hybrids are unfit, producing the diagnostic pattern of reproductive character displacement. Haldane's rule predicts that the heterogametic sex is more often inviable or sterile in crosses. Modern genomics reveals that divergence is often concentrated in genomic islands of speciation while much of the genome retains gene flow, underscoring that speciation is typically a protracted, heterogeneous process rather than a single clean break.