The genomes of cytoplasmic organelles (mitochondria and plastids) are maternally inherited in most eukaryotes, thus excluding organellar genomes from the benefits of sexual reproduction and recombination. The mechanisms underlying maternal inheritance are largely unknown. Here we demonstrate that two independently acting mechanisms ensure maternal inheritance of the plastid (chloroplast) genome. Conducting large-scale genetic screens for paternal plastid transmission, we discovered that mild chilling stress during male gametogenesis leads to increased entry of paternal plastids into sperm cells and strongly increased paternal plastid transmission. We further show that the inheritance of paternal plastid genomes is controlled by the activity of a genome-degrading exonuclease during pollen maturation. Our data reveal that (1) maternal inheritance breaks down under specific environmental conditions, (2) an organelle exclusion mechanism and a genome degradation mechanism act in concert to prevent paternal transmission of plastid genes and (3) plastid inheritance is determined by complex gene–environment interactions.