Thermoelectrics enable direct heat-to-electricity transformation, but their performance has so far been restricted by the closely coupled carrier and phonon transport. Here, we demonstrate that the quantum gaps, a class of planar defects characterized by nano-sized potential wells, can decouple carrier and phonon transport by selectively scattering phonons while allowing carriers to pass effectively. We choose the van der Waals gap in GeTe-based materials as a representative example of the quantum gap to illustrate the decoupling mechanism. The nano-sized potential well of the quantum gap in GeTe-based materials is directly visualized by in situ electron holography. Moreover, a more diffused distribution of quantum gaps results in further reduction of lattice thermal conductivity, which leads to a peak ZT of 2.6 at 673 K and an average ZT of 1.6 (323–723 K) in a GeTe system. The quantum gap can also be engineered into other thermoelectrics, which provides a general method for boosting their thermoelectric performance.