Improving the quantum coherence of solid-state systems is a decisive factor in realizing solid-state quantum technologies. The key to optimize quantum coherence lies in reducing the detrimental influence of noise sources such as spin noise and charge noise. Here we demonstrate that we can utilize highly-excited Rydberg excitons to neutralize charged impurities in the semiconductor Cuprous Oxide - an effect we call purification. Purification reduces detrimental electrical stray fields drastically. We observe that the absorption of the purified crystal increases by up to 25% and that the purification effect is long-lived and may persist for hundreds of microseconds or even longer. We investigate the interaction between Rydberg excitons and impurities and find that it is long-ranged and based on charge-induced dipole interactions. Using a time-resolved pump-probe technique, we can discriminate purification from Rydberg blockade, which has been a long-standing goal in excitonic Rydberg systems.