Cell plasticity and neuroendocrine differentiation in prostate and lung adenocarcinomas are one of the major reasons for therapeutic resistance to targeted therapy. Whether and how metabolic changes contribute to this adenocarcinoma-to-neuroendocrine cell fate transition remains largely unclear. Here we show that neuroendocrine prostate or lung cancer cells possess mostly fragmented mitochondria with low membrane potential and rely on glycolysis for energy metabolism. We further show an important role of the cell fate determinant Numb in mitochondrial quality control via binding to Parkin and facilitating Parkin-mediated mitophagy. Deficiency in the Numb/Parkin pathway in prostate or lung adenocarcinomas causes a metabolic reprogramming featured with a significant increase in production of lactate acid, which subsequently leads to an upregulation of histone lactylation and transcription of neuroendocrine-associated genes. Collectively, the Numb/Parkin-directed mitochondrial fitness is a key metabolic switch and a promising therapeutic target on cancer cell plasticity through the regulation of histone lactylation.