This study explored the breakage kinetics of cement clinker in a lab-scale ball mill loaded with steel or alumina balls of 20, 30, and 40 mm sizes and their mixtures. The temporal evolution of the particle size distribution (PSD) was measured by sieving and laser diffraction. A global optimizer-based back-calculation method, based on a population balance model (PBM), was developed to estimate the breakage parameters. The ball motion in the mill was simulated via discrete element method (DEM). Our results show that steel balls achieved faster breakage of clinker into finer particles than alumina balls, which was explained by the higher total–mean energy dissipation rates of the steel balls. The PSD became finer as smaller balls were used. This study suggests that replacement of steel balls with alumina balls in continuous ball mills could provide significant energy savings if one accounts for the slower breakage with the alumina balls.