The sensitivity of recrystallization kinetics in metals to the heterogeneity of microstructure and deformation history is a widely accepted experimental fact. However, most of the available recrystallization models employ either a mean field approach or use grain-averaged parameters, and thus neglecting the mesoscopic heterogeneity induced by prior deformation. In the present study, we investigate the impact of deformation-induced dislocation (subgrain) structure on the kinetics of recrystallization in metals using the phase-field approach. The primary focus here is upon the role of dislocation cell boundaries. The free energy formulation of the phase-field model accounts for the heterogeneity of the microstructure by assigning localized energy to the resulting dislocation microstructure realizations generated from experimental data. These microstructure realizations are created using the universal scaling laws for the spacing and the misorientation angles of both the geometrically necessary and incidental dislocation boundaries. The resulting free energy is used into an Allen-Cahn based model of recrystallization kinetics, which are solved using the finite element method. The solutions thus obtained shed light on the critical role of the spatial heterogeneity of deformation in the non-smooth growth of recrystallization nuclei and on the final grain structure. The results showed that, in agreement with experiment, the morphology of recrystallization front exhibits protrusions and retrusions. By resolving the subgrain structure, the presented algorithm paves the way for developing predictive kinetic models that fully account for the deformed state of recrystallizing metals.