In this paper, an innovative Physical Model-driven Neural Network (PMNN) method is proposed to solve time-fractional differential equations. It establishes a temporal iteration scheme based on physical model-driven neural networks which effectively combines deep neural networks (DNNs) with interpolation approximation of fractional derivatives. Specifically, once the fractional differential operator is discretized, DNNs are employed as a bridge to integrate interpolation approximation techniques with differential equations. On the basis of this integration, we construct a neural-based iteration scheme. Subsequently, by training DNNs to learn this temporal iteration scheme, approximate solutions to the differential equations can be obtained. The proposed method aims to preserve the intrinsic physical information within the equations as far as possible. It fully utilizes the powerful fitting capability of neural networks while maintaining the efficiency of the difference schemes for fractional differential equations. The experimental results show that the PMNN maintains precision comparable to traditional methods while exhibiting superior computational efficiency. This implies the potential of PMNN in addressing large-scale problems. Moreover, when considering both error and convergence rate, PMNN consistently outperforms fPINN. Additionally, the performance of PMNN on $\mathrm{L}2-1_{\sigma }$ surpasses that on $\mathrm{L}1$ in an overall comparison. The data and code can be found at https://github.com/DouMiao1226/PMNN.