Climate change and human activities have greatly altered the ecological flow of rivers, and the conflict between human water use and natural water demand is becoming more and more prominent. Using two ecological flow indicators (ecodeficit and ecosurplus), this study focuses on assessing the characteristics of ecological flow changes at multiple time scales and introduces the Long Short-Term Memory model to construct a meteorological streamflow model for the Xiangjiang River (XJR) basin, using a separation framework to quantify the effects of human disturbance and climate change on ecological flow at multiple time scales. In addition, the fluvial biodiversity Shannon Index (SI) was used to assess the response processes of riverine ecosystems under changing conditions. The results show that the increase of XJR flow is larger (11%) after 1991, the increase in precipitation and potential evapotranspiration in the basin is 5.60%, and the decrease is 3.09%, respectively, and there are obvious cycles of all three on annual and seasonal scales. The annual ecosurplus increased, and the annual ecodeficit decreased after the hydrological variation; on the seasonal scale, the ecodeficit decreased significantly in summer and autumn, and the ecosurplus increased substantially in winter. Climatic factors were the main drivers of the increased frequency and magnitude of annual, summer, and fall high flows (91%, 94%, and 65% contributions, respectively), while urbanization expansion and reservoir diversions drove the increase in spring ecodeficit. Changes in river flow maintained the ecosurplus at a low level after 2002, further causing a decrease in river biodiversity, and the annual and summer ecosurplus were highly correlated with SI indicators (0.824 and 0.711, respectively). Our study contributes to the development of effective ecological flow regulation policies for the XJR basin.