While the anaerobic-anoxic-oxic (AAO) process is the most widely applied biological wastewater treatment process in municipal wastewater treatment plants (WWTPs), it struggles to meet the increasing demands on biological toxicity control of the treated effluent. To tackle this challenge, this study develops machine learning (ML)-based models for optimizing the AAO treatment process towards improving its toxicity reduction efficacy for the effluent. The water quality parameters, treatment process parameters, and biological toxicity information (based on the nematode bioassay) of the effluent collected from 122 WWTPs in China are used to train the models. The validated models accurately predict the effluent’s quality parameters (average R² = 0.81) and the biological toxicity reduction ratio of treatment process (R² = 0.86). To further improve the toxicity reduction, we developed a multiple objective optimization framework to optimize the AAO process via unit process recombination. In the short-range unit combination, the toxicity reduction ratio of the four-unit combined processes (up to 79.8% of anaerobic-aerobic-anaerobic-aerobic) is significantly higher than others. After optimization, it helps to improve the average toxicity reduction efficacy of 122 WWTPs from 48.6% to 70.7%, with a maximum of 87.5%. The methodologies and findings derived from this work are expected to provide the foundation for the optimization, expansion, and technical transformation of biological wastewater treatment in WWTPs.