Peony pod is a favorable natural source of flavonoids, which is faced with enormous losses of resources due to its insufficient research and development. In order to facilitating the comprehensive utilization of peony pods, this work have obtained the optimum technical situations for ultrasound-assisted extraction of total flavonoids from peony pods, along with characterized the potential performance of peony pods as antioxidants. Through the optimization of the ultrasound-assisted extraction parameters employing single factor experiments and response surface analysis, the highest yield of total flavonoids (5.67% ± 0.04) was achieved at 81% ethanol concentration, solvent to material ratio of 47 mL/mg, 59 °C ultrasonic temperature, 44 min of ultrasonic time, and 300 W ultrasonic power. Antioxidant activity assessment of the total flavonoids extract from peony pods presented an excellent DPPH radical scavenging ability (73.8% highest) higher than ascorbic acid (VC), while the superoxide radical scavenging activity (56.9% highest) and hydroxyl radical scavenging activity (57.8% highest) were lower than those of VC. Particularly, after the extraction was treated with artificial gastric fluid and intestinal fluid respectively, the hydroxyl radical scavenging activity was both improved prominently (p < 0.05). The obtained results not only suggested that total flavonoids of peony pods can be efficiently extracted through the assistance of ultrasound, but also provided information on its antioxidant activity and bioaccessibility in vitro. Moreover, this study also offered an alternative method for facilitating the comprehensive utilization of peony pods, which have broad prospects due to the rich bioactive substances.