A new magnesium-calcium oxysulfate cement (MCOSC) was proposed by replacing MgSO4 in magnesium oxysulfate cement (MOSC) with untreated flue gas desulfurization gypsum (FGDG) at levels of 25%, 50%, 75% and 100%. The synergistic effect of FGDG and chemical additives, including citric acid, ammonium citrate tribasic and ammonium dihydrogen phosphate on the mechanical performance and hydration mechanism of MCOSC was investigated. The results showed that three additives all promoted the formation of 5 Mg(OH)2·MgSO4·7H2O (5·1·7) phase and hindered Mg(OH)2 precipitation, improving mechanical properties of MCOSC. FGDG incorporation induced no new crystalline phase, but gypsum crystal provided the space for nucleation of 5·1·7 phase. FGDG also reacted with MOSC to form amorphous phase, which was identified as a magnesium-sulfide-calcium-hydrate gel. Moreover, it affected the morphology of 5·1·7 phase and lowered the temperature of endothermic peaks in the paste during the heating process. The additive-incorporated specimens with 25% FGDG replacement presented a superior compressive strength, water resistance and volume stability. This confirmed the effective application of FGDG in preparing MCOSC, and it also provided an approach for recycling waste gypsum.