Eucommia ulmoides gum (EUG) can be used as an underwater sound-absorbing material, but the effect of the phase structure of its composites on the sound absorption performance has not been fully investigated. EUG was blended with chloroprene rubber (CR) and solution-polymerized styrene butadiene rubber (SSBR), and then epoxidized natural rubber (ENR) was added to the EUG/CR composites. The results showed that when CR was supplemented with 20% EUG, the composite showed the sea-island structure, EUG existed mainly in β-crystal form, and the average sound absorption coefficient of the composite was 0.4 at 0 MPa pressure, which was twice as high as that of pure CR. However, the sound absorption performance of the EUG/SSBR composite obtained by the same method of EUG/CR composites was reduced. In addition, the average sound absorption coefficient improved to 0.62 with the addition of ENR based on EUG/CR composites, and the compression resistance was greatly improved. The key factors for EUG to improve the underwater sound absorption performance of rubber materials lie in the generation of well-dispersed sea-island structures while retaining its own crystals. This work will provide some reference for the preparation of EUG composite sound-absorbing materials that are resistant to high water pressure.