To investigate the effects of heterogeneous microstructure, precipitates, and pore defects on dynamic impact toughness of Al-Cu alloy fabricated by arc-directed energy deposition (Arc-DED). Samples with diverse microstructures were prepared by adjusting the forming parameters, namely air cooling (AC), water cooling (WC), and the post-treatment, hot isostatic pressing (HIP), and heat treatment (T6) samples. The impact fracture behavior of samples with Charpy V-notched was studied by an instrumented drop hammer impact testing machine which can provide a load-displacement curve. The important load parameters and the absorbed energy at different stages can be determined by the load-displacement curve. The results show that the impact absorbed energy of the AC/WC/HIP/T6 sample was 1.33 ± 0.20, 1.96 ± 0.26, 1.91 ± 0.18, and 2.54 ± 0.21 J, respectively. The impact fracture mechanism was discussed as the corresponding fractographic observations. Brittle fractures with inhomogeneous dimples were observed in all the samples. The key determinant of the impact toughness was the block-shaped brittle θ phase that precipitated along the grain boundaries and pore defects. The equiaxed/columnar heterogeneous microstructure affects not only the coordinated deformation ability of grains but also the distribution of brittle θ phases. Therefore, compared with the AC sample, the impact toughness of the WC sample with a low-volume fraction θ phase, the HIP sample with low porosity, and the T6 sample with a large amount of ductile θ″/θ' phases was significantly improved. This study provides valuable insights for improving the dynamic impact resistance of Arc-DED fabricated Al-Cu alloy.