The reinforcement of polymer composites using plant fibres has received vast and growing attention for investigation owing to their capability to serve as semi-structural materials, especially as a substitute for fiberglass. However, in this type of research, the structural complexity, due to geometrical mismatches and inherent variability to the properties of vegetable fibres and plant fibre composites would benefit from more effective non-invasive inspection techniques. In this work, a novel infrared-terahertz fused approach was proposed, and a new unsupervised deep residual fusion algorithm was developed in order to combine the outputs from two different physical mechanisms. Specifically, infrared imaging provided the details of the defects and the morphological information from the surface and subsurface, while terahertz imaging recorded the distribution of the fibres and the resin. Finally, the fused infrared-terahertz slices were reconstructed into a form of 3D exhibition for inspection and evaluation of two plant fibre-reinforced polymer composites in first time using multiphysical imaging and fusion.