With the high demand and sub-nanometer design for integrated circuits, surface defect complexity and frequency for semiconductor wafers have increased; subsequently emphasizing the need for highly accurate fault detection and root-cause analysis systems as manual defect diagnosis is more time-intensive, and expensive. As such, machine learning and deep learning methods have been integrated to automated inspection systems for wafer map defect recognition and classification to enhance performance, overall yield, and cost-efficiency. Concurrent with algorithm and hardware advances, in particular the onset of neural networks like the convolutional neural network, the literature for wafer map defect detection exploded with new developments to address the limitations of data preprocessing, feature representation and extraction, and model learning strategies. This article aims to provide a comprehensive review on the advancement of machine learning and deep learning applications for wafer map defect recognition and classification. The defect recognition and classification methods are introduced and analyzed for discussion on their respective advantages, limitations, and scalability. The future challenges and trends of wafer map detection research are also presented.