In the rapidly changing cybersecurity landscape, threat hunting has become a critical proactive defense against sophisticated cyber threats. While traditional security measures are essential, their reactive nature often falls short in countering malicious actors’ increasingly advanced tactics. This paper explores the crucial role of threat hunting, a systematic, analyst-driven process aimed at uncovering hidden threats lurking within an organization’s digital infrastructure before they escalate into major incidents. Despite its importance, the cybersecurity community grapples with several challenges, including the lack of standardized methodologies, the need for specialized expertise, and the integration of cutting-edge technologies like artificial intelligence (AI) for predictive threat identification. To tackle these challenges, this survey paper offers a comprehensive overview of current threat hunting practices, emphasizing the integration of AI-driven models for proactive threat prediction. Our research explores critical questions regarding the effectiveness of various threat hunting processes and the incorporation of advanced techniques such as augmented methodologies and machine learning. Our approach involves a systematic review of existing practices, including frameworks from industry leaders like IBM and CrowdStrike. We also explore resources for intelligence ontologies and automation tools. The background section clarifies the distinction between threat hunting and anomaly detection, emphasizing systematic processes crucial for effective threat hunting. We formulate hypotheses based on hidden states and observations, examine the interplay between anomaly detection and threat hunting, and introduce iterative detection methodologies and playbooks for enhanced threat detection. Our review encompasses supervised and unsupervised machine learning approaches, reasoning techniques, graph-based and rule-based methods, as well as other innovative strategies. We identify key challenges in the field, including the scarcity of labeled data, imbalanced datasets, the need for integrating multiple data sources, the rapid evolution of adversarial techniques, and the limited availability of human expertise and data intelligence. The discussion highlights the transformative impact of artificial intelligence on both threat hunting and cybercrime, reinforcing the importance of robust hypothesis development. This paper contributes a detailed analysis of the current state and future directions of threat hunting, offering actionable insights for researchers and practitioners to enhance threat detection and mitigation strategies in the ever-evolving cybersecurity landscape.

中文翻译：

---

不断发展的网络威胁搜寻技术：系统评价


在瞬息万变的网络安全环境中，威胁追踪已成为抵御复杂网络威胁的关键主动防御措施。虽然传统的安全措施是必不可少的，但它们的反应性往往无法对抗恶意行为者日益先进的策略。本白皮书探讨了威胁搜寻的关键作用，威胁搜寻是一个由分析师驱动的系统性流程，旨在发现潜伏在组织数字基础设施中的隐藏威胁，防止其升级为重大事件。尽管网络安全很重要，但网络安全社区仍面临着多项挑战，包括缺乏标准化方法、需要专业知识以及集成人工智能 （AI） 等尖端技术以进行预测性威胁识别。为了应对这些挑战，本调查报告全面概述了当前的威胁搜寻实践，强调集成 AI 驱动的模型以进行主动威胁预测。我们的研究探讨了有关各种威胁搜寻过程的有效性以及增强方法和机器学习等先进技术的整合的关键问题。我们的方法涉及对现有实践进行系统审查，包括来自 IBM 和 CrowdStrike 等行业领导者的框架。我们还探索了智能本体和自动化工具的资源。背景部分阐明了威胁搜寻和异常检测之间的区别，强调了对有效威胁搜寻至关重要的系统流程。 我们根据隐藏状态和观察结果提出假设，检查异常检测和威胁搜寻之间的相互作用，并引入迭代检测方法和手册以增强威胁检测。我们的审查包括有监督和无监督的机器学习方法、推理技术、基于图和基于规则的方法，以及其他创新策略。我们确定了该领域的主要挑战，包括标记数据的稀缺性、数据集的不平衡、集成多个数据源的需求、对抗性技术的快速发展以及人类专业知识和数据智能的可用性有限。该讨论强调了人工智能对威胁搜寻和网络犯罪的变革性影响，强调了稳健假设发展的重要性。本文详细分析了威胁搜寻的现状和未来方向，为研究人员和从业者提供了可操作的见解，以在不断发展的网络安全环境中增强威胁检测和缓解策略。