In information security, random number quality is closely related to cryptographic system security; moreover, random number quality depends on the corresponding entropy source quality. Therefore, evaluating the entropy source quality is extremely important. For existing evaluation methods, the ability of statistical-based entropy estimators to extract and learn data information is weak, resulting in lower entropy evaluation accuracies for some complex entropy sources. The prediction-based (especially neural network-based) entropy estimators with machine learning techniques have strong data-fitting and feature-extracting capabilities and can more accurately estimate the entropy values of complex entropy sources. However, owing to the relatively simple architecture of 1D neural networks, the 1D neural networks used by these estimators frequently reach bottlenecks, seriously limiting the further improvement in entropy estimation accuracy. Considering the above issues, this paper innovatively proposes an entropy estimation method based on a 2D-CNN and a multi-head self-attention mechanism. First, we built the MA-DG Net model. This model converts 1D random number sequences into 2D images via the GAF and DFT methods and then uses a 2D-CNN to extract and learn feature information from 2D images while retaining the original 1D sequential feature information via a multi-head self-attention mechanism. Next, we train the model to find its optimal parameters. Finally, we test the evaluation effect of the model using simulated datasets with known min-entropy and a real-world dataset with unknown min-entropy. The results show that compared with the entropy estimators in the experiment, our model achieves the lowest average relative error in entropy estimation on the simulated dataset of only 1.03%. In the real-world dataset, our model achieves the lowest entropy estimation value, which is an average of 0.88 lower than that of the other entropy estimators in the experiment.

中文翻译：

---

MA-DG：通过 2D-CNN 和多头自注意力学习不同维度序列的特征以进行最小熵评估


在信息安全中，随机数质量与密码系统安全密切相关；此外，随机数质量取决于相应的熵源质量。因此，评估熵源质量极其重要。现有的评估方法中，基于统计的熵估计器提取和学习数据信息的能力较弱，导致对一些复杂熵源的熵评估精度较低。采用机器学习技术的基于预测（尤其是基于神经网络）的熵估计器具有很强的数据拟合和特征提取能力，可以更准确地估计复杂熵源的熵值。然而，由于一维神经网络的结构相对简单，这些估计器使用的一维神经网络经常遇到瓶颈，严重限制了熵估计精度的进一步提高。考虑到上述问题，本文创新性地提出了一种基于2D-CNN和多头自注意力机制的熵估计方法。首先，我们构建了 MA-DG Net 模型。该模型通过GAF和DFT方法将1D随机数序列转换为2D图像，然后使用2D-CNN从2D图像中提取和学习特征信息，同时通过多头自注意力机制保留原始1D序列特征信息。接下来，我们训练模型以找到其最佳参数。最后，我们使用最小熵已知的模拟数据集和最小熵未知的真实数据集来测试模型的评估效果。 结果表明，与实验中的熵估计器相比，我们的模型在模拟数据集上实现了最低的熵估计平均相对误差仅为1.03%。在现实数据集中，我们的模型实现了最低的熵估计值，比实验中其他熵估计器平均低 0.88。