Thanks to the strong representation capability of the pre-trained language models, supervised grammatical error correction has achieved promising performance. However, traditional model training depends significantly on the large scale of similar distributed samples. The model performance decreases sharply once the distributions of training and testing data are inconsistent. To address this issue, we propose an automatic sampling approach to effectively select high-quality samples from different corpora and filter out irrelevant or harmful ones. Concretely, we first provide a detailed analysis of error type and sentence length distributions on all datasets. Second, our corpus weighting approach is exploited to yield different weights for each sample automatically based on analysis results, thus emphasizing beneficial samples and ignoring the noisy ones. Finally, we enhance typical Seq2Seq and Seq2Edit grammatical error correction models with pre-trained language models and design a model ensemble algorithm for integrating the advantages of heterogeneous models and weighted samples. Experiments on the benchmark datasets demonstrate that the proper utilization of different corpora is extremely helpful in enhancing the accuracy of grammatical error correction. The detailed analysis gains more insights into the effect of different corpus weighting strategies.

由于预训练语言模型具有强大的表示能力，监督语法纠错取得了可喜的性能。然而，传统的模型训练在很大程度上依赖于大规模的相似分布式样本。一旦训练和测试数据的分布不一致，模型性能就会急剧下降。为了解决这个问题，我们提出了一种自动抽样方法，可以有效地从不同的语料库中选择高质量的样本，并过滤掉不相关或有害的样本。具体来说，我们首先提供了对所有数据集上的错误类型和句子长度分布的详细分析。其次，利用我们的语料库加权方法，根据分析结果自动为每个样本产生不同的权重，从而强调有益的样本，忽略有噪声的样本。最后，我们用预先训练的语言模型增强了典型的 Seq2Seq 和 Seq2Edit 语法纠错模型，并设计了一种模型集成算法，以整合异构模型和加权样本的优势。在基准数据集上的实验表明，正确利用不同的语料库对提高语法纠错的准确性非常有帮助。详细分析可以更深入地了解不同语料库加权策略的效果。