Hostname: page-component-848d4c4894-nr4z6 Total loading time: 0 Render date: 2024-05-16T07:38:03.523Z Has data issue: false hasContentIssue false
  • English
  • Français

Sense-aware connective-based indices of cohesion and their relationship to cohesion ratings of English language learners’ written production

Published online by Cambridge University Press:  21 March 2024

Xiaofei Lu Open the ORCID record for Xiaofei Lu [Opens in a new window]  and
Renfen Hu Open the ORCID record for Renfen Hu [Opens in a new window]
Xiaofei Lu
Affiliation:
The Pennsylvania State University
Renfen Hu*
Affiliation:
Beijing Normal University
*
Corresponding author: Renfen Hu; Email: irishu@mail.bnu.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

The use of connectives has been considered important for assessing the cohesion of written texts (Crossley et al., 2019). However, existing connective-based indices have not systematically addressed two issues of ambiguity, namely, that between discourse and non-discourse use of polysemous word forms and that in terms of the specific discourse relations marked by polysemous discourse connectives (Pitler & Nenkova, 2009). This study proposes 34 sense-aware connective-based indices of cohesion that account for these issues and assesses their predictive power for cohesion ratings in comparison to 25 existing indices. Results from the analysis of 3,911 argumentative essays from the English Language Learner Insight, Proficiency and Skills Evaluation Corpus show that 23 sense-aware indices but only three existing indices correlated significantly and meaningfully with cohesion ratings. The sense-aware indices also exhibited greater predictive power for cohesion ratings than existing indices. The implications of our findings for future cohesion research are discussed.

Type
Research Article
Information
Studies in Second Language Acquisition , First View , pp. 1 - 19
Check for updates
Open Practices
Open materials Open data
Copyright
© The Author(s), 2024. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Biler, A. (2018). The role of cohesion in second language reading comprehension (Unpublished doctoral dissertation). University of South Carolina, Columbia, SC.Google Scholar
Bilki, Z. (2014). A close observation of second language (L2) readers and texts: meaning representation and construction through cohesion (Unpublished doctoral dissertation). University of Iowa, Iowa City, IA.CrossRefGoogle Scholar
Bayraktar, H. (2011). The role of lexical cohesion in L2 reading comprehension: Awareness of lexical cohesive links and L2 reading test performance. VDM Verlag Dr. Müller.Google Scholar
Chen, D., & Manning, C. D. (2014). A fast and accurate dependency parser using neural networks. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar. https://doi.org/10.3115/v1/d14-1082CrossRefGoogle Scholar
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.Google Scholar
Crossley, S. A., & McNamara, D. S. (2011). Text coherence and judgments of essay quality: Models of quality and coherence. In Carlson, L., Hoelscher, C., & Shipley, T. F. (Eds.), Proceedings of the 29th Annual Conference of the Cognitive Science Society (pp. 1236–1241). Cognitive Science Society.Google Scholar
Crossley, S. A., Kyle, K., & Dascalu, M. (2019). The tool for the automatic analysis of Cohesion 2.0: Integrating semantic similarity and text overlap. Behavior Research Methods, 51, 14–27. https://doi.org/10.3758/s13428-018-1142-4CrossRefGoogle ScholarPubMed
Crossley, S. A., Kyle, K., & McNamara, D. S. (2016a). The development and use of cohesive devices in L2 writing and their relations to judgments of essay quality. Journal of Second Language Writing, 32, 1–16. https://doi.org/10.1016/j.jslw.2016.01.003CrossRefGoogle Scholar
Crossley, S. A., Kyle, K., & McNamara, D. S. (2016b). The tool for the automatic analysis of text cohesion (TAACO): Automatic assessment of local, global, and text cohesion. Behavior Research Methods, 48, 1227–1237. https://doi.org/10.3758/s13428-015-0651-7CrossRefGoogle ScholarPubMed
Crossley, S. A., & McNamara, D. S. (2012). Predicting second language writing proficiency: The role of cohesion, readability, and lexical difficulty. Journal of Research in Reading, 35, 115–135. https://doi.org/10.1111/j.1467-9817.2010.01449.xCrossRefGoogle Scholar
Degand, L., & Sanders, T. (2002). The impact of relational markers on expository text comprehension both in L1 and L2. Reading and Writing, 15, 739–757. https://doi.org/10.1023/A:1020932715838CrossRefGoogle Scholar
Ferenci, T. (2017). Variable selection should be blinded to the outcome. International Journal of Epidemiology, 46, 1077–1079. https://doi.org/10.1093/ije/dyx048CrossRefGoogle ScholarPubMed
Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, 36, 193–202. https://doi.org/10.3758/bf03195564Google ScholarPubMed
Guo, L., Crossley, S. A., & McNamara, D. S. (2013). Predicting human judgments of essay quality in both integrated and independent second language writing samples: A comparison study. Assessing Writing, 18, 218–238. https://doi.org/10.1016/j.asw.2013.05.002CrossRefGoogle Scholar
Halliday, M., & Hasan, R. (1976). Cohesion in English. Longman. http://archives.umc.edu.dz/handle/123456789/111528Google Scholar
Jonz, J. (1987). Textual cohesion and second language comprehension. Language Learning, 37, 409–438. https://doi.org/10.1111/j.1467-1770.1987.tb00578.xCrossRefGoogle Scholar
Kim, M., & Crossley, S. A. (2018). Modeling second language writing quality: A structural equation investigation of lexical, syntactic, and cohesive features in source-based and independent writing. Assessing Writing, 37, 39–56. https://doi.org/10.1016/j.asw.2018.03.002CrossRefGoogle Scholar
Louwerse, M. M. (2001). From cohesion in text to coherence in comprehension (Doctoral dissertation). University of Edinburgh. https://era.ed.ac.uk/handle/1842/22424Google Scholar
Louwerse, M. M. (2002). An analytic and cognitive parameterization of coherence relations. Cognitive Linguistics, 12, 291–315. https://doi.org/10.1515/cogl.2002.005CrossRefGoogle Scholar
Lu, X. (2012). The relationship of lexical richness to the quality of ESL learners’ oral narratives. The Modern Language Journal, 96, 190–208. https://doi.org/10.1111/j.1540-4781.2011.01232_1.xCrossRefGoogle Scholar
Lu, X., & Hu, R. (2022). Sense-aware lexical sophistication indices and their relationship to second language writing quality. Behavior Research Methods, 54, 1444–1460. https://doi.org/10.3758/s13428-021-01675-6CrossRefGoogle ScholarPubMed
Manning, C. D., Surdeanu, M., Bauer, J. A., Finkel, J. R., Bethard, S., & McClosky, D. (2014). The Stanford CoreNLP Natural Language Processing Toolkit. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Baltimore, MD. https://doi.org/10.3115/v1/p14-5010Google Scholar
McNamara, D. S., Graesser, A. C., McCarthy, P., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge University Press.CrossRefGoogle Scholar
Pitler, E., & Nenkova, A. (2009). Using syntax to disambiguate explicit discourse connectives in text. Proceedings of the ACL-IJCNLP 2009 Conference Short Papers. Suntec, Singapore. https://doi.org/10.3115/1667583.1667589CrossRefGoogle Scholar
Prasad, R. B., Dinesh, N., Lee, A. T. K., Miltsakaki, E., Robaldo, L., Joshi, A. K., & Webber, B. (2008). The Penn Discourse TreeBank 2.0. Proceedings of the Sixth International Conference on Language Resources and Evaluation (pp. 2961–2968). https://people.cs.pitt.edu/~huynv/research/argument-mining/The%20Penn%20Discourse%20TreeBank%202.pdfGoogle Scholar
Smith, G. (2018). Step away from stepwise. Journal of Big Data, 5, Article number 32. https://doi.org/10.1186/s40537-018-0143-6CrossRefGoogle Scholar
Sun, G., Shook, T., & Kay, G. L. (1996). Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis. Journal of Clinical Epidemiology, 49, 907–916. https://doi.org/10.1016/0895-4356(96)00025-xCrossRefGoogle ScholarPubMed
Vanderbilt University, & The Learning Agency Lab. (2022). Kaggle Feedback Prize English Language Learning Competition [Dataset]. Retrieved from https://www.kaggle.com/competitions/feedback-prize-english-language-learning/dataGoogle Scholar
Zhang, X., Lu, X., & Li, W. (2022). Beyond differences: Assessing effects of shared linguistic features on L2 writing quality of two genres. Applied Linguistics, 43, 168–195. https://doi.org/10.1093/applin/amab007CrossRefGoogle Scholar
Supplementary material: File

Lu and Hu supplementary material

Lu and Hu supplementary material
Download Lu and Hu supplementary material(File)
File 716.9 KB