Home → Releases → № 4 (74) →

DEVELOPMENT OF AN EXPERIMENT TO IDENTIFY THE FACTORS THAT DETERMINE THE USE OF GENERATIVE AI IN WRITING TEXTS

Artificial Intelligence in Education , UDC: 004.8 DOI: 10.24412/2072-9014-2025-474-18-33

Authors

  • Biryukova Anastasia D.
  • Vafina Rinata R.
  • Nikitina Sofya S.
  • Fayzullin Rinat V. Candidate of Economic Sciences

Annotation

The article presents the results of an empirical study on the impact of generative artificial intelligence (GII) on the essay writing process by students. The purpose of the work is to identify how the use of GII, in comparison with traditional search and independent work, affects the time spent and the subjective perception of the value of one’s own work. The study is based on an experiment involving 97 students of 1–3 courses of RANEPA, who wrote three essays each: completely independently, using traditional search and with the help of GII. The results of the study are of practical importance for developing strategies for assessing academic integrity and adapting educational practices to new technological conditions.
Tags:
artificial intelligence; generative artificial intelligence; education; experiment; student essays; analytics

How to link insert

Biryukova, A. D., Vafina, R. R., Nikitina, S. S. & Fayzullin, R. V. (2025). DEVELOPMENT OF AN EXPERIMENT TO IDENTIFY THE FACTORS THAT DETERMINE THE USE OF GENERATIVE AI IN WRITING TEXTS Bulletin of the Moscow City Pedagogical University. Series "Pedagogy and Psychology", № 4 (74), 18. https://doi.org/10.24412/2072-9014-2025-474-18-33
References
1. 1. Pospelova E. A. Generative Artificial Intelligence in Education: Analysis of Trends and Prospects / E. A. Pospelova, P. L. Ototsky, E. N. Gorlacheva [et al.] // Professional Education and the Labor Market. 2024. Vol. 12. No. 3 (58). P. 6–21.
2. 2. Nikitina S. S. Areas of Application of Artificial Intelligence in Education / S. S. Nikitina, R. V. Fayzullin // Modern Development Trends and Prospects for the Implementation of Innovative Technologies in Mechanical Engineering, Education and Economics. 2025. Vol. 1. No. 1. P. 231–234.
3. 3. Vavilova D. D. Assessing the scaling potential of artificial intelligence tools in higher education: Russian and foreign experience / D. D. Vavilova, E. V. Kasatkina, R. V. Fayzullin // Education and science. 2025. Vol. 27. No. 9. P. 128–157. https://doi.org/10.17853/1994-5639-2025-8-128-157
4. 4. Biryukov D. N. International experience of regulation and implementation of generative artificial intelligence in the field of higher education / D. N. Biryukov, A. F. Suprun, A. D. Biryukova // Problems of information security. Computer systems. 2025. No. 3. P. 192–212. https://doi.org/10.48612/jisp/hz8x-hdxv-b83d
5. 5. Kasatkina E. Assessment of the Disciplinary Landscape of Training AI Specialists Based on a Comparative Analysis of the Curricula of Russian Universities / E. Kasatkina, D. Vavilova, R. Faizullin // 2025 5th International Conference on Technology Enhanced Learning in Higher Education (TELE). Lipetsk, 2025. P. 314–319.
6. 6. Brandl R. Survey: ChatGPT and AI Content — Can people tell the difference / R. Brandl, C. Ellis. 2023. URL: https://www.tooltester.com/en/blog/chatgpt-survey-canpeople-tell-the-difference/ (accessed: 18.10.2025).
7. 7. Clark E. All that’s “human” is not gold: Evaluating human evaluation of generated text / E. Clark, T. August, S. Serrano [et al.] // arXiv: Computer Science – Computation and Language. 2021. URL: https://arxiv.org/abs/2107.00061 (accessed: 18.10.2025).
8. 8. Dugan L. Real or fake text?: Investigating human ability to detect boundaries between human-written and machine-generated text / L. Dugan, D. Ippolito, A. Kirubarajan [et al.] // Proceedings of the AAAI Conference on Artificial Intelligence. 2023. Vol. 37. No. 11. P. 12763–12771. https://doi.org/10.48550/arXiv.2212.12672
9. 9. Uchendu A. Understanding individual and team-based human factors in detecting deepfake texts / A. Uchendu, J. Yо. Lee, H. Shen [et al.] // ACM. Woodstock (NY): ACM, 2018. Р. 1–26.
10. 10. Gehrmann S. GLTR: Statistical detection and visualization of generated text / S. Gehrmann, H. Strobelt, A. M. Rush // Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Florence: ACL, 2019. Р. 111–116.
11. 11. Pan H. Exploring Human-AI Collaboration in Educational Contexts: Insights from Writing Analytics and Authorship Attribution / H. Pan, E. Araujo Oliveira, R. Ferreira // Proceedings of the 15th International Conference on Learning Analytics and Knowledge Lak. Chicago: ACM, 2025. P. 903–909.
12. 12. Przystalski K. Stylometry recognizes human and LLM-generated texts in short samples / K. Przystalski, J. K. Argasiński, I. Grabska-Gradzińska [et al.] // Expert Systems with Applications. 2025. Vol. 296. P. 129001.
13. 13. Hakam H. Who Wrote It? Human-Written vs AI-Generated Texts in Orthopaedics, a Comparative Qualitative Analysis / H. Hakam, R. Prill, J. Lettner [et al.] // JMIR Formative Research. 2024. Vol. 8. P. e52164.
14. 14. Lee H. P. The impact of generative AI on critical thinking: Self-reported reductions in cognitive effort and confidence effects from a survey of knowledge workers / H. P. Lee, A. Sarkar, L. Tankelevitch [et al.] // Proceedings of the 2025 CHI conference on human factors in computing systems. NY: ACM, 2025. P. 1–22.
15. 15. Seßler K. Can AI grade your essays? A comparative analysis of large language models and teacher ratings in multidimensional essay scoring / K. Seßler, M. Fürstenberg, B. Bühler [et al.] // Proceedings of the 15th International Learning Analytics and Knowledge Conference. 2025. P. 462–472.
16. 16. Kosmyna N. Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant for essay writing task / N. Kosmyna, E. Hauptmann, Y. T. Yuan [et al.] // arXiv: Computer Science – Artificial Intelligence. 2025. URL: https://arxiv.org/abs/2506.08872 (accessed: 18.10.2025).
Download file .pdf 400.2 kb