Empowering Teachers in LMOOC Design by Using a Taxonomy of Participants’ Temporal Patterns

Document Type : Research Article


Universidad Nacional de Educación a Distancia (UNED), Madrid, Spain


A decade of research into MOOCs (massive online open courses) for language learning (LMOOCs) shows that they seem to have consolidated their position as a subfield of computer-assisted language learning (CALL). Since the appearance of LMOOCs in 2013, 3 key systematic reviews have been carried out; these confirm that research into student profiles is a recurring trend, with the focus on avoiding dropout rates by creating personalized learning pathways. One of the challenges for teachers and LMOOC developers is that they are not cognizant of their students or their study habits. If we could learn how students organize their study in LMOOCs, a taxonomy could be established according to their profiles. This would enable teachers and LMOOC developers to improve their course design and so create personalized learning pathways, making the courses better suited to students’ specific learning preferences. In this study, we use techniques of learning analytics (LA) to explore the temporal patterns of LMOOC participants in order to understand the way they manage and invest their time during their online courses. As a result of this study, we propose a new taxonomy of LMOOC participant profiles based on temporal patterns—one which would provide teachers with a tool to support them when personalizing the design and development of LMOOCs and which would, therefore, help them adapt their courses to the specific learning preferences of each profile.


Ahmadi, N. (2015). Which learning style do you prefer to improve EFL learning? Journal of Research in Applied Linguistics, 5(Special Issue), 26-34.  https://doi.org/10.22055/rals.2015.11266
Alario-Hoyos, C., Pérez-Sanagustín, M., Delgado-Kloos, C., Parada, H. A., & MuñozOrganero, M. (2014). Delving into participants’ profiles and use of social tools in MOOC. IEEE Transactions on Learning Technologies, 7(3), 260-266. https://doi.org /10.1109/TLT.2014.2311807
Alexander, B., Ashford-Rowe, K., Barajas-Murphy, N., Dobbin, G., Knott, J., McCormack, M., Pomerantz, J., Seilhamer, R., & Weber, N. (2019). EDUCAUSE horizon report: 2019 higher education edition. EDUCAUSE.
Anderson, A., Huttenlocher, D., Kleinberg, J., & Leskovec, J. (2014). Engaging with massive online courses. In Proceedings of the 23rd International Conference on World Wide Web (pp. 687-698). ACM. https://doi.org/10.1145/2566486.2568042.
Arora, S., Goel, M., Sabitha, A. S., & Mehrotra, D. (2017). Learner groups in massive open online courses. American Journal of Distance Education, 31(2), 80-97. https://doi.org/10.1080/08923647.2017.1300461
Barcena, E., & Martín-Monje, E. (2014). Language MOOCs: Providing learning, transcending boundaries. In E. Martín-Monje & E. Barcena (Eds.), Language MOOCs: Providing learning, transcending boundaries (pp. 175-189). Walter de Gruyter GmbH.
Bialystok, E. (1978). A theoretical model of second language learning 1. Language learning, 28(1), 69-83.
Brown, M., McCormack, M., Reeves, J., Brooks, D. C., Grajek, S., Alexander, B., Bali, M., Bulger, S., Dark, S., Engelbert, N., Gannon, K., Gauthier, A., Gibson, D., Gibson, R., Lundin, B., Veletsianos, G., & Weber, N. (2020). EDUCAUSE horizon report, teaching and learning edition. EDUCAUSE.
Castrillo, M. D., & Sedano, B. (2021). Joining Forces toward social inclusion: Language MOOC design for refugees and migrants through the lens of maker culture. CALICO Journal, 38(1), 79-102. https://doi.org/10.1558/cj.40900
Chaker, R., & Bachelet, R. (2020). Internationalizing professional development: Using educational data mining to analyze learners’ performance and dropouts in a French MOOC. The International Review of Research in Open and Distributed Learning, 21(4), 199-221. https://doi.org/10.19173/irrodl.v21i4.4787
Chatti, M. A., Dyckhoff, A.L., Schroeder, U., & Thüs, H. (2012). A reference model for learning analytics. International Journal of Technology Enhanced Learning, 4(5 & 6), 318-331. https://doi.org/10.1504/IJTEL.2012.051815
Chong, S. W., Khan, M. A., & Reinders, H. (2022). A critical review of design features of LMOOCs. Computer Assisted Language Learning, 1-21. https://doi.org/10.1080/09588221.2022.2038632
Coleman, C. A., Seaton, D. T., & Chuang, I. (2015). Probabilistic use cases: Discovering behavioral patterns for predicting certification. In 2nd ACM Conference on Learning@ Scale (pp 141–148). ACM. 
Díez-Arcón, P., & Martín-Monje, E. (2023). The coming of age of LMOOC research. A systematic review (2019-21). Innovation in Language Learning and Teaching, 17(3), 535-551.
Dhorne, L., Deflandre, J., Bernaert, O., Bianchi, S., & Thirouard, M. (2017). Mentoring learners in MOOCs: A new way to improve completion rates? In Digital Education: Out to the World and Back to the Campus (Vol. 10254, Lecture Notes in Computer Science, pp. 29-37). Cham: Springer International Publishing.
Ferguson, R., & Clow, D. (2015). Examining engagement: Analyzing learner subpopulations in massive open online courses (MOOCs). Proceedings of the Fifth International Conference on Learning Analytics and Knowledge (pp. 51-58). Vancouver, Canada: ACM.
Garreta, M., Aguado, G., Mor, Y., Fernandez, C., & Riviou, K. (2015). A peer-mentoring approach for the continuous professional development of educators in a MOOC setting. In Proceedings of the European Distance and E-Learning Network 2015 Annual Conference (pp. 149-158). Barcelona, Spain.
Gillespie, J. (2020). CALL research: Where are we now? ReCALL, 32(2), 127-144. https://doi.org/10.1017/S0958344020000051
Halawa, S., Greene, D., & Mitchell, J. (2014). Dropout prediction in MOOCs using learner activity features. In U. Cress & C. Delgado Kloos (Eds.), Proceedings of the European MOOC stakeholder summit (pp. 58-65). P.A.U. Education.
Hill, P. (2013). Emerging student patterns in MOOCs: A (revised) graphical view. Retrieved from http://mfeldstein.com/emerging-student-patterns-in-moocs-a-revised-graphical-view
Ho, A. D., Chuang, I., Reich, J., Coleman, C. A., Whitehill, J., Northcutt, C. G., et al. (2015). Harvardx and mitx: Two years of open online courses fall 2012-summer 2014. https://doi.org/10.2139/ssrn.2586847
Hsu, L. (2023). What makes good LMOOCs for EFL learners? Learners’ personal characteristics and information system success model. Computer Assisted Language Learning, 36(1-2), 1-25.
Ifenthaler, D., Mah, D., & Yau, Y. J. (Eds.). (2019). Utilizing learning analytics to support study success. Springer.
Jitpaisarnwattana, N., Reinders, H., & Darasawang, P. (2021) Understanding the roles of personalization and social learning in a language MOOC through learning analytics. Online Learning, 25(4), 324-343. 
Jordan, K. (2015). Massive open online course completion rates revisited: Assessment, length, and attrition. The International Review of Research in Open and Distributed Learning, 16(3), 30-42. http://dx.doi.org/10.19173/irrodl.v16i3.2112
Khalil, M., & Ebner, M. (2015). Learning analytics: Principles and constraints. In S. Carliner, C. Fulford, & N. Ostashewski (Eds.), Proceedings of EdMedia 2015—World Conference on Educational Media and Technology (pp. 1789-1799). Montreal, Quebec, Canada: Association for the Advancement of Computing in Education (AACE).
Khalil, M., & Ebner, M. (2016). What is learning analytics about? A survey of different methods used in 2013-2015. In Proceedings of Smart Learning Conference (pp. 294-304). Dubai: HBMSU Publishing House.
Kim, P., & Chung, C. (2015). Creating a temporary spontaneous mini-ecosystem through a MOOC. In C. J. Bonk, M. M. Lee, T. C. Reeves, & T. H. Reynolds (Eds.), MOOCs and open education around the world (pp. 157-168). New York, NY: Routledge.
Kizilcec, R. F., Piech, C., & Schneider, E. (2013). Deconstructing disengagement: Analyzing learner subpopulations in massive open online courses. In 3rd International Conference on Learning Analytics and Knowledge (pp. 170-179). New York: ACM.
Koller, D., Ng, A., Do, C., & Chen, Z. (2013). Retention and intention in massive open online courses: In depth. Educause Review, 48(3), 62-63.
Krashen, S., & Scarcella, R. (1978). On routines and patterns in language acquisition and performance Language Learning, 28(2), 283-300.
Lang, C., Siemens, G., Wise, A., & Gaševic, D. (Eds.). (2017). Handbook of learning analytics. SOLAR.
Leys, C., Christophe Ley, C., Klein, O., Bernard, P., & Licata, L. (2013). Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median. Journal of Experimental Social Psychology, 49(4), 764-766. http://dx.doi.org/10.1016/j.jesp.2013.03.013
Li, Q, & Baker, R. (2018). The different relationships between engagement and outcomes across participant subgroups in Massive Open Online Courses. Computers & Education, 127, 41-65. https://doi.org/10.1016/j.compedu.2018.08.005
Li, S., Wang, S., Du, J., Pei, Y., & Shen, X. (2022). MOOC learners’ time—investment patterns and temporal—learning characteristics. Journal of Computer Assisted Learning, 38(1), 152-166. https://doi.org/10.1111/jcal.12597
Liu, B. (2006). Web data mining. Springer.
León Urrutia, M., Vázquez Cano, E., & López-Meneses, E. (2017). MOOC learning analytics using real-time dynamic metrics. @tic. Revista D'innovació Educativa, 18, 38-47.  https://doi.org/10.7203/attic.18.10022
Lester, J., Klein, C., Johri, A., & Rangwala, H. (Eds.) (2018). Learning analytics in higher education: Current innovations, future potential, and practical applications. Routledge.
Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCs: A systematic study of the published literature 2008-2012. International Review of Research in Open and Distributed Learning, 14(3), 202-227. https://doi.org/10.19173/irrodl.v14i3.1455
Martín-Monje, E. (in press). Language MOOCs as an emerging field of research: From theory to practice. In M. M. Suárez & W. El-Henawy (Eds.), Optimizing ONLINE English language learning and teaching (pp. 25-43). London: Springer.
Martín-Monje, E., Castrillo, M. D., & Mañana-Rodríguez, J. (2018). Understanding online interaction in language MOOCs through learning analytics. Computer Assisted Language Learning, 31(3), 251-272. https://doi.org/10.1080/09588221.2017.1378237
Maseleno, A., Sabani, N., Huda, M., Ahmad, R., Jasmi, K. A., & Basiron, B. (2018). Demystifying learning analytics in personalised learning. International Journal of Engineering & Technology, 7(3), 1124-1129. https://doi.org/10.14419/ijet.v7i3.9789
Martín-Monje, E., & Barcena, E. (Eds.). (2014). Language MOOCs: Providing learning, transcending boundaries. De Gruyter Open.
Martín-Monje, E., & Borthwick, K. (2021). Researching massive open online courses for language teaching and learning. ReCALL, 33(2), 107-110. https://doi.org/10.1017/S0958344021000094
Martín-Monje, E., Castrillo, M. D., & Mañana-Rodríguez, J. (2018). Understanding online interaction in language MOOCs through learning analytics. Computer Assisted Language Learning, 31(3), 251-272. https://doi.org/10.1080/09588221.2017.1378237
Maya-Jariego, I., Holgado, D., González-Tinoco, E., Castaño-Muñoz, J., & Punie, Y. (2020). Typology of motivation and learning intentions of users in MOOCs: The MOOCKNOWLEDGE study. Educational Technology Research and Development, 68(1), 203-224. https://doi.org/10.1007/s11423-019-09682-3
Mehnert, U. (1998). The effects of different lengths of time for planning on second language performance. Studies in Second Language Acquisition, 20, 83-108. https://doi.org/10.1017/S0272263198001041
Mullaney, T. (2014). Making sense of MOOCs: A reconceptualization of HarvardX courses and their students. Retrieved from https://ssrn.com/abstract=2463736
Nurieva, G. R., & Garaeva, L, M. (2020). Zoom-based distance learning of English as a foreign language. Journal of Research in Applied Linguistics, 11, 439-448. https://doi.org/10.22055/rals.2020.16344
Panagiotidis, P. (2019). MOOCs for language learning: Reality and prospects. Society for Information Technology & Teacher Education International Conference.
del Peral Pérez, J. J. (2019). Patrones temporales de participación en MOOC. Estudio de un MOOC de lenguas. RIED. Revista Iberoamericana de Educación a Distancia, 22(2), 287-303. http://dx.doi.org/10.5944/ried.22.2.23109
del Peral Pérez, J. J. (2022). El perfil del estudiante en los MOOC: identificación de patrones temporales de uso a partir del análisis de dos MOOC de lenguas. [Unpublished doctoral dissertation, Universidad Nacional de Educación a Distancia (UNED)].
Poellhuber, B., Roy, N., & Bouchoucha, I. (2019). Understanding participant’s behavior in massively open online courses. International Review of Research in Open and Distributed Learning, 20(1), 28-41. https://doi.org/10.19173/irrodl.v20i1.3709
Poy, Y., & Gonzales-Aguilar, A. (2014). Factores de éxito de los MOOC: algunas consideraciones críticas. Revista Ibérica de Sistemas y Tecnologías de Información, SPE1, 105-118. https://doi.org/10.4304/risti.e1.105-118
Read, T., & Sedano, B. (2021). The role of scaffolding in LMOOCs for displaced people. Lengua y Migración/Language and Migration, 13(2), 115-133.
Reich, J. (2014). MOOC completion and retention in the context of student intent. EDUCAUSE Review. Retrieved from http://www.educause.edu/ero/article/mooc-completion-and-retentioncontext-student-intent
Reich, J., & Ruipérez-Valiente, J. A. (2019). The MOOC pivot. Science 363, 130-13.  https://doi.org/10.1126/science.aav7958
Romero, C., Cerezo, R., Bogarín, A., & Sánchez-Santillán, M. (2016). Educational process mining: A tutorial and case study using Moodle data sets. In S. ElAtia, D. Ipperciel, & O. R. Zaïane (Eds.), Data mining and learning analytics (pp. 1-28). London: Wiley.
Romero, C., & Ventura, S. (2020) Educational data mining and learning analytics: An updated survey. WIREs Data Mining Knowl Discov, 10:e1355. Retrieved from https://onlinelibrary.wiley.com/doi/abs/10.1002/widm.1355
Sallam, M. H., Martín-Monje, E., & Li, Y. (2022). Research trends in language MOOC studies: A systematic review of the published literature (2012-2018). Computer Assisted Language Learning, 35(4), 764-791. https://doi.org/10.1080/09588221.2020.1744668
Sanz Gil, M. (2021). LMOOC para la integración de personas desplazadas. Anales de Filología Francesa, 29, 463-477. https://doi.org/10.6018/analesff.483181
Sclater, N. (2017). Learning analytics explained. Routledge.
Singh, D., & Singh, B. (2020). Investigating the impact of data normalization on classification performance. Applied Soft Computing, 97, 105524. https://doi.org/10.1016/j.asoc.2019.105524
Sunar, A., Abbasi, R., Davis, H., White, S., & Aljohani, N. (2020). Modelling MOOC learners' social behaviors. Computers in Human Behavior, 107, 50-62.  https://doi.org/10.1016/j.chb.2018.12.013
Tatman, R. (2018). Data cleaning challenge: Scale and normalize data. Kaggle. Retrieved from https://www.kaggle.com/rtatman/data-cleaning-challenge-scale-and- normalize-data
Tavakoli, P., & Foster, P. (2011). Task design and second language performance: The effect of narrative type on learner output. Language Learning, 61, 37-72. https://doi.org/10.1111/j.1467-9922.2011.00642.x
Tibshirani, R., Walther, G., & Hastie, T. (2001). Estimating the number of clusters in a data set via the gap statistic. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63(2), 411-423. https://doi.org/10.1111/1467-9868.00293
Xu, B., & Yang, D. (2016). Motivation classification and grade prediction for MOOCs learners. Computational Intelligence and Neuroscience, 4, 1-7. https://doi.org/10.1155/2016/2174613
Zeng, S., Zhang, J., Gao, M., Xu, K. M., & Zhang, J. (2020). Using learning analytics to understand collective attention in language MOOCs. Computer Assisted Language Learning, 35(7), 1594-1619. https://doi.org/10.1080/09588221.2020.1825094 
Zhang, Y., & Sun, R. (2023). LMOOC research 2014 to 2021: What have we done and where are we going next? ReCALL, 1-16. https://doi.org/10.1017/S0958344022000246