Abstract
Many studies have explored using different indicators to support students’ self-monitoring. This has motivated the development of student-facing analytics, such as dashboards and chatbots. However, there is a limited understanding of how learners interpret these indicators and act on that information. This study evaluates different indicators from a student perspective by adapting the card sorting technique, which is employed in Human-Centered Design. We chose eight indicators based on different comparative reference frames from the literature to create 16 cards to present both a visual and a text representation per indicator. Qualitative and quantitative data were collected from 21 students of three majors at two Latin American universities. According to the quantitative results, students’ agreement level about the indicators’ interpretability and actionability was relatively low. Nonetheless, the indicators that included temporality were found to be less interpretable but more actionable than those that did not. The analysis indicates that several students would use this information to improve their study habits only if their performance in the course is lower than expected. These findings might be used as a starting point to design student-facing analytics. Also, adapting the card sorting technique could be replicated to understand learners’ use of indicators in other TEL contexts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Matcha, W., Ahmad Uzir, N., Gasevic, D., Pardo, A.: A systematic review of empirical studies on learning analytics dashboards: a self-regulated learning perspective. IEEE Trans. Learn. Technol. 1 (2019). https://doi.org/10.1109/TLT.2019.2916802
Jivet, I., Scheffel, M., Schmitz, M., Robbers, S., Specht, M., Drachsler, H.: From students with love: an empirical study on learner goals, self-regulated learning and sense-making of learning analytics in higher education. Internet High. Educ. 47, 100758 (2020). https://doi.org/10.1016/j.iheduc.2020.100758
Jørnø, R.L., Gynther, K.: What constitutes an ‘Actionable Insight’ in learning analytics? J. Learn. Anal. 5 (2018). https://doi.org/10.18608/jla.2018.53.13
van Leeuwen, A., Teasley, S.D., Wise, A.F.: Teacher and student facing learning analytics. In: Handbook of Learning Analytics, p. 11 (2022)
Pérez-Álvarez, R., Jivet, I., Perez-Sanagustin, M., Scheffel, M., Verbert, K.: Tools designed to support self-regulated learning in online learning environments: a systematic review. IEEE Trans. Learn. Technol. 15, 508–522 (2022). https://doi.org/10.1109/TLT.2022.3193271
Schwendimann, B.A., et al.: Perceiving learning at a glance: a systematic literature review of learning dashboard research. IEEE Trans. Learn. Technol. 10, 30–41 (2017). https://doi.org/10.1109/TLT.2016.2599522
Vytasek, J.M., Patzak, A., Winne, P.H.: Analytics for student engagement. In: Virvou, M., Alepis, E., Tsihrintzis, G.A., Jain, L.C. (eds.) Machine Learning Paradigms: Advances in Learning Analytics, pp. 23–48. Springer International Publishing, Cham (2020). https://doi.org/10.1007/978-3-030-13743-4_3
Wise, A.F.: Designing pedagogical interventions to support student use of learning analytics. In: Proceedings of the Fourth International Conference on Learning Analytics And Knowledge, pp. 203–211. ACM, Indianapolis Indiana USA (2014). https://doi.org/10.1145/2567574.2567588
Lim, L., Dawson, S., Joksimovic, S., Gašević, D.: Exploring students’ sensemaking of learning analytics dashboards: does frame of reference make a difference? In: Proceedings of the 9th International Conference on Learning Analytics & Knowledge, pp. 250–259. ACM, Tempe AZ USA (2019). https://doi.org/10.1145/3303772.3303804
Jivet, I., Scheffel, M., Drachsler, H., Specht, M.: Awareness is not enough: pitfalls of learning analytics dashboards in the educational practice. In: Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M. (eds.) Data Driven Approaches in Digital Education, pp. 82–96. Springer International Publishing, Cham (2017). https://doi.org/10.1007/978-3-319-66610-5_7
Molenaar, I., Wise, A.F.: Temporal Aspects of Learning Analytics - Grounding Analyses in Concepts of Time. In: Handbook of Learning Analytics, p. 11 (2022)
Mendez, G., Galárraga, L., Chiluiza, K.: Showing academic performance predictions during term planning: effects on students’ decisions, behaviors, and preferences. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–17. ACM, Yokohama Japan (2021). https://doi.org/10.1145/3411764.3445718
Creswell, J., Guetterman, T.: Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, 6th Edn (2018)
Chatti, M.A., et al.: How to design effective learning analytics indicators? a human-centered design approach. In: Alario-Hoyos, C., Rodríguez-Triana, M.J., Scheffel, M., Arnedillo-Sánchez, I., Dennerlein, S.M. (eds.) Addressing Global Challenges and Quality Education: 15th European Conference on Technology Enhanced Learning, EC-TEL 2020, Heidelberg, Germany, September 14–18, 2020, Proceedings, pp. 303–317. Springer International Publishing, Cham (2020). https://doi.org/10.1007/978-3-030-57717-9_22
Dimitriadis, Y., Martínez-Maldonado, R., Wiley, K.: Human-centered design principles for actionable learning analytics. In: Tsiatsos, T., Demetriadis, S., Mikropoulos, A., Dagdilelis, V. (eds.) Research on E-Learning and ICT in Education: Technological, Pedagogical and Instructional Perspectives, pp. 277–296. Springer International Publishing, Cham (2021). https://doi.org/10.1007/978-3-030-64363-8_15
Spencer, D.: Card Sorting: Designing Usable Categories. Rosenfeld Media, Brooklyn, New York (2009)
Guarte, J.M., Barrios, E.B.: Estimation under purposive sampling. Commun. Stat. Simul. Comput. 35, 277–284 (2006). https://doi.org/10.1080/03610910600591610
Purposive Sample. In: Encyclopedia of Survey Research Methods. Sage Publications, Inc., 2455 Teller Road, Thousand Oaks California 91320 United States of America (2008). https://doi.org/10.4135/9781412963947.n419
Nowell, L.S., Norris, J.M., White, D.E., Moules, N.J.: Thematic analysis: striving to meet the trustworthiness criteria. Int. J. Qual. Methods. 16, 160940691773384 (2017). https://doi.org/10.1177/1609406917733847
Kendall, M.G.: The treatment of ties in ranking problems. Biometrika 33, 239–251 (1945). https://doi.org/10.2307/2332303
Zimmerman, B.J.: Attaining self-regulation. In: Handbook of Self-Regulation, pp. 13–39. Elsevier (2000). https://doi.org/10.1016/B978-012109890-2/50031-7
Boekaerts, M.: Self-regulated learning at the junction of cognition and motivation. Eur. Psychol. 1, 100–112 (1996). https://doi.org/10.1027/1016-9040.1.2.100
Teasley, S.D.: Student facing dashboards: one size fits all? Technol. Knowl. Learn. 22(3), 377–384 (2017). https://doi.org/10.1007/s10758-017-9314-3
Jivet, I., Wong, J., Scheffel, M., Valle Torre, M., Specht, M., Drachsler, H.: Quantum of choice: how learners’ feedback monitoring decisions, goals and self-regulated learning skills are related. In: LAK21: 11th International Learning Analytics and Knowledge Conference, pp. 416–427. ACM, Irvine CA USA (2021). https://doi.org/10.1145/3448139.3448179
Acknowledgments
This paper has been funded by the ANR LASER (156322) and the Chilean National Agency for Research and Development (ANID), under the ‘Student Experience in Higher Education in Chile: Expectations and Realities’ Millennium Nucleus. The authors acknowledge PROF-XXI, which is an Erasmus+ Capacity Building in the Field of Higher Education project funded by the European Commission (609767-EPP-1-2019-1- ES-EPPKA2-CBHE-JP). This publication reflects the views only of the authors and funders cannot be held responsible for any use which may be made of the information contained therein.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Villalobos, E., Hilliger, I., Pérez-Sanagustín, M., González, C., Celis, S., Broisin, J. (2023). Analyzing Learners’ Perception of Indicators in Student-Facing Analytics: A Card Sorting Approach. In: Viberg, O., Jivet, I., Muñoz-Merino, P., Perifanou, M., Papathoma, T. (eds) Responsive and Sustainable Educational Futures. EC-TEL 2023. Lecture Notes in Computer Science, vol 14200. Springer, Cham. https://doi.org/10.1007/978-3-031-42682-7_29
Download citation
DOI: https://doi.org/10.1007/978-3-031-42682-7_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42681-0
Online ISBN: 978-3-031-42682-7
eBook Packages: Computer ScienceComputer Science (R0)