Toward Personalised Learning Experiences: Beyond Prompt Engineering
Authors: 
Joost Kruis, 
Maria Soledad Pera, Zoë ten Napel, Monica Landoni, Emiliana Murgia, Theo Huibers, Remco FeskensAuthors Info & Claims
Joost Kruis, Maria Soledad Pera, Zoë ten Napel, Monica Landoni, Emiliana Murgia, Theo Huibers, Remco FeskensAuthors Info & ClaimsPages 644 - 649
Abstract
We discuss the foundation of a collaborative effort to explore AI’s role in supporting (teachers and) children in their learning experiences. We integrate principles of educational psychology, AI, and HCI, and align with best practices in education while undertaking a human-centered focus on design and development that puts the student at the centre and keeps the expert-in-the-loop. Initially, we study assessment items—questions or tasks tied to a learning target. These items vary in complexity, serve as indicators of students’ grasp of specific concepts and spotlight areas where support may be needed. This preliminary analysis will help us outline a framework to guide the design and evaluation of AI technology for K-12 education. Such a framework would ensure that assessment item generation technology goes beyond the current one-dimensional approach by incorporating multifaceted, adaptable perspectives that consider the variegated landscape of learners’ needs, subject matter complexities, and pedagogical goals.
References
[1]
Ibrahim Adeshola and Adeola P. Adepoju. 2023. The opportunities and challenges of ChatGPT in education. Interactive Learning Environments (2023), 1–14. https://doi.org/10.1080/10494820.2023.2253858
[2]
Vincent Aleven, Richard Baraniuk, Emma Brunskill, Scott Crossley, Dora Demszky, Stephen Fancsali, Shivang Gupta, Kenneth Koedinger, Chris Piech, Steve Ritter, Danielle R. Thomas, Simon Woodhead, and Wanli Xing. 2023. Towards the Future of AI-Augmented Human Tutoring in Math Learning. In Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky, Ning Wang, Genaro Rebolledo-Mendez, Vania Dimitrova, Noboru Matsuda, and Olga C. Santos (Eds.). Springer Nature Switzerland, Cham, 26–31. https://doi.org/10.1007/978-3-031-36336-8_3
[3]
Fx. Risang Baskara. 2023. Fostering Culturally Grounded Learning: Generative Ai, Digital Storytelling, And Early Childhood Education. International Conference of Early Childhood Education in Multiperspectives 2 (2023), 351–361.
[4]
Aras Bozkurt and Ramesh C. Sharma. 2023. Generative AI and prompt engineering: The art of whispering to let the genie out of the algorithmic world. Asian Journal of Distance Education 18, 2 (2023), i–vii. https://doi.org/10.5281/zenodo.8174941
[5]
Karin Bügel and Piet F. Sanders. 1998. Richtlijnen voor de ontwikkeling van onpartijdige toetsen. Cito Instituut voor Toetsontwikkeling, Arnhem, The Netherlands.
[6]
Dhawaleswar Rao CH and Sujan Kumar Saha. 2020. Automatic Multiple Choice Question Generation From Text: A Survey. IEEE Transactions on Learning Technologies 13, 1 (2020), 14–25. https://doi.org/10.1109/tlt.2018.2889100
[7]
Banghao Chen, Zhaofeng Zhang, Nicolas Langrené, and Shengxin Zhu. 2023. Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review. arxiv:2310.14735 [cs.CL]
[8]
Jennifer Chubb, Sondess Missaoui, Shauna Concannon, Liam Maloney, and James Alfred Walker. 2022. Interactive storytelling for children: A case-study of design and development considerations for ethical conversational AI. International Journal of Child-Computer Interaction 32, Article 100403 (2022), 14 pages. https://doi.org/10.1016/j.ijcci.2021.100403
[9]
Debby R.E. Cotton, Peter A. Cotton, and J. Reuben Shipway. 2024. Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International 61, 2 (2024), 228–239. https://doi.org/10.1080/14703297.2023.2190148
[10]
Eva de Schipper, Remco Feskens, and Jos Keuning. 2021. Personalized and Automated Feedback in Summative Assessment Using Recommender Systems. Frontiers in Education 6 (2021), 11 pages. https://doi.org/10.3389/feduc.2021.652070
[11]
Paul Denny, Viraj Kumar, and Nasser Giacaman. 2023. Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1. Association for Computing Machinery, New York, NY, USA, 1136–1142. https://doi.org/10.1145/3545945.3569823
[12]
Brody Downs, Aprajita Shukla, Mikey Krentz, Maria Soledad Pera, Katherine Landau Wright, Casey Kennington, and Jerry Fails. 2020. Guiding the selection of child spellchecker suggestions using audio and visual cues. In Proceedings of the Interaction Design and Children Conference. Association for Computing Machinery, New York, NY, USA, 398–408. https://doi.org/10.1145/3392063.3394390
[13]
Scott F. Gregory. 2024. Empowering Teaching With Prompt Engineering: How to Integrate Curriculum, Standards, and Assessment for a New Age. In Transforming Education With Generative AI: Prompt Engineering and Synthetic Content Creation, Ramesh C. Sharma and Aras Bozkurt (Eds.). IGI Global, Hershey, PA, USA, 239–260. https://doi.org/10.4018/979-8-3693-1351-0.ch012
[14]
Ariel Han and Zhenyao Cai. 2023. Design implications of generative AI systems for visual storytelling for young learners. In Proceedings of the 22nd Annual ACM Interaction Design and Children Conference. Association for Computing Machinery, New York, NY, USA, 470–474. https://doi.org/10.1145/3585088.3593867
[15]
Jussi S. Jauhiainen and Agustín Garagorry Guerra. 2023. Generative AI and ChatGPT in School Children’s Education: Evidence from a School Lesson. Sustainability 15, 18, Article 14025 (2023), 22 pages. https://doi.org/10.3390/su151814025
[16]
Nimrita Koul. 2023. Prompt Engineering for Large Language Models. Nimrita Koul.
[17]
Ghader Kurdi, Jared Leo, Bijan Parsia, Uli Sattler, and Salam Al-Emari. 2019. A Systematic Review of Automatic Question Generation for Educational Purposes. International Journal of Artificial Intelligence in Education 30, 1 (2019), 121–204. https://doi.org/10.1007/s40593-019-00186-y
[18]
Ehsan Latif and Xiaoming Zhai. 2024. Fine-tuning ChatGPT for automatic scoring. Computers and Education: Artificial Intelligence 6, Article 100210 (2024), 10 pages. https://doi.org/10.1016/j.caeai.2024.100210
[19]
Alwyn Vwen Yen Lee, Seng Chee Tan, and Chew Lee Teo. 2023. Designs and practices using generative AI for sustainable student discourse and knowledge creation. Smart Learning Environments 10, 59 (2023), 1–18. https://doi.org/10.1186/s40561-023-00279-1
[20]
Unggi Lee, Haewon Jung, Younghoon Jeon, Younghoon Sohn, Wonhee Hwang, Jewoong Moon, and Hyeoncheol Kim. 2023. Few-shot is enough: exploring ChatGPT prompt engineering method for automatic question generation in english education. Education and Information Technologies (2023), 1–33. https://doi.org/10.1007/s10639-023-12249-8
[21]
Chung Kwan Lo. 2023. What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature. Education Sciences 13, 4, Article 410 (2023), 15 pages. https://doi.org/10.3390/educsci13040410
[22]
Leo S. Lo. 2023. The CLEAR path: A framework for enhancing information literacy through prompt engineering. The Journal of Academic Librarianship 49, 4, Article 102720 (2023), 3 pages. https://doi.org/10.1016/j.acalib.2023.102720
[23]
Louis Major, Gill A. Francis, and Maria Tsapali. 2021. The effectiveness of technology-supported personalised learning in low- and middle-income countries: A meta-analysis. British Journal of Educational Technology 52, 5 (2021), 1935–1964. https://doi.org/10.1111/bjet.13116
[24]
Emiliana Murgia, Zahra Abbasiantaeb, Mohammad Aliannejadi, Theo Huibers, Monica Landoni, and Maria Soledad Pera. 2023. ChatGPT in the Classroom: A Preliminary Exploration on the Feasibility of Adapting ChatGPT to Support Children’s Information Discovery. In Adjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization. Association for Computing Machinery, New York, NY, USA, 22–27. https://doi.org/10.1145/3563359.3597399
[25]
Emiliana Murgia, Maria Soledad Pera, Monica Landoni, and Theo Huibers. 2023. Children on ChatGPT Readability in an Educational Context: Myth or Opportunity?. In Adjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization. Association for Computing Machinery, New York, NY, USA, 311–316. https://doi.org/10.1145/3563359.3596996
[26]
José Noijons. 2017. Het construeren van gesloten vragen. In Toetsen op School, Piet F. Sanders (Ed.). Stichting Cito Instituut voor Toetsontwikkeling, Arnhem, The Netherlands, 95–108. https://cito.nl/media/avhgajek/toetsen-op-school-algemeen.pdf
[27]
OpenAI. 2024. OpenAI Platform. https://platform.openai.com/. Accessed: 20-03-24.
[28]
Organisation for Economic Co-operation and Development (OECD). 2012. PISA 2012 Released Field Mathematics Items - Question: Sailing Ships [PM923Q01]. https://www.oecd.org/pisa/pisaproducts/pisa2012-2006-rel-items-maths-ENG.pdf. Accessed: 2024-03-21.
[29]
Organisation for Economic Co-operation and Development (OECD). 2015. PISA 2015 Released Field Trial Cognitive Test Items - Question: Bird Migration [S656]. https://www.oecd.org/pisa/PISA2015Questions/platform/index.html?user=&domain=SCI&unit=S656-BirdMigration&lang=eng-ZZZ. Accessed: 2024-03-21.
[30]
Organisation for Economic Co-operation and Development (OECD). 2024. Programme for International Student Assessment (PISA). https://www.oecd.org/pisa/. Accessed: 20-03-24.
[31]
Maria Soledad Pera, Katherine L. Wright, Casey Kennington, and Jerry A. Fails. 2023. Children and Information Access: Fostering a Sense of Belonging. In Joint Proceedings of the IUI 2023 Workshops: HAI-GEN, ITAH, MILC, SHAI, SketchRec, SOCIALIZE(CEUR Workshop Proceedings, Vol. 3359), Alison Smith-Renner and Paul Taele (Eds.). CEUR, Sydney, Australia, 254–257. https://doi.org/10.1145/3563359.3596996
[32]
Daniel D. Pratt. 2002. Good Teaching: One Size Fits All?New Directions for Adult and Continuing Education 2002, 93 (2002), 5–16. https://doi.org/10.1002/ace.45
[33]
Sherry Ruan, Jiayu He, Rui Ying, Jonathan Burkle, Dunia Hakim, Anna Wang, Yufeng Yin, Lily Zhou, Qianyao Xu, Abdallah AbuHashem, Griffin Dietz, Elizabeth L. Murnane, Emma Brunskill, and James A. Landay. 2020. Supporting children’s math learning with feedback-augmented narrative technology. In Proceedings of the Interaction Design and Children Conference. Association for Computing Machinery, New York, NY, USA, 567–580. https://doi.org/10.1145/3392063.3394400
[34]
Lena Ivannova Ruiz-Rojas, Patricia Acosta-Vargas, Javier De-Moreta-Llovet, and Mario Gonzalez-Rodriguez. 2023. Empowering Education with Generative Artificial Intelligence Tools: Approach with an Instructional Design Matrix. Sustainability 15, 15, Article 11524 (2023), 20 pages. https://doi.org/10.3390/su151511524
[35]
Piet F. Sanders (Ed.). 2017. Toetsen op School. Stichting Cito Instituut voor Toetsontwikkeling, Arnhem, The Netherlands. https://cito.nl/media/avhgajek/toetsen-op-school-algemeen.pdf
[36]
Piet F. Sanders and Theo J.H.M. Eggen. 1993. Inleiding. In Psychometrie in de praktijk, Theo J.H.M. Eggen and Piet F. Sanders (Eds.). Stichting Cito Instituut voor Toetsontwikkeling, Arnhem, The Netherlands, 1–16.
[37]
Cornelis Sluijter. 1998. Toetsen en beslissen. Cito Instituut voor Toetsontwikkeling, Arnhem, The Netherlands.
[38]
Catherine Stinson. 2022. Algorithms are not neutral: Bias in collaborative filtering. AI and Ethics 2, 4 (2022), 763–770. https://doi.org/10.1007/s43681-022-00136-w
[39]
Danielle R. Thomas, Shivang Gupta, and Kenneth R. Koedinger. 2023. Comparative Analysis of Learnersourced Human-Graded and AI-Generated Responses for Autograding Online Tutor Lessons. In Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky, Ning Wang, Genaro Rebolledo-Mendez, Vania Dimitrova, Noboru Matsuda, and Olga C. Santos (Eds.). Springer Nature Switzerland, Cham, 714–719. https://doi.org/10.1007/978-3-031-36336-8_110
[40]
Jessica Van Brummelen, Viktoriya Tabunshchyk, and Tommy Heng. 2021. “Alexa, Can I Program You?”: Student Perceptions of Conversational Artificial Intelligence Before and After Programming Alexa. In Proceedings of the 20th Annual ACM Interaction Design and Children Conference. Association for Computing Machinery, New York, NY, USA, 305–313. https://doi.org/10.1145/3459990.3460730
[41]
Steven Vertovec. 2023. Superdiversity: Migration and social complexity. Taylor & Francis, New York, NY, USA. https://doi.org/10.4324/9780203503577
[42]
Ge Wang, Jun Zhao, Max Van Kleek, and Nigel Shadbolt. 2022. Informing Age-Appropriate AI: Examining Principles and Practices of AI for Children. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, Article 536, 29 pages. https://doi.org/10.1145/3491102.3502057
[43]
Ge Wang, Jun Zhao, Max Van Kleek, and Nigel Shadbolt. 2024. Challenges and opportunities in translating ethical AI principles into practice for children. Nature Machine Intelligence 6, 3 (2024), 265–270. https://doi.org/10.1038/s42256-024-00805-x
Index Terms
- Toward Personalised Learning Experiences: Beyond Prompt Engineering
Recommendations
Personalized E-learning system with self-regulated learning assisted mechanisms for promoting learning performance
With the rapid development of Internet technologies, the conventional computer-assisted learning (CAL) is gradually moving toward to web-based learning. Additionally, instructors typically base their teaching methods to simultaneously interact with all ...
From UCD to HCD and Beyond. Conciliating the Human Aims Between Philosophy and Design Education
HCI International 2020 - Late Breaking Papers: User Experience Design and Case StudiesAbstractUser-Centered Design (UCD) is defined as both design philosophy and process. Current education in UCD and Human-Centered Design (HCD) mainly focuses on teaching the process and set of specific UCD methods. Although HCD emphasizes on human factors, ...
Towards Personalized Instruction: Co-designing a Teacher-Centered Dashboard for Learning Engagement Analysis in Blended Learning Environments
Learning and Collaboration TechnologiesAbstractLearning engagement is an important factor in academic success and instructional quality. In authentic settings, analyzing learner engagement status can be challenging due to the complexity of its multidimensional construct. It requires constant ...
Comments
Information & Contributors
Information
Published In
June 2024
1049 pages
ISBN:9798400704420
DOI:10.1145/3628516
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 June 2024
Check for updates
Author Tags
Qualifiers
- Extended-abstract
- Research
- Refereed limited
Conference
IDC '24
Sponsor:
Acceptance Rates
Overall Acceptance Rate 172 of 578 submissions, 30%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 182Total Downloads
- Downloads (Last 12 months)182
- Downloads (Last 6 weeks)44
Reflects downloads up to 23 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format