Abstract
Visually impaired children have a great disadvantage in the modern society since their ability to use modern computer technology is limited due to inappropriate user interfaces. The aim of the work presented in this paper was to develop a multimodal software architecture and applications to support learning of visually impaired children. The software architecture is based on software agents, and has specific support for visual, auditory and haptic interaction. It has been used successfully with different groups of 7-8 year-old and 12 year-old visually impaired children. In this paper we discuss the enabling software technology and interaction techniques aimed to realize our goal and our experiences in the actual use of the system.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Patomäki S, Raisamo R, Salo J, Pasto V, Hippula A (2004) Experiences on haptic interfaces for visually impaired young children. In: Proceedings of sixth international conference on multimodal interfaces (ICMI’04): ACM Press, pp 281-288
SensAble Technologies Inc. http://www.sensable.com
Reachin Technologies AB. http://www.reachin.se
3Dconnexion. http://www.3dconnexion.com/
Kangassalo M (1997) The Formation of Children’s Conceptual Models Concerning a Particular Natural Phenomenon Using PICCO, a Pictorial Computer Simulation. Doctoral dissertation. Acta Universitatis Tamperensis 559. University of Tampere. Tampere, p 188
Kangassalo M (1991/1999) PICCO -kuvallinen tietokonesimulaatio [PICCO, Pictorial Computer Simulation]. CD-ROM 951-98035-0-5
Kangassalo M (1992) The pictorial computer-based simulation in natural sciences for children’s use. In: Ohsuga S, Kangassalo H, Jaakkola H, Hori K, Yonezaki N (eds) Information modelling and knowledge bases 3rd Foundations, theory and applications. IOS Press, Amsterdam, pp 511-524
Jansson G, Billberger K (1999) The PHANToM used without visual guidance. In: The first phantom users research symposium (PURS 99). http://mbi.dkfz-heidelberg.de/purs99
Magnusson C, Rassmus-Gröhn K, Sjöström C, Danielsson H (2002) Navigation and recognition in complex haptic virtual environments—reports from an extensive study with blind users. In: Proceedings of Eurohaptics 2002
Sjöström C (2001) Designing haptic computer interfaces for blind people. In: proceedings of sixth international symposium on signal processing and its applications (ISSPA 2001). IEEE, pp68-71
Sjöström C (2002) Non-visual haptic interaction design: guidelines and applications. Doctoral dissertation, Certec, Lund Institute of Technology
Computer graphics access for Blind people through a haptic virtual environment. http://www.grab-eu.com
Wood J, Magennis M, Arias E, Gutierrez T, Graupp H, Bergamasco M (2003) The design and evaluation of a computer game for the blind in the GRAB haptic audio virtual environment. In: Proceedings of Eurohaptics 2003
Moran D-B, Cheyer AJ, Julia LE, Martin DL, Park S (1997) Multimodal user interfaces in the open agent architecture. In: Proceedings of the 2nd international conference on intelligent user interfaces. ACM Press, pp61-68
Cheyer A, Julia L (1999) InfoWiz: An animated voice interactive information system. In: Third international conference on autonomous agents (Agents’99), communicative agents workshop
Moore R, Dowding J, Bratt H, Gawron J M, Gorfu Y, Cheyer A (1997) CommandTalk: A spoken-language interface for battlefield simulations. In: Proceedings of the fifth conference on applied natural language processing. morgan kaufmann publishers inc. San Francisco, CA, USA, pp1-7
Cheyer A, Julia L (1998) Multimodal Maps: An agent-based approach. multimodal human-computer communication. lecture notes in artificial intelligence 1374. Springer, pp111-121
Coutaz J (1987) PAC, an object oriented model for dialog design. in: proceedings of interact ‘87. North-Holland, pp431-436
Nigay L, Coutaz J (1995) A generic platform for addressing the multimodal challenge. In: Proceedings of ACM CHI’95. ACM press, pp98-105
Hietala P, Niemirepo T (1995) A framework for building agent-based learning environments. In: Proceedings of AI-ED 95: Artificial intelligence in education. AACE, p 578
Foundation for Intelligent Physical Agents. http://www.fipa.org/
CLIPS: A Tool for Building Expert Systems. http://www.ghg.net/clips/CLIPS.html
SQLite. http://www.sqlite.org/
Kangassalo M (1998) Modelling a natural phenomenon for a pictorial computer-based simulation. In: Kangassalo H, Charrel PJ, Jaakkola H (eds) Information modeling and knowledge bases IX. IOS press, Amsterdam, pp 239-254
Kangassalo M (1996) PICCO as a Cognitive Tool. In: Tanaka Y, Kangassalo H, Jaakkola H, Yamamoto A (eds) Information modelling and knowledge bases VII. IOS press, Amsterdam, pp 344-357
Lonka K, Hakkarainen K, Sintonen M (2000) Progressive inquiry learning for children–Experiences, Possibilities, Limitations. Eur Early Childhood Educ Res J 8:7-23
Kangassalo M, Raisamo R, Hietala P, Järvi J, Peltola K, Saarinen R, Tuominen E, Hippula A (2005) Proactive agents that support children’s exploratory learning. In: Kiyoki Y, Wangler B, Jaakkola H, Kangassalo H (eds) Information modelling and knowledge bases XVI. IOS press, Amsterdam, pp 123-133
Acknowledgments
This research was funded by Academy of Finland, Proactive Computing Research Program (grants 202179 and 202180).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Saarinen, R., Järvi, J., Raisamo, R. et al. Supporting visually impaired children with software agents in a multimodal learning environment. Virtual Reality 9, 108–117 (2006). https://doi.org/10.1007/s10055-005-0011-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10055-005-0011-5