skip to main content
research-article

Memory Manipulations in Extended Reality

Published: 19 April 2023 Publication History

Abstract

Human memory has notable limitations (e.g., forgetting) which have necessitated a variety of memory aids (e.g., calendars). As we grow closer to mass adoption of everyday Extended Reality (XR), which is frequently leveraging perceptual limitations (e.g., redirected walking), it becomes pertinent to consider how XR could leverage memory limitations (forgetting, distorting, persistence) to induce memory manipulations. As memories highly impact our self-perception, social interactions, and behaviors, there is a pressing need to understand XR Memory Manipulations (XRMMs). We ran three speculative design workshops (n=12), with XR and memory researchers creating 48 XRMM scenarios. Through thematic analysis, we define XRMMs, present a framework of their core components and reveal three classes (at encoding, pre-retrieval, at retrieval). Each class differs in terms of technology (AR, VR) and impact on memory (influencing quality of memories, inducing forgetting, distorting memories). We raise ethical concerns and discuss opportunities of perceptual and memory manipulations in XR.

Supplementary Material

MP4 File (3544548.3580988-video-preview.mp4)
Video Preview
MP4 File (3544548.3580988-talk-video.mp4)
Pre-recorded Video Presentation

References

[1]
Mahdi Nasrullah Al-Ameen, Matthew Wright, and Shannon Scielzo. 2015. Towards Making Random Passwords Memorable: Leveraging Users’ Cognitive Ability Through Multiple Cues. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems(CHI ’15). Association for Computing Machinery, New York, NY, USA, 2315–2324. https://doi.org/10.1145/2702123.2702241
[2]
Cristina Alberini. 2011. The Role of Reconsolidation and the Dynamic Process of Long-Term Memory Formation and Storage. Frontiers in Behavioral Neuroscience 5 (2011).
[3]
Cristina M. Alberini and Joseph E. LeDoux. 2013. Memory Reconsolidation. Current Biology 23, 17 (Sept. 2013), R746–R750. https://doi.org/10.1016/j.cub.2013.06.046
[4]
Nadisha-Marie Aliman and Leon Kester. 2020. Malicious Design in AIVR, Falsehood and Cybersecurity-oriented Immersive Defenses. In 2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR). 130–137. https://doi.org/10.1109/AIVR50618.2020.00031
[5]
Mahdi Azmandian, Mark Hancock, Hrvoje Benko, Eyal Ofek, and Andrew D. Wilson. 2016. Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences. In Proceedings of the 2016 Chi Conference on Human Factors in Computing Systems. 1968–1979.
[6]
Alan Baddeley, Vivien Lewis, Margery Eldridge, and Neil Thomson. 1984. Attention and Retrieval from Long-Term Memory. Journal of Experimental Psychology: General 113 (1984), 518–540. https://doi.org/10.1037/0096-3445.113.4.518
[7]
Harry P. Bahrick. 2000. Long-Term Maintenance of Knowledge.(2000).
[8]
Stefano Baldassi, Tadayoshi Kohno, Franziska Roesner, and Moqian Tian. 2018. Challenges and New Directions in Augmented Reality, Computer Security, and Neuroscience–Part 1: Risks to Sensation and Perception. arXiv preprint arXiv:1806.10557(2018). arxiv:1806.10557
[9]
Emma Berry, Narinder Kapur, Lyndsay Williams, Steve Hodges, Peter Watson, Gavin Smyth, James Srinivasan, Reg Smith, Barbara Wilson, and Ken Wood. 2007 Aug-Oct. The Use of a Wearable Camera, SenseCam, as a Pictorial Diary to Improve Autobiographical Memory in a Patient with Limbic Encephalitis: A Preliminary Report. Neuropsychological Rehabilitation 17, 4-5 (2007 Aug-Oct), 582–601. https://doi.org/10.1080/09602010601029780
[10]
Agon Bexheti, Marc Langheinrich, Ivan Elhart, and Nigel Davies. 2019. Securely Storing and Sharing Memory Cues in Memory Augmentation Systems: A Practical Approach. In 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom. 1–10. https://doi.org/10.1109/PERCOM.2019.8767389
[11]
Mark Billinghurst, Adrian Clark, and Gun Lee. 2015. A Survey of Augmented Reality. Foundations and Trends® in Human–Computer Interaction 8, 2-3 (March 2015), 73–272. https://doi.org/10.1561/1100000049
[12]
Olaf Blanke, Lucie Bréchet, Robin Mange, Bruno Herbelin, Baptiste Gauthier, and Andrea Serino. 2018. Viewing One’s Body during Encoding Boosts Episodic Memory. https://doi.org/10.1101/318956
[13]
Philippe Blondé, Jean-Charles Girardeau, Marco Sperduti, and Pascale Piolino. 2022. A Wandering Mind Is a Forgetful Mind: A Systematic Review on the Influence of Mind Wandering on Episodic Memory Encoding. Neuroscience & Biobehavioral Reviews 132 (Jan. 2022), 774–792. https://doi.org/10.1016/j.neubiorev.2021.11.015
[14]
Susan Bluck, Nicole Alea, Tilmann Habermas, and David C. Rubin. 2005. A TALE of Three Functions: The Self–Reported Uses of Autobiographical Memory. Social Cognition 23, 1 (Feb. 2005), 91–117. https://doi.org/10.1521/soco.23.1.91.59198
[15]
Joseph Bonneau and Stuart Schechter. 2014. Towards Reliable Storage of 56-Bit Secrets in Human Memory. In 23rd USENIX Security Symposium (USENIX Security 14). 607–623.
[16]
Fred B. Bryant, Colette M. Smart, and Scott P. King. 2005. Using the Past to Enhance the Present: Boosting Happiness Through Positive Reminiscence. Journal of Happiness Studies 6, 3 (Sept. 2005), 227–260. https://doi.org/10.1007/s10902-005-3889-4
[17]
Larry Cahill and James L. McGaugh. 1998. Mechanisms of Emotional Arousal and Lasting Declarative Memory. Trends in Neurosciences 21, 7 (July 1998), 294–299. https://doi.org/10.1016/S0166-2236(97)01214-9
[18]
Niamh Caprani, John Greaney, and Nicola Porter. 2006. A Review of Memory Aid Devices for an Ageing Population. PsychNology Journal 4, 3 (2006), 205–243.
[19]
Peter Casey, Ibrahim Baggili, and Ananya Yarramreddy. 2021. Immersive Virtual Reality Attacks and the Human Joystick. IEEE Transactions on Dependable and Secure Computing 18, 2 (March 2021), 550–562. https://doi.org/10.1109/TDSC.2019.2907942
[20]
Claude Castelluccia, Markus Duermuth, Maximilian Golla, and Fatma Deniz. 2017. Towards Implicit Visual Memory-Based Authentication. In Proceedings 2017 Network and Distributed System Security Symposium. Internet Society, San Diego, CA. https://doi.org/10.14722/ndss.2017.23292
[21]
Neil Charness, Ryan Best, and Dustin Souders. 2012. Memory Function and Supportive Technology. Gerontechnology : international journal on the fundamental aspects of technology to serve the ageing society 11, 1 (2012), 10.4017/gt.2012.11.01.006.00. https://doi.org/10.4017/gt.2012.11.01.006.00
[22]
Kaiming Cheng, Jeffery F Tian, Tadayoshi Kohno, and Franziska Roesner. 2023. Exploring User Reactions and Mental Models Towards Perceptual Manipulation Attacks in Mixed Reality. USENIX Security 2023(2023), 18.
[23]
Lung-Pan Cheng, Eyal Ofek, Christian Holz, Hrvoje Benko, and Andrew D. Wilson. 2017. Sparse Haptic Proxy: Touch Feedback in Virtual Environments Using a General Passive Prop. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI ’17). Association for Computing Machinery, New York, NY, USA, 3718–3728. https://doi.org/10.1145/3025453.3025753
[24]
Yi Fei Cheng, Hang Yin, Yukang Yan, Jan Gugenheimer, and David Lindlbauer. 2022. Towards Understanding Diminished Reality. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI ’22). Association for Computing Machinery, New York, NY, USA, 1–16. https://doi.org/10.1145/3491102.3517452
[25]
Sarah Clinch, Cathleen Cortis Mack, Geoff Ward, and Madeleine Steeds. 2021. Technology-Mediated Memory Impairment. In Technology-Augmented Perception and Cognition, Tilman Dingler and Evangelos Niforatos (Eds.). Springer International Publishing, Cham, 71–124. https://doi.org/10.1007/978-3-030-30457-7_4
[26]
Dan Cosley, Kathy Akey, Brian Alson, Jonathan Baxter, Mark Broomfield, Soyoung Lee, and Chethan Sarabu. 2009. Using Technologies to Support Reminiscence. In British Computer Society. 480–484.
[27]
Nelson Cowan. 1988. Evolving Conceptions of Memory Storage, Selective Attention, and Their Mutual Constraints within the Human Information-Processing System.Psychological bulletin 104, 2 (1988), 163.
[28]
Fergus I. M. Craik, Richard Govoni, Moshe Naveh-Benjamin, and Nicole D. Anderson. 1996. The Effects of Divided Attention on Encoding and Retrieval Processes in Human Memory. Journal of Experimental Psychology: General 125 (1996), 159–180. https://doi.org/10.1037/0096-3445.125.2.159
[29]
Anne A. Cuperus and Ineke J. M. van der Ham. 2016. Virtual Reality Replays of Sports Performance: Effects on Memory, Feeling of Competence, and Performance. Learning and Motivation 56 (Nov. 2016), 48–52. https://doi.org/10.1016/j.lmot.2016.09.005
[30]
Sauvik Das, David Lu, Taehoon Lee, Joanne Lo, and Jason I. Hong. 2019. The Memory Palace: Exploring Visual-Spatial Paths for Strong, Memorable, Infrequent Authentication. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology(UIST ’19). Association for Computing Machinery, New York, NY, USA, 1109–1121. https://doi.org/10.1145/3332165.3347917
[31]
Nigel Davies, Adrian Friday, Sarah Clinch, Corina Sas, Marc Langheinrich, Geoff Ward, and Albrecht Schmidt. 2015. Security and Privacy Implications of Pervasive Memory Augmentation. IEEE Pervasive Computing 14 (Jan. 2015), 44–53. https://doi.org/10.1109/MPRV.2015.13
[32]
Robert Desimone and John Duncan. 1995. Neural Mechanisms of Selective Visual Attention. Annual review of neuroscience 18, 1 (1995), 193–222.
[33]
JoAnn Difede and Judith Cukor. 2007. Virtual Reality Exposure Therapy for the Treatment of Posttraumatic Stress Disorder Following September 11, 2001. The Journal of Clinical Psychiatry 68, 11 (Nov. 2007), 1682–1689. https://doi.org/10.4088/JCP.v68n1102
[34]
H.Q. Dinh, N. Walker, L.F. Hodges, Chang Song, and A. Kobayashi. 1999. Evaluating the Importance of Multi-Sensory Input on Memory and the Sense of Presence in Virtual Environments. In Proceedings IEEE Virtual Reality (Cat. No. 99CB36316). 222–228. https://doi.org/10.1109/VR.1999.756955
[35]
Jayesh Doolani, Matthew Wright, Rajesh Setty, and S M Taiabul Haque. 2021. LociMotion: Towards Learning a Strong Authentication Secret in a Single Session. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI ’21). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3411764.3445105
[36]
John V. Draper, David B. Kaber, and John M. Usher. 1998. Telepresence. Human Factors 40, 3 (Sept. 1998), 354–375. https://doi.org/10.1518/001872098779591386
[37]
Hermann Ebbinghaus (1885). 1885. Memory: A Contribution to Experimental Psychology. Annals of Neurosciences 20, 4 (1885), 155–156. https://doi.org/10.5214/ans.0972.7531.200408
[38]
Emmaline Drew Eliseev and Elizabeth J. Marsh. 2021. Externalizing Autobiographical Memories in the Digital Age. Trends in Cognitive Sciences 25, 12 (Dec. 2021), 1072–1081. https://doi.org/10.1016/j.tics.2021.08.005
[39]
Douglas C Engelbart. 1962. Augmenting human intellect: A conceptual framework. Menlo Park, CA (1962), 21.
[40]
Michael W. Eysenck and Mark T. Keane. 2020. Cognitive Psychology: A Student’s Handbook. Psychology Press.
[41]
Anna Felnhofer, Oswald D. Kothgassner, Mareike Schmidt, Anna-Katharina Heinzle, Leon Beutl, Helmut Hlavacs, and Ilse Kryspin-Exner. 2015. Is Virtual Reality Emotionally Arousing? Investigating Five Emotion Inducing Virtual Park Scenarios. International Journal of Human-Computer Studies 82 (Oct. 2015), 48–56. https://doi.org/10.1016/j.ijhcs.2015.05.004
[42]
Andreas��Rene Fender and Christian Holz. 2022. Causality-Preserving Asynchronous Reality. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI ’22). Association for Computing Machinery, New York, NY, USA, 1–15. https://doi.org/10.1145/3491102.3501836
[43]
Ajoy S. Fernandes, Ranxiao Frances Wang, and Daniel J. Simons. 2015. Remembering the Physical as Virtual: Source Confusion and Physical Interaction in Augmented Reality. In Proceedings of the ACM SIGGRAPH Symposium on Applied Perception(SAP ’15). Association for Computing Machinery, New York, NY, USA, 127–130. https://doi.org/10.1145/2804408.2804423
[44]
The Neurorights Foundation. 2021. Mission. Retrieved January 25, 2023 from https://neurorightsfoundation.org/mission
[45]
Doron Friedman, Rodrigo Pizarro, Keren Or-Berkers, Solène Neyret, Xueni Pan, and Mel Slater. 2014. A Method for Generating an Illusion of Backwards Time Travel Using Immersive Virtual Reality—an Exploratory Study. Frontiers in psychology 5 (2014), 943.
[46]
Maryrose Gerardi, Barbara Olasov Rothbaum, Kerry Ressler, Mary Heekin, and Albert Rizzo. 2008. Virtual reality exposure therapy using a virtual Iraq: Case report. Journal of Traumatic Stress 21, 2 (2008), 209–213. https://doi.org/10.1002/jts.20331
[47]
Mar Gonzalez-Franco and Jaron Lanier. 2017. Model of Illusions and Virtual Reality. Frontiers in Psychology 8 (2017), 1125. https://doi.org/10.3389/fpsyg.2017.01125
[48]
Jan Gugenheimer. 2016. Nomadic Virtual Reality: Exploring New Interaction Concepts for Mobile Virtual Reality Head-Mounted Displays. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. 9–12.
[49]
Jan Gugenheimer, Christian Mai, Mark McGill, Julie Williamson, Frank Steinicke, and Ken Perlin. 2019. Challenges Using Head-Mounted Displays in Shared and Social Spaces. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1–8.
[50]
Jan Gugenheimer, Mark McGill, Samuel Huron, Christian Mai, Julie Williamson, and Michael Nebeling. 2020. Exploring Potentially Abusive Ethical, Social and Political Implications of Mixed Reality Research in HCI. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems(CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3375180
[51]
Cathal Gurrin, Alan F. Smeaton, and Aiden R. Doherty. 2014. LifeLogging: Personal Big Data. Foundations and Trends® in Information Retrieval 8, 1 (June 2014), 1–125. https://doi.org/10.1561/1500000033
[52]
Morgan Harvey, Marc Langheinrich, and Geoff Ward. 2016. Remembering through Lifelogging: A Survey of Human Memory Augmentation. Pervasive and Mobile Computing 27 (April 2016), 14–26. https://doi.org/10.1016/j.pmcj.2015.12.002
[53]
Linda A. Henkel. 2014. Point-and-Shoot Memories: The Influence of Taking Photos on Memory for a Museum Tour. Psychological Science 25, 2 (Feb. 2014), 396–402. https://doi.org/10.1177/0956797613504438
[54]
Jan Herling and Wolfgang Broll. 2012. Pixmix: A Real-Time Approach to High-Quality Diminished Reality. In 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 141–150.
[55]
H.G. Hoffman, K.C. Hullfish, and S.J. Houston. 1995. Virtual-Reality Monitoring. In Proceedings Virtual Reality Annual International Symposium ’95. 48–54. https://doi.org/10.1109/VRAIS.1995.512479
[56]
Hunter G. Hoffman, Azucena Garcia-Palacios, Ayanna K. Thomas, and Anne Schmidt. 2001. Virtual Reality Monitoring: Phenomenal Characteristics of Real, Virtual, and False Memories. CyberPsychology & Behavior 4, 5 (Oct. 2001), 565–572. https://doi.org/10.1089/109493101753235151
[57]
Ellen Isaacs, Artie Konrad, Alan Walendowski, Thomas Lennig, Victoria Hollis, and Steve Whittaker. 2013. Echoes from the Past: How Technology Mediated Reflection Improves Well-Being. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1071–1080.
[58]
Matthew Jamieson, Breda Cullen, Marilyn McGee-Lennon, Stephen Brewster, and Jonathan J. Evans. 2014. The Efficacy of Cognitive Prosthetic Technology for People with Memory Impairments: A Systematic Review and Meta-Analysis. Neuropsychological Rehabilitation 24, 3-4 (July 2014), 419–444. https://doi.org/10.1080/09602011.2013.825632
[59]
Matthew Jamieson, Brian O’Neill, Breda Cullen, Marilyn Lennon, Stephen Brewster, and Jonathan Evans. 2017. ForgetMeNot: Active Reminder Entry Support for Adults with Acquired Brain Injury. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI ’17). Association for Computing Machinery, New York, NY, USA, 6012–6023. https://doi.org/10.1145/3025453.3025888
[60]
Marcia K. Johnson, Shahin Hashtroudi, and D. Stephen Lindsay. 1993. Source Monitoring. Psychological Bulletin 114, 1 (1993), 3–28. https://doi.org/10.1037/0033-2909.114.1.3
[61]
Marcia K. Johnson and Carol L. Raye. 1981. Reality Monitoring. Psychological Review 88, 1 (1981), 67–85. https://doi.org/10.1037/0033-295X.88.1.67
[62]
Zeinab Joudaki, Julie Thorpe, and Miguel Vargas Martin. 2018. Reinforcing System-Assigned Passphrases Through Implicit Learning. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security(CCS ’18). Association for Computing Machinery, New York, NY, USA, 1533–1548. https://doi.org/10.1145/3243734.3243764
[63]
Joanna Kisker, Thomas Gruber, and Benjamin Schöne. 2021. Virtual Reality Experiences Promote Autobiographical Retrieval Mechanisms: Electrophysiological Correlates of Laboratory and Virtual Experiences. Psychological Research 85, 7 (Oct. 2021), 2485–2501. https://doi.org/10.1007/s00426-020-01417-x
[64]
Cadu Klier and Luciano Grüdtner Buratto. 2020. Stress and Long-Term Memory Retrieval: A Systematic Review. Trends in Psychiatry and Psychotherapy 42, 3 (2020), 284–291. https://doi.org/10.1590/2237-6089-2019-0077
[65]
Artie Konrad, Ellen Isaacs, and Steve Whittaker. 2016. Technology-Mediated Memory: Is Technology Altering Our Memories And Interfering With Well-Being?ACM Transactions on Computer-Human Interaction 23, 4 (Aug. 2016), 23:1–23:29. https://doi.org/10.1145/2934667
[66]
Artie Konrad, Simon Tucker, John Crane, and Steve Whittaker. 2016. Technology and Reflection: Mood and Memory Mechanisms for Well-Being. Psychology of Well-Being 6 (2016), 5. https://doi.org/10.1186/s13612-016-0045-3
[67]
Michael Kotlyar, Christopher Donahue, Paul Thuras, Matt G. Kushner, Natalie O’Gorman, Erin A Smith, and David E. Adson. 2008. Physiological Response to a Speech Stressor Presented in a Virtual Reality Environment. Psychophysiology 45, 6 (2008), 1034–1037. https://doi.org/10.1111/j.1469-8986.2008.00690.x
[68]
Eric Krokos, Catherine Plaisant, and Amitabh Varshney. 2019. Virtual Memory Palaces: Immersion Aids Recall. Virtual Reality 23, 1 (March 2019), 1–15. https://doi.org/10.1007/s10055-018-0346-3
[69]
Kiron Lebeck, Tadayoshi Kohno, and Franziska Roesner. 2016. How to Safely Augment Reality: Challenges and Directions. In Proceedings of the 17th International Workshop on Mobile Computing Systems and Applications. 45–50.
[70]
Kiron Lebeck, Kimberly Ruth, Tadayoshi Kohno, and Franziska Roesner. 2017. Securing Augmented Reality Output. In 2017 IEEE Symposium on Security and Privacy (SP). IEEE, 320–337.
[71]
Eric L. G. Legge, Christopher R. Madan, Enoch T. Ng, and Jeremy B. Caplan. 2012. Building a Memory Palace in Minutes: Equivalent Memory Performance Using Virtual versus Conventional Environments with the Method of Loci. Acta Psychologica 141, 3 (Nov. 2012), 380–390. https://doi.org/10.1016/j.actpsy.2012.09.002
[72]
Eric L. G. Legge, Christopher R. Madan, Enoch T. Ng, and Jeremy B. Caplan. 2012. Building a Memory Palace in Minutes: Equivalent Memory Performance Using Virtual versus Conventional Environments with the Method of Loci. Acta Psychologica 141, 3 (Nov. 2012), 380–390. https://doi.org/10.1016/j.actpsy.2012.09.002
[73]
Diane Lenormand and Pascale Piolino. 2022. In Search of a Naturalistic Neuroimaging Approach: Exploration of General Feasibility through the Case of VR-fMRI and Application in the Domain of Episodic Memory. Neuroscience & Biobehavioral Reviews 133 (Feb. 2022), 104499. https://doi.org/10.1016/j.neubiorev.2021.12.022
[74]
David Lindlbauer and Andy D. Wilson. 2018. Remixed Reality: Manipulating Space and Time in Augmented Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems(CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3173703
[75]
Elizabeth F. Loftus. 2005. Planting Misinformation in the Human Mind: A 30-Year Investigation of the Malleability of Memory. Learning & Memory 12, 4 (Jan. 2005), 361–366. https://doi.org/10.1101/lm.94705
[76]
Elizabeth F. Loftus and Jacqueline E. Pickrell. 1995. The Formation of False Memories. Psychiatric Annals 25, 12 (Dec. 1995), 720–725. https://doi.org/10.3928/0048-5713-19951201-07
[77]
Dominique Makowski, Marco Sperduti, Serge Nicolas, and Pascale Piolino. 2017. “Being There” and Remembering It: Presence Improves Memory Encoding. Consciousness and cognition 53 (2017), 194–202.
[78]
Marieke AG Martens, Angus Antley, Daniel Freeman, Mel Slater, Paul J Harrison, and Elizabeth M Tunbridge. 2019. It Feels Real: Physiological Responses to a Stressful Virtual Reality Environment and Its Impact on Working Memory. Journal of Psychopharmacology 33, 10 (Oct. 2019), 1264–1273. https://doi.org/10.1177/0269881119860156
[79]
Sebastian Marwecki, Andrew D. Wilson, Eyal Ofek, Mar Gonzalez Franco, and Christian Holz. 2019. Mise-Unseen: Using Eye Tracking to Hide Virtual Reality Scene Changes in Plain Sight. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology(UIST ’19). Association for Computing Machinery, New York, NY, USA, 777–789. https://doi.org/10.1145/3332165.3347919
[80]
A. McDonald, C. Haslam, P. Yates, B. Gurr, G. Leeder, and A. Sayers. 2011. Google Calendar: A New Memory Aid to Compensate for Prospective Memory Deficits Following Acquired Brain Injury. Neuropsychological Rehabilitation 21, 6 (Dec. 2011), 784–807. https://doi.org/10.1080/09602011.2011.598405
[81]
James L McGaugh and Benno Roozendaal. 2002. Role of Adrenal Stress Hormones in Forming Lasting Memories in the Brain. Current Opinion in Neurobiology 12, 2 (April 2002), 205–210. https://doi.org/10.1016/S0959-4388(02)00306-9
[82]
Arthur W. Melton. 1963. Implications of Short-Term Memory for a General Theory of Memory. Journal of Verbal Learning and Verbal Behavior 2, 1 (July 1963), 1–21. https://doi.org/10.1016/S0022-5371(63)80063-8
[83]
Abraham Hani Mhaidli and Florian Schaub. 2021. Identifying Manipulative Advertising Techniques in XR Through Scenario Construction. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI ’21). Association for Computing Machinery, New York, NY, USA, 1–18. https://doi.org/10.1145/3411764.3445253
[84]
Paul Milgram and Fumio Kishino. 1994. A Taxonomy of Mixed Reality Visual Displays. IEICE Trans. Information Systems vol. E77-D, no. 12 (Dec. 1994), 1321–1329.
[85]
Shohei Mori, Sei Ikeda, and Hideo Saito. 2017. A Survey of Diminished Reality: Techniques for Visually Concealing, Eliminating, and Seeing through Real Objects. IPSJ Transactions on Computer Vision and Applications 9, 1 (June 2017), 17. https://doi.org/10.1186/s41074-017-0028-1
[86]
Robert A. Nash, Kimberley A. Wade, and D. Stephen Lindsay. 2009. Digitally Manipulating Memory: Effects of Doctored Videos and Imagination in Distorting Beliefs and Memories. Memory & Cognition 37, 4 (June 2009), 414–424. https://doi.org/10.3758/MC.37.4.414
[87]
BBC World News. 2020. Click, Virtual Time Machine: Watching My Life from 2019. Retrieved January 25, 2023 from https://www.bbc.co.uk/programmes/p08nwkw4
[88]
Alasdair Newson, Andrés Almansa, Matthieu Fradet, Yann Gousseau, and Patrick Pérez. 2013. Towards Fast, Generic Video Inpainting. In Proceedings of the 10th European Conference on Visual Media Production(CVMP ’13). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/2534008.2534019
[89]
OpenAI. 2022. DALL · E 2. Retrieved January 25, 2023 from https://openai.com/dall-e-2/
[90]
Sylvain Penaud, Najate Jebara, Mohamed Zaoui, Eric Orriols, Alain Berthoz, and Pascale Piolino. 2022. Episodic Memory and Self-Reference in a Naturalistic Context: New Insights Based on a Virtual Walk in the Latin Quarter of Paris. Journal of Environmental Psychology 81 (2022), 101801.
[91]
James W. Pennebaker and Sandra K. Beall. 1986. Confronting a Traumatic Event: Toward an Understanding of Inhibition and Disease.Journal of abnormal psychology 95, 3 (1986), 274.
[92]
PetaPixel. 2018. Facebook Can Reconstruct a 3D World from Your Photos and Videos. Retrieved January 25, 2023 from https://petapixel.com/2018/05/02/facebook-can-reconstruct-a-3d-world-from-your-photos-and-videos/
[93]
Elizabeth A Phelps. 2004. Human Emotion and Memory: Interactions of the Amygdala and Hippocampal Complex. Current Opinion in Neurobiology 14, 2 (April 2004), 198–202. https://doi.org/10.1016/j.conb.2004.03.015
[94]
Laurence Picard, Maria Abram, Eric Orriols, and Pascale Piolino. 2017. Virtual Reality as an Ecologically Valid Tool for Assessing Multifaceted Episodic Memory in Children and Adolescents. International Journal of Behavioral Development 41, 2 (March 2017), 211–219. https://doi.org/10.1177/0165025415616198
[95]
Michael I. Posner, Mary J. Nissen, and Raymond M. Klein. 1976. Visual Dominance: An Information-Processing Account of Its Origins and Significance.Psychological review 83, 2 (1976), 157.
[96]
Zhen Qin, Qingliang Zeng, Yixin Zong, and Fan Xu. 2021. Image Inpainting Based on Deep Learning: A Review. Displays 69 (Sept. 2021), 102028. https://doi.org/10.1016/j.displa.2021.102028
[97]
Sharif Razzaque, David Swapp, Mel Slater, Mary Whitton, and Anthony Steed. 2002. Redirected Walking in Place. Eight Eurographics Workshop on Virtual Environments (Jan. 2002).
[98]
E. F. Risko, M. O. Kelly, P. Patel, and C. Gaspar. 2019. Offloading Memory Leaves Us Vulnerable to Memory Manipulation. Cognition 191 (Oct. 2019), 103954. https://doi.org/10.1016/j.cognition.2019.04.023
[99]
Franziska Roesner and Tadayoshi Kohno. 2021. Security and Privacy for Augmented Reality: Our 10-Year Retrospective. In VR4Sec: 1st International Workshop on Security for XR and XR for Security.
[100]
Franziska Roesner, Tadayoshi Kohno, and David Molnar. 2014. Security and Privacy for Augmented Reality Systems. Commun. ACM 57, 4 (2014), 88–96.
[101]
Oscar Rosello, Marc Exposito, and Pattie Maes. 2016. NeverMind: Using Augmented Reality for Memorization. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology(UIST ’16 Adjunct). Association for Computing Machinery, New York, NY, USA, 215–216. https://doi.org/10.1145/2984751.2984776
[102]
Marius Rubo, Nadine Messerli, and Simone Munsch. 2021. The Human Source Memory System Struggles to Distinguish Virtual Reality and Reality. Computers in Human Behavior Reports 4 (Aug. 2021), 100111. https://doi.org/10.1016/j.chbr.2021.100111
[103]
Maria V. Sanchez-Vives and Mel Slater. 2005. From Presence to Consciousness through Virtual Reality. Nature Reviews Neuroscience 6, 4 (April 2005), 332–339. https://doi.org/10.1038/nrn1651
[104]
Susan J. Sara. 2000. Retrieval and Reconsolidation: Toward a Neurobiology of Remembering. Learning & Memory 7, 2 (Jan. 2000), 73–84. https://doi.org/10.1101/lm.7.2.73
[105]
Corina Sas and Steve Whittaker. 2013. Design for Forgetting: Disposing of Digital Possessions after a Breakup. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’13). Association for Computing Machinery, New York, NY, USA, 1823–1832. https://doi.org/10.1145/2470654.2466241
[106]
Daniel L. Schacter. 2002. The Seven Sins of Memory: How the Mind Forgets and Remembers. HMH.
[107]
Daniel L. Schacter. 2022. Media, Technology, and the Sins of Memory. Memory, Mind & Media 1(2022), e1. https://doi.org/10.1017/mem.2021.3
[108]
Daniel L. Schacter and Chad S. Dodson. 2001. Misattribution, False Recognition and the Sins of Memory. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences 356, 1413 (Sept. 2001), 1385–1393. https://doi.org/10.1098/rstb.2001.0938
[109]
Kathryn Y. Segovia and Jeremy N. Bailenson. 2009. Virtually True: Children’s Acquisition of False Memories in Virtual Reality. Media Psychology 12, 4 (Nov. 2009), 371–393. https://doi.org/10.1080/15213260903287267
[110]
Abigail J. Sellen, Andrew Fogg, Mike Aitken, Steve Hodges, Carsten Rother, and Ken Wood. 2007. Do Life-Logging Technologies Support Memory for the Past? An Experimental Study Using Sensecam. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’07). Association for Computing Machinery, New York, NY, USA, 81–90. https://doi.org/10.1145/1240624.1240636
[111]
Andrew M. Sherrill, Alex O. Rothbaum, Lauren B. McSweeney, and Barbara O. Rothbaum. 2019. Virtual Reality Exposure Therapy for Posttraumatic Stress Disorder. Psychiatric Annals 49, 8 (Aug. 2019), 343–347. https://doi.org/10.3928/00485713-20190716-02
[112]
R. M. Shiffrin and R. C. Atkinson. 1969. Storage and Retrieval Processes in Long-Term Memory. Psychological Review 76(1969), 179–193. https://doi.org/10.1037/h0027277
[113]
Youngsoo Shin, Ruth Barankevich, Jina Lee, and Saleh Kalantari. 2021. PENCODER: Design for Prospective Memory and Older Adults. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems(CHI EA ’21). Association for Computing Machinery, New York, NY, USA, 1–7. https://doi.org/10.1145/3411763.3451795
[114]
Panote Siriaraya and Chee Siang Ang. 2014. Recreating Living Experiences from Past Memories through Virtual Worlds for People with Dementia. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’14). Association for Computing Machinery, New York, NY, USA, 3977–3986. https://doi.org/10.1145/2556288.2557035
[115]
Mel Slater. 2009. Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments. Philosophical Transactions of the Royal Society B: Biological Sciences 364, 1535 (2009), 3549–3557.
[116]
Mel Slater, Cristina Gonzalez-Liencres, Patrick Haggard, Charlotte Vinkers, Rebecca Gregory-Clarke, Steve Jelley, Zillah Watson, Graham Breen, Raz Schwarz, William Steptoe, 2020. The Ethics of Realism in Virtual and Augmented Reality. Frontiers in Virtual Reality 1 (2020), 1.
[117]
Mel Slater and Sylvia Wilbur. 1997. A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments. Presence: Teleoperators & Virtual Environments 6, 6(1997), 603–616.
[118]
S. Adam Smith. 2019. Virtual Reality in Episodic Memory Research: A Review. Psychonomic Bulletin & Review 26, 4 (Aug. 2019), 1213–1237. https://doi.org/10.3758/s13423-019-01605-w
[119]
Bernhard Spanlang, Jean-Marie Normand, David Borland, Konstantina Kilteni, Elias Giannopoulos, Ausiàs Pomés, Mar González-Franco, Daniel Perez-Marcos, Jorge Arroyo-Palacios, Xavi Navarro Muncunill, and Mel Slater. 2014. How to Build an Embodiment Lab: Achieving Body Representation Illusions in Virtual Reality. Frontiers in Robotics and AI 1 (2014).
[120]
Jonathan D Spence. 1984. The Memory Palace of Matteo Ricci. Viking.
[121]
Stability.Ai. 2022. Stable Diffusion 2.0 Release. Retrieved January 25, 2023 from https://stability.ai/blog/stable-diffusion-v2-release
[122]
Frank Steinicke, Gerd Bruder, Jason Jerald, Harald Frenz, and Markus Lappe. 2010. Estimation of Detection Thresholds for Redirected Walking Techniques. IEEE Transactions on Visualization and Computer Graphics 16, 1 (Jan. 2010), 17–27. https://doi.org/10.1109/TVCG.2009.62
[123]
Qi Sun, Anjul Patney, Li-Yi Wei, Omer Shapira, Jingwan Lu, Paul Asente, Suwen Zhu, Morgan Mcguire, David Luebke, and Arie Kaufman. 2018. Towards Virtual Reality Infinite Walking: Dynamic Saccadic Redirection. ACM Transactions on Graphics 37, 4 (July 2018), 67:1–67:13. https://doi.org/10.1145/3197517.3201294
[124]
Charles P. Thompson, John J. Skowronski, Steen F. Larsen, and Andrew L. Betz. 2013. Autobiographical Memory: Remembering What and Remembering When. Psychology Press.
[125]
Yung-Chin Tsao, Chun-Chieh Shu, and Tian-Syung Lan. 2019. Development of a Reminiscence Therapy System for the Elderly Using the Integration of Virtual Reality and Augmented Reality. Sustainability 11, 17 (Jan. 2019), 4792. https://doi.org/10.3390/su11174792
[126]
Wen-Jie Tseng, Elise Bonnail, Mark McGill, Mohamed Khamis, Eric Lecolinet, Samuel Huron, and Jan Gugenheimer. 2022. The Dark Side of Perceptual Manipulations in Virtual Reality. arXiv:2202.13200 [cs] (Feb. 2022). https://doi.org/10.1145/3491102.3517728 arxiv:2202.13200 [cs]
[127]
Chai M. Tyng, Hafeez U. Amin, Mohamad N. M. Saad, and Aamir S. Malik. 2017. The Influences of Emotion on Learning and Memory. Frontiers in Psychology 8 (Aug. 2017), 1454. https://doi.org/10.3389/fpsyg.2017.01454
[128]
Eunjoon “Rachel” Um, Jan L. Plass, Elizabeth O. Hayward, and Bruce D. Homer. 2012. Emotional Design in Multimedia Learning. Journal of Educational Psychology 104, 2 (2012), 485–498. https://doi.org/10.1037/a0026609
[129]
Melina R. Uncapher and Michael D. Rugg. 2009. Selecting for Memory? The Influence of Selective Attention on the Mnemonic Binding of Contextual Information. Journal of Neuroscience 29, 25 (June 2009), 8270–8279. https://doi.org/10.1523/JNEUROSCI.1043-09.2009
[130]
Robert J. van Beers, Daniel M. Wolpert, and Patrick Haggard. 2002. When Feeling Is More Important than Seeing in Sensorimotor Adaptation. Current biology 12, 10 (2002), 834–837.
[131]
Joakim Vindenes, Angelica Ortiz de Gortari, and Barbara Wasson. 2018. Mnemosyne: Adapting the Method of Loci to Immersive Virtual Reality. In Augmented Reality, Virtual Reality, and Computer Graphics(Lecture Notes in Computer Science), Lucio Tommaso De Paolis and Patrick Bourdot (Eds.). Springer International Publishing, Cham, 205–213. https://doi.org/10.1007/978-3-319-95270-3_16
[132]
Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and Tell: A Neural Image Caption Generator. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3156–3164.
[133]
Susanne Vogel and Lars Schwabe. 2016. Learning and Memory under Stress: Implications for the Classroom. npj Science of Learning 1, 1 (June 2016), 1–10. https://doi.org/10.1038/npjscilearn.2016.11
[134]
Kimberley A. Wade, Maryanne Garry, J. Don Read, and D. Stephen Lindsay. 2002. A Picture Is Worth a Thousand Lies: Using False Photographs to Create False Childhood Memories. Psychonomic Bulletin & Review 9, 3 (Sept. 2002), 597–603. https://doi.org/10.3758/BF03196318
[135]
Oliver T Wolf. 2017. Stress and Memory Retrieval: Mechanisms and Consequences. Current Opinion in Behavioral Sciences 14 (April 2017), 40–46. https://doi.org/10.1016/j.cobeha.2016.12.001
[136]
Bob Woods, Laura O’Philbin, Emma M. Farrell, Aimee E. Spector, and Martin Orrell. 2018. Reminiscence Therapy for Dementia. Cochrane Database of Systematic Reviews3 (2018). https://doi.org/10.1002/14651858.CD001120.pub3
[137]
Jiaxin Xu and Baosheng Wang. 2020. Efficacy of VR-Based Reminiscence Therapy in Improving Autobiographical Memory for Chinese Patients with AD. 339–349. https://doi.org/10.1007/978-3-030-51038-1_47
[138]
Yuki Yamada, Keisuke Irie, Kota Gushima, Fumiko Ishizawa, Mohammed Al Sada, and Tatsuo Nakajima. 2017. HoloMoL: Human Memory Augmentation with Mixed-Reality Technologies. In Proceedings of the 21st International Academic Mindtrek Conference(AcademicMindtrek ’17). Association for Computing Machinery, New York, NY, USA, 235–238. https://doi.org/10.1145/3131085.3131097
[139]
Zhong-Qiu Zhao, Peng Zheng, Shou-Tao Xu, and Xindong Wu. 2019. Object Detection With Deep Learning: A Review. IEEE Transactions on Neural Networks and Learning Systems 30, 11 (Nov. 2019), 3212–3232. https://doi.org/10.1109/TNNLS.2018.2876865
[140]
Patrick Zimmer, Benjamin Buttlar, Georg Halbeisen, Eva Walther, and Gregor Domes. 2019. Virtually Stressed? A Refined Virtual Reality Adaptation of the Trier Social Stress Test (TSST) Induces Robust Endocrine Responses. Psychoneuroendocrinology 101 (March 2019), 186–192. https://doi.org/10.1016/j.psyneuen.2018.11.010

Cited By

View all
  • (2024)The Impact of Near-Future Mixed Reality Contact Lenses on Users' Lives via an Immersive Speculative Enactment and Focus GroupsProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682099(1-13)Online publication date: 7-Oct-2024
  • (2024)Augmented Reality on the Move: A Systematic Literature Review for Vulnerable Road UsersProceedings of the ACM on Human-Computer Interaction10.1145/36764908:MHCI(1-30)Online publication date: 24-Sep-2024
  • (2024)What Makes XR Dark? Examining Emerging Dark Patterns in Augmented and Virtual Reality through Expert Co-DesignACM Transactions on Computer-Human Interaction10.1145/366034031:3(1-39)Online publication date: 22-Apr-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
April 2023
14911 pages
ISBN:9781450394215
DOI:10.1145/3544548
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2023

Permissions

Request permissions for this article.

Check for updates

Badges

  • Honorable Mention

Author Tags

  1. Augmented Reality
  2. Extended Reality
  3. Perceptual Manipulations
  4. Speculative Design
  5. Virtual Reality
  6. XR Memory Manipulations

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

CHI '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1,071
  • Downloads (Last 6 weeks)182
Reflects downloads up to 19 Oct 2024

Other Metrics

Citations

Cited By

View all
  • (2024)The Impact of Near-Future Mixed Reality Contact Lenses on Users' Lives via an Immersive Speculative Enactment and Focus GroupsProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682099(1-13)Online publication date: 7-Oct-2024
  • (2024)Augmented Reality on the Move: A Systematic Literature Review for Vulnerable Road UsersProceedings of the ACM on Human-Computer Interaction10.1145/36764908:MHCI(1-30)Online publication date: 24-Sep-2024
  • (2024)What Makes XR Dark? Examining Emerging Dark Patterns in Augmented and Virtual Reality through Expert Co-DesignACM Transactions on Computer-Human Interaction10.1145/366034031:3(1-39)Online publication date: 22-Apr-2024
  • (2024)Augmented Object Intelligence with XR-ObjectsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676379(1-15)Online publication date: 13-Oct-2024
  • (2024)Toward a New Framework for XR-Adaptive Ethical Design: a State of the Art: Vers un nouveau modèle pour une conception de technologie XR-adaptative éthique : un état de l’artProceedings of the 35th International Francophone Conference on Human-Computer Interaction10.1145/3650104.3652904(1-8)Online publication date: 25-Mar-2024
  • (2024)A Critical Review of Virtual and Extended Reality Immersive Police Training: Application Areas, Benefits & VulnerabilitiesProceedings of the 30th ACM Symposium on Virtual Reality Software and Technology10.1145/3641825.3687707(1-21)Online publication date: 9-Oct-2024
  • (2024)Novel Approaches for Understanding and Mitigating Emerging New Harms in Immersive and Embodied Virtual Spaces: A Workshop at CHI 2024Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636288(1-7)Online publication date: 11-May-2024
  • (2024)Using the Visual Language of Comics to Alter Sensations in Augmented RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642351(1-17)Online publication date: 11-May-2024
  • (2024)Was it Real or Virtual? Confirming the Occurrence and Explaining Causes of Memory Source Confusion between Reality and Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641992(1-17)Online publication date: 11-May-2024
  • (2024)Enhancing Smishing Detection in AR Environments: Cross-Device Solutions for Seamless Reality2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00108(565-572)Online publication date: 16-Mar-2024
  • Show More Cited By

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media