skip to main content
research-article

Bio-Sensed and Embodied Participation in Interactive Performance

Published: 20 March 2017 Publication History

Abstract

Designing for interactive performances is challenging both in terms of technology design, and of understanding the interplay between technology, narration, and audience interactions. Bio-sensors and bodily tracking technologies afford new ways for artists to engage with audiences, and for audiences to become part of the artwork. Their deployment raises a number of issues for designers of interactive performances. This paper explores such issues by presenting five design ideas for interactive performance afforded by bio-sensing and bodily tracking (i.e. Microsoft Kinect) developed during two design workshops. We use these ideas, and the related scenarios to discuss three emerging issues namely: temporality of input, autonomy and control, and visibility of input in relation to the deployment of bio-sensors and bodily tracking technologies in the context of interactive performances.

References

[1]
Marina Abramovic and Klaus Peter Biesenbach. 2010. Marina Abramovi?: The artist is present. The Museum of Modern Art.
[2]
Lars Adde, Jorunn L. Helbostad, Alexander Refsum Jensenius, Gunnar Taraldsen, and Ragnhild Støen. 2009. Using computer-based video analysis in the study of fidgety movements. Early Human Development 85, 9: 541--547.
[3]
Sarah Fdili Alaoui, Baptiste Caramiaux, Marcos Serrano, and Frédéric Bevilacqua. 2012. Movement Qualities As Interaction Modality. In Proceedings of the Designing Interactive Systems Conference (DIS '12), 761--769.
[4]
Alissa N. Antle, Milena Droumeva, and Greg Corness. 2008. Playing with the Sound Maker: Do Embodied Metaphors Help Children Learn? In Proceedings of the 7th International Conference on Interaction Design and Children (IDC '08), 178--185.
[5]
Saskia Bakker, Alissa N. Antle, and Elise Van Den Hoven. 2012. Embodied Metaphors in Tangible Interaction Design. Personal Ubiquitous Comput. 16, 4: 433--449.
[6]
Louise Barkhuus and Tobias Jørgensen. 2008. Engaging the Crowd: Studies of Audience-performer Interaction. In CHI '08 Extended Abstracts on Human Factors in Computing Systems (CHI EA '08), 2925--2930.
[7]
Louise Barkhuus and Chiara Rossitto. Acting with Technology: Rehearsing for Mixed-Media Live Performances. In Proceedings of the 34th Annual ACM Conference on Human Factors in Computing Systems (CHI'16).
[8]
Steve Benford, Martin Flintham, Adam Drozd, Rob Anastasi, Duncan Rowland, Nick Tandavanitj, Matt Adams, Ju Row-Farr, Amanda Oldroyd, and Jon Sutton. 2004. Uncle Roy All Around You: Implicating the city in a location-based performance. Proc. Advances in Computer Entertainment (ACE 2004) 21: 47.
[9]
Steve Benford, Chris Greenhalgh, Gabriella Giannachi, Brendan Walker, Joe Marshall, and Tom Rodden. 2012. Uncomfortable Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12), 2005--2014.
[10]
Kirsten Boehner, Rogério DePaula, Paul Dourish, and Phoebe Sengers. 2007. How emotion is made and measured. International Journal of Human-Computer Studies 65, 4: 275--29
[11]
Francesco Cafaro, Leilah Lyons, Raymond Kang, Josh Radinsky, Jessica Roberts, and Kristen Vogt. 2013. Framed Guessability: Using Embodied Allegories to Increase User Agreement on Gesture Sets. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction (TEI '14), 197--204.
[12]
Teresa Cerratto-Pargman, Chiara Rossitto, and Louise Barkhuus. 2014. Understanding Audience Participation in an Interactive Theater Performance. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI '14), 608--617.
[13]
Alexis Clay, Gaël Domenger, Julien Conan, Axel Domenger, and Nadine Couture. 2014. Integrating augmented reality to enhance expression, interaction amp; collaboration in live performances: A ballet dance case study. In 2014 IEEE International Symposium on Mixed and Augmented Reality - Media, Art, Social Science, Humanities and Design (ISMAR-MASH'D), 21--29.
[14]
Franco Curmi, Maria Angela Ferrario, Jen Southern, and Jon Whittle. 2013. HeartLink: Open Broadcast of Live Biometric Data to Social Networks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 1749--1758.
[15]
Scott deLahunta and Frédéric Bevilacqua. 2007. Sharing descriptions of movement. International Journal of Performance Arts and Digital Media 3, 1: 3--16.
[16]
J. P. Djajadiningrat, William W. Gaver, and Joep W. Frens. 2000. Interaction Relabelling and Extreme Characters: Methods for Exploring Aesthetic Interactions. In Proceedings of the 3rd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (DIS '00), 66--71.
[17]
Nicky Donald and Marco Gillies. 2015. Novel Dramatic and Ludic Tensions Arising from Mixed Reality Performance as Exemplified in Better Than Life. In Interactive Storytelling, Henrik Schoenau-Fog, Luis Emilio Bruni, Sandy Louchart and Sarune Baceviciute (eds.). Springer International Publishing, 297--308. Retrieved November 16, 2016 from http://link.springer.com/chapter/10.1007/978-3-319-27036-4_28
[18]
Paul Dourish. 2004. Where the action is: the foundations of embodied interaction. MIT press.
[19]
Magy Seif El-Nasr. 2007. Interaction, narrative, and drama: Creating an adaptive interactive narrative using performance arts theories. Interaction Studies 8, 2: 209--240.
[20]
Moshe Gabel, Ran Gilad-Bachrach, Erin Renshaw, and Assaf Schuster. 2012. Full body gait analysis with Kinect. In 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 1964--1967.
[21]
Kaj Grønbæk, Karen Johanne Kortbek, Claus Møller, Jesper Nielsen, and Liselott Stenfeldt. 2012. Designing Playful Interactive Installations for Urban Environments -- The SwingScape Experience. In Advances in Computer Entertainment, Anton Nijholt, Teresa Romão and Dennis Reidsma (eds.). Springer Berlin Heidelberg, 230--245. Retrieved August 9, 2016 from http://link.springer.com/chapter/10.1007/978-3-642-34292-9_16
[22]
Daniel Hernández, Gabriel Villarrubia, Alberto L. Barriuso, Álvaro Lozano, Jorge Revuelta, and Juan F. De Paz. 2016. Multi Agent Application for Chronic Patients: Monitoring and Detection of Remote Anomalous Situations. In Highlights of Practical Applications of Scalable Multi-Agent Systems. The PAAMS Collection, Javier Bajo, María José Escalona, Sylvain Giroux, Patrycja Hoffa-Dabrowska, Vicente Julián, Paulo Novais, Nayat Sánchez-Pi, Rainer Unland and Ricardo Azambuja-Silveira (eds.). Springer International Publishing, 27--36. Retrieved August 5, 2016 from http://link.springer.com/chapter/10.1007/978-3-319-39387-2_3
[23]
Noura Howell, Laura Devendorf, Rundong (Kevin) Tian, Tomás Vega Galvez, Nan-Wei Gong, Ivan Poupyrev, Eric Paulos, and Kimiko Ryokai. 2016. Biosignals As Social Cues: Ambiguity and Emotional Interpretation in Social Displays of Skin Conductance. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (DIS '16), 865--870.
[24]
Katherine Isbister and Kaho Abe. 2015. Costumes As Game Controllers: An Exploration of Wearables to Suit Social Play. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '15), 691--696.
[25]
Ahmad Khawaji, Jianlong Zhou, Fang Chen, and Nadine Marcus. 2015. Using Galvanic Skin Response (GSR) to Measure Trust and Cognitive Load in the Text-Chat Environment. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15), 1989--1994.
[26]
Nicholas D. Lane, Mashfiqui Mohammod, Mu Lin, Xiaochao Yang, Hong Lu, Shahid Ali, Afsaneh Doryab, Ethan Berke, Tanzeem Choudhury, and Andrew T. Campbell. 2011. Bewell: A smartphone application to monitor, model and promote wellbeing. In in Intl. ICST Conf. on Pervasive Computing Technologies for Healthcare.
[27]
Astrid Twenebowa Larssen, Toni Robertson, and Jenny Edwards. 2007. The Feel Dimension of Technology Interaction: Exploring Tangibles Through Movement and Touch. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI '07), 271--278.
[28]
Celine Latulipe, Erin A. Carroll, and Danielle Lottridge. 2011. Love, Hate, Arousal and Engagement: Exploring Audience Responses to Performing Arts. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11), 1845--1854.
[29]
Christine La etitia Lisetti and Fatma Nasoz. 2004. Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals. EURASIP J. Appl. Signal Process. 2004: 1672--1687.
[30]
Lian Loke and George Poonkhin Khut. 2011. Surging Verticality: An Experience of Balance. In Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '11), 237--240.
[31]
Lian Loke and Toni Robertson. 2013. Moving and Making Strange: An Embodied Approach to Movement-based Interaction Design. ACM Trans. Comput.-Hum. Interact. 20, 1: 7:1--7:25.
[32]
Elena Márquez Segura, Laia Turmo Vidal, and Asreen Rostami. 2016. Bodystorming for movement-based interaction design. Human Technology: An Interdisciplinary Journal on Humans in ICT Environments.
[33]
Elena Márquez Segura, Laia Turmo Vidal, Asreen Rostami, and Annika Waern. Embodied Sketching. In Proceedings of the 34th Annual ACM Conference on Human Factors in Computing Systems (CHI'16).
[34]
Joe Marshall. 2009. I Seek the Nerves Under Your Skin. In Proceedings of the Seventh ACM Conference on Creativity and Cognition (C&C '09), 477--478.
[35]
John McCarthy and Peter Wright. 2015. Taking {A}part. MIT Press.
[36]
W. Scott Meador, Timothy J. Rogers, Kevin O'Neal, Eric Kurt, and Carol Cunningham. 2004. Mixing Dance Realities: Collaborative Development of Live-motion Capture in a Performing Arts Environment. Comput. Entertain. 2, 2: 12--1
[37]
Helena M. Mentis and Carolina Johansson. 2013. Seeing Movement Qualities. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 3375--3384.
[38]
Helena M. Mentis, Jarmo Laaksolahti, and Kristina Höök. 2014. My Self and You: Tension in Bodily Sharing of Experience. ACM Trans. Comput.-Hum. Interact. 21, 4: 20:1--20:26. https://doi.org/10.1145/
[39]
Don Norman. 2013. The Design of Everyday Things: Revised and Expanded Edition. Basic Books, New York, New York.
[40]
Leif Oppermann, Clemens Putschli, Constantin Brosda, Oleksandr Lobunets, and Fabien Prioville. 2015. The Smartphone Project: An Augmented Dance Performance. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 2569--2572.
[41]
Alex F. Osborn and Lee Hastings Bristol. 1979. Applied Imagination: Principles and Procedures of Creative Thinking. Charles Scribner's Sons, New York.
[42]
Antti Oulasvirta, Esko Kurvinen, and Tomi Kankainen. 2003. Understanding contexts by being there: case studies in bodystorming. Personal and Ubiquitous Computing 7, 2: 125--134.
[43]
Martin Ouwerkerk, Pierre Dandine, Dennis Bolio, Rafal Kocielnik, Johanna Mercurio, Henk Huijgen, and Joyce Westerink. 2013. Wireless Multi Sensor Bracelet with Discreet Feedback. In Proceedings of the 4th Conference on Wireless Health (WH '13), 6:1--6:8.
[44]
Galen Panger. 2012. Kinect in the Kitchen: Testing Depth Camera Interactions in Practical Home Environments. In CHI '12 Extended Abstracts on Human Factors in Computing Systems (CHI EA '12), 1985--1990.
[45]
Tomas Pfister, Xiaobai Li, Guoying Zhao, and Matti Pietikäinen. 2011. Recognising spontaneous facial micro-expressions. In 2011 International Conference on Computer Vision, 1449--1456.
[46]
Michalis Raptis, Darko Kirovski, and Hugues Hoppe. 2011. Real-time Classification of Dance Gestures from Skeleton Animation. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '11), 147--156.
[47]
Omer Rashid, Ian Mullins, Paul Coulton, and Reuben Edwards. 2006. Extending Cyberspace: Location Based Games Using Cellular Phones. Comput. Entertain. 4, 1.
[48]
Stuart Reeves. 2011. Designing Interfaces in Public Settings. Springer London, London. Retrieved August 8, 2016 from http://link.springer.com/10.1007/978-0-85729-265-0
[49]
Joseph Butch Rovan, Robert Wechsler, and Frieder Weiss. 2001. Seine hohle Form: Artistic Collaboration in an Interactive Dance and Music Performance Environment. Crossings: eJournal of Art and Technology 1, 2. Retrieved November 16, 2016 from http://crossings.tcd.ie/issues/1.2/Rovan/
[50]
Thecia Schiphorst. 2005. Exhale: Breath Between Bodies. In ACM SIGGRAPH 2005 Electronic Art and Animation Catalog (SIGGRAPH '05), 62--63.
[51]
Dennis Schleicher, Peter Jones, and Oksana Kachur. 2010. Bodystorming As Embodied Designing. interactions 17, 6: 47--51.
[52]
Vygandas Šimbelis, Anders Lundström, Kristina Höök, Jordi Solsona, and Vincent Lewandowski. 2014. Metaphone: Machine Aesthetics Meets Interaction Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 1--10.
[53]
Bart Simon. 2009. Wii are Out of Control: Bodies, Game Screens and the Production of Gestural Excess. Social Science Research Network, Rochester, NY. Retrieved January 17, 2016 from http://papers.ssrn.com/abstract=1354043
[54]
Dag Svanaes. 2013. Interaction Design for and with the Lived Body: Some Implications of Merleau-ponty's Phenomenology. ACM Trans. Comput.-Hum. Interact. 20, 1: 8:1--8:30.
[55]
Melanie Swan. 2012. Sensor Mania! The Internet of Things, Wearable Computing, Objective Metrics, and the Quantified Self 2.0. Journal of Sensor and Actuator Networks 1, 3: 217--253.
[56]
Emmanuel Munguia Tapia, Stephen S. Intille, William Haskell, Kent Larson, Julie Wright, Abby King, and Robert Friedman. 2007. Real-Time Recognition of Physical Activities and Their Intensities Using Wireless Accelerometers and a Heart Rate Monitor. In 2007 11th IEEE International Symposium on Wearable Computers, 37--40.
[57]
Paul Tennent, Stuart Reeves, Steve Benford, Brendan Walker, Joe Marshall, Patrick Brundell, Rupert Meese, and Paul Harter. 2012. The Machine in the Ghost: Augmenting Broadcasting with Biodata. In CHI '12 Extended Abstracts on Human Factors in Computing Systems (CHI EA '12), 91--100.
[58]
Jakob Tholander and Stina Nylander. 2015. Snot, Sweat, Pain, Mud, and Snow: Performance and Experience in the Use of Sports Watches. 2913--2922.
[59]
Bin Umino, Jeffrey Scott Longstaff, and Asako Soga. 2009. Feasibility study for ballet e-learning: automatic composition system for ballet enchaînement with online 3D motion data archive. Research in Dance Education 10, 1: 17--32.
[60]
Jordan Beth Vincent, Kim Vincs, and J. McCormick. 2015. Splitting centre?: directing attention in trans-media dance performance. ISEA 2015?: Proceedings of the 21st International Symposium on Electronic Art: 1--1.
[61]
Chen Wang, Erik N. Geelhoed, Phil P. Stenton, and Pablo Cesar. 2014. Sensing a Live Audience. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14), 1909--1912.
[62]
Qiao Wang, Pavan Turaga, Grisha Coleman, and Todd Ingalls. 2014. SomaTech: An Exploratory Interface for Altering Movement Habits. In Proceedings of the Extended Abstracts of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI EA '14), 1765--1770.
[63]
Lisa N. Woodside, V. K. Kumar, and Ronald J. Pekala. 1997. Monotonous Percussion Drumming and Trance Postures: A Controlled Evaluation of Phenomenological Effects. Anthropology of Consciousness 8, 2--3: 69--87.
[64]
Kuan-Ju Wu, Mark D. Gross, and Mark Baskinger. 2012. Giffi: A Gift for Future Inventors. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (TEI '12), 335--336.
[65]
Yan Zhang, Lei Wang, Jing Zhu, Yadong Hu, Wanli Xing, and Jing Cheng. 2011. Real-time monitoring of extracellular matrix-mediated PC12 cell attachment and proliferation using an electronic biosensing device. Biotechnology Letters 34, 2: 397--404.
[66]
Touching a Stranger: Designing for Engaging Experience in Embodied Interaction. International Journal of Dsign. Retrieved January 17, 2016 from http://www.ijdesign.org/ojs/index.php/IJDesign/article/view/976
[67]
Thinking Through the Body | Body-focussed Interactive Art and Somatics. Retrieved January 17, 2016 from http://thinkingthroughthebody.net/

Cited By

View all
  • (2024)Developing a Multitasking Augmented Reality Application for Theatrical and Cultural ContentHeritage10.3390/heritage70701697:7(3578-3596)Online publication date: 5-Jul-2024
  • (2024)Brain Body Jockey project: Transcending Bodily Limitations in Live Performance via Human AugmentationProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675621(1-14)Online publication date: 27-Oct-2024
  • (2024)Best Practices for Technology-Mediated Audience Interaction in Dance PerformancesProceedings of the 9th International Conference on Movement and Computing10.1145/3658852.3659083(1-6)Online publication date: 30-May-2024
  • Show More Cited By

Index Terms

  1. Bio-Sensed and Embodied Participation in Interactive Performance

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    TEI '17: Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction
    March 2017
    806 pages
    ISBN:9781450346764
    DOI:10.1145/3024969
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 20 March 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. audience engagement
    2. bio-sensing
    3. biodata
    4. bodily tracking
    5. interactive performances

    Qualifiers

    • Research-article

    Conference

    TEI '17
    Sponsor:

    Acceptance Rates

    TEI '17 Paper Acceptance Rate 41 of 151 submissions, 27%;
    Overall Acceptance Rate 393 of 1,367 submissions, 29%

    Upcoming Conference

    TEI '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)43
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 23 Oct 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Developing a Multitasking Augmented Reality Application for Theatrical and Cultural ContentHeritage10.3390/heritage70701697:7(3578-3596)Online publication date: 5-Jul-2024
    • (2024)Brain Body Jockey project: Transcending Bodily Limitations in Live Performance via Human AugmentationProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675621(1-14)Online publication date: 27-Oct-2024
    • (2024)Best Practices for Technology-Mediated Audience Interaction in Dance PerformancesProceedings of the 9th International Conference on Movement and Computing10.1145/3658852.3659083(1-6)Online publication date: 30-May-2024
    • (2023)Linking Audience Physiology to ChoreographyACM Transactions on Computer-Human Interaction10.1145/355788730:1(1-32)Online publication date: 7-Mar-2023
    • (2022)Sound Feedback for Social Distance: The Case for Public Interventions during a PandemicElectronics10.3390/electronics1114215111:14(2151)Online publication date: 9-Jul-2022
    • (2022)Designing Interactive Visuals for Dance from Body Maps: Machine Learning and Composite Animation ApproachesProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533467(204-216)Online publication date: 13-Jun-2022
    • (2022)The Normal Natural Troubles of Virtual Reality in Mixed-Reality PerformancesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3502139(1-22)Online publication date: 29-Apr-2022
    • (2022)Workshops in TEI: Development, Evaluation, Exploration, and ImplementationProceedings of the Sixteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3490149.3505562(1-9)Online publication date: 13-Feb-2022
    • (2022)Effects of Augmenting Real-Time Biofeedback in An Immersive VR Performance2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00159(751-756)Online publication date: Oct-2022
    • (2021)Expanding the Design Space for Technology-Mediated Theatre ExperiencesProceedings of the 2021 ACM Designing Interactive Systems Conference10.1145/3461778.3462123(2026-2038)Online publication date: 28-Jun-2021
    • Show More Cited By

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media