SinYu Deng, School of Music, Theatre & Dance
Collaborators: Stephen Rush, Professor, SMTD; Malcolm Tulip, Associate Professor, SMTD; Sile O’Modhrain, Associate Professor, SMTD, UMSI; Mia Brooks, SMTD; Stella Lansill, SMTD
Bare Display is proposed to be a 10-15 minute interactive audiovisual dance performance in the Brehm Technology Suite in the middle of April. Two dancers, outfitted in apparel embedded with mini IMU sensors, perform within human-size transparent exhibit displays.
Downward projections from two projectors map visuals onto the dancers and displays, utilizing body projection mapping techniques. Additionally, smaller exhibit displays will feature diverse visual designs presented through holographic projection.
The visual designs, facilitated by the Qualisys System, aim to create a dynamic and immersive experience. At the end of the 20th and beginning of the 21st century, the performance involved in diverse mediums became more ubiquitous due to technological inventiveness, which included the combination of music, visual art, film and digital technology.
This multidisciplinary collaboration and experimentation opened up new possibilities for audio-visual and dance arts. Technical media practitioners are now combining various interactive methods, and creating different tools and systems to make their own diverse work.
The combination of audio-visual and dance arts provide a rich and varied performance experience, allowing artists to experiment with crossover between media, and conveying richer narratives by integrating differing artistic elements. These combinations can provide freedom and expression in artistic creation and performance.
Motion capture, the technology of measuring the position and orientation of an object in physical space and recording information in a computer programming language technology, was accomplished via rotoscoping, invented by Animator Max Fleischer in 1919 in the early film industry, however, in the initial stage of the development, the motion capture technology had minimal precision.
This condition allowed only a rough range of body movements to be captured, which couldn’t wholly demonstrate the performer’s skills and the aesthetics the director wanted to present. After more than 100 years of development, motion capture technology has significantly advanced and developed into different types, such as optical and non-optical systems, including inertial, magnetic, and mechanical.
Nowadays, with the combination of technology and arts, computer software and internet technology have become a new territory for dance performance. Motion capture plays an increasingly important role in dance creation. The moving body is viewed as a creative material that requires physical exploration and can generate unexpected responses and insights.
Bare Display prominently employs motion capture technology, serving as the primary technological framework. This technology is not solely integral to dance performance, which is focused on the commodity of the body and the relationship between body and space as the conceptual crux, but it also operates at a technical level.
It aims to explore the potential of the dancer’s physical form within interactive audiovisual contexts. Additionally, it involves the creation of dance costumes intricately embedded with concealed mini-IMU (Inertial Measurement Unit) sensors.
The concept of the body as an object, victim, and subject to oppression is central to this project’s artistic vision and theoretical framework. By placing emphasis on the commodification of the body, this endeavor explores the intricate dynamics between interactive dance performances and spatial limitations.
It manifests through a confined, museum-like display platform of restricted dimensions, functioning as the stage for the dancers’ expressions. In this setup, the performing bodies are deliberately unveiled, allowing audiences to observe, scrutinize, and engage with the performance, blurring the boundaries between performer and exhibit. This method confines the dancers within a space devoid of avenues for personal expression, rendering them akin to body sculptures and artworks exhibited for viewers’ scrutiny and interpretation.
Within Bare Display, the perpetual surveillance of the performers’ bodies accentuates the project’s endeavor to shed light on the inherently political nature embedded in the act of observing and subjecting bodies to relentless scrutiny and surveillance.
In addition to artistic originality, the project will present new technological innovations in design and scientific research: a suit with multiple mini IMU sensors and an embodied approach to movement-based interaction design. After experimenting with different motion capture methods, the selection of multiple mini IMU sensors to be woven into the costumes is based on two main considerations: the mobility and adaptability of the performance, and the aesthetics of the dancers and the work.
Comparison of relevant and feasible methods: employing the Qualisys system necessitates specialized black attire, which would significantly deviate from the intended “naked” aesthetic I aim to present. Additionally, the system’s reliance on a designated space with specific cameras and systems installed poses limitations, constraining the potential extension of this research to external spaces like galleries, museums, or shopping malls in the future. The project, therefore, may only utilize this system as the pre-produced body projection mapping visual design.
The camera motion capture stands as the preferred method. However, beyond considerations of lighting and environmental factors, an additional aspect of interest pertains to the exploration of the interplay between motion capture technology and the dancer’s body.
The research exploring the analysis of movement data with a scientific basis is advancing. One of the most famous forms is Laban movement analysis (LMA). Rudolph von Laban, a founding father of expressionist dance, documented human movement, used symbols to record dances, and created the Labanotation system.
In the HCI domain, many sensor-based experience frameworks use the perspectives of user actions and corresponding system effects to help orient designers to the particular issues and concerns of both users and systems in interaction. The user’s interactions, resembling improvisational dance, equate the designer’s role to that of a “director or choreographer” guiding and choreographing the experience.
In other words, in a motion capture-based dance performance, beyond capturing the intrinsic correlation between body and movement, the integration of ancillary performance elements like audio-visual design, set arrangement, and presentation methodologies becomes pivotal.
This integration goes beyond executing artistic concepts; it entails comprehensive design, analytical scrutiny, and theoretical exploration. The project confronts the relative nature of IMU values, posing inquiries into optimal sensor placement for enhanced feedback and intricacies related to motion scheduling. These components stand as focal points of the design investigation and challenge within this endeavor.
By conducting systematic experimentation, this project seeks to construct a scientific framework that facilitates more effective utilization of wearable devices, thereby yielding real-time feedback. This will also drastically change the choreography of the video and audio-visual design.
In a nutshell, this project necessitates a suit with several wireless mini IMU sensors for the performances, transparent exhibit displays will be built of varying dimensions to accommodate the dancers, two projectors for the visual design, two high-brightness displays for hologram, and the utilization of the Qualisys system within the Brehm Technology Suite.
Additionally, it potentially involves utilizing 3D scanning, photogrammetry, or motion capture visualization studios at Duderstadt for pre-production visualization designs.
In essence, Bare Display, my thesis project, is slated for a mid-April performance, complemented by the completion of research writing in early May.
My objective is to showcase this project at prestigious technology art conferences, including the International Conference on Movement and Computing (MOCO) by January 15, 2024, the SIGGRAPH Art Gallery and Emerging Technologies Program by mid-January, NIME 2024 before January 31, 2024, and the International Symposium on Electronic Art (ISEA) in May 2024.
Additionally, I aspire to actively engage in TAICHI academic and user research-oriented conferences, along with diverse HCI-related events, furthering my explorations and contributions in this evolving field.
Overall, Bare Display symbolizes not just a performance but a convergence of interdisciplinary exploration, encompassing technology, performance art, and theoretical frameworks. This project aims to challenge conventional paradigms, offering a compelling nexus of technology and the performing arts while contributing fresh perspectives to the academic dialogue surrounding movement, space, and the human form within the realm of performance art.