Nova’s Reality: A Cinematic XR Experience

Akash Dewan, School of Information

Collaborators: Adithya Sastry, SMTD; Elvis Xiang, CoE; Renee Thomas, SI; Gokul Madathil, Ross; Tanvi Naik, LSA

“Nova’s Reality” will be a multidisciplinary, cinematic, XR experience that takes us into an alternate reality through the eyes and sounds of electronic jazz musician and technologist Nova Zaii.

It’s extremely interdisciplinary in nature. We will have a team of engineers working with visual artists to design a custom 3-dimensional environment in Unreal Engine that elevates the listening experience, an audio technician and musician handling the streams of audio inputs and how they interact with the 3-D software, art students with filmmaking backgrounds who will handle lighting and the filming of the event, and a UX design student double majoring in Cognitive Science who will be both designing the web page and researching visual strategies to draw our audience into a synesthetic flow state. And that’s all just for the XR performance. There will also be numerous artists and storytellers who come together to tell Nova’s story through a documentary-style format.

Everything revolves around graphic, visual, and musical art. I envision everyone involved with every component, no matter their specialty. We have a lot to learn from each other and I want everyone to come away from this having picked up a new skill. Because every component is intertwined, it’s imperative that we are constantly bouncing ideas off of each other and keeping each other informed on each segment to promote overall cohesion of the project. Much of this project will be in the planning and development phase. There are numerous technical logistics to work out with the MIDEN and Ted Hall in addition to the creation of custom 3-D graphics.

We aim to film the XR Performance and corresponding video components by mid-April, with early summer as a fallback for anything that we did not finish during the semester. The entire summer will likely be dedicated to post-production and web page design with the final product likely being ready by mid to late summer.

As a former U-M grad, we expect Nova’s story to have a tremendous impact on the local community here, as he was a product of the very program that many students (including one of our collaborators) are in at the moment. Beyond the campus community, Nova dug his roots in the Detroit music scene before moving back to Chicago, and much of this album and visual narrative is him paying homage to these musical roots. We hope for this project
and his story to serve as inspiration for young musicians like him to continue to push the frontier of this creative medium.