Is it possible to experience being another life form?

TreeSense is a sensory VR system that transforms a person into a tree, from a seedling to its full-size form, to its final destiny. The person experiences what it feels like to be a tree by seeing and feeling her arms turning into branches and her body into a trunk. To evoke these sensations, we used Electronic Muscle Stimulation (EMS) on the user’s forearms to stimulate muscles and the skin, so that she can feel branches growing, a worm crawling, or a bird landing on her arm. This intimate, visceral experience dramatically creates an illusion of being a different life form, and thus develops a personal, immediate identification with a need of environmental protection. This project was created collaboratively with Xin LIU in Fluid Interfaces Group at MIT Media Lab. 

 

TIME: 10 Weeks ( Sept - Present, 2016 )

SCHOOL: MIT Media Lab, Fluid Interfaces Group

TEAM:  Xin LIU

FOCUS: VR, Tactile Feedback, Body Ownership Illusion

SKILLS: VR Development (Unity3D, Cinema4D, LeapMotion), Hardware Prototype (Electronic Muscle Stimulation, Arduino)


Extended Project: VR Film <TREE>

Collaborating with Artist Milica Zec and Winslow Porter, we are presenting the short VR film <Tree> at the Sundance Film Festival and TriBeCa Film Festival in 2017.

My main contribution: 1) created the first Unity3D demo with motion tracking and EMS haptic; 2) designed, prototyped and produced the vibration haptic sleeves on the forearm for final Sundance showcase; 3) supported Xin in the haptic feedback system

More information, check my TREE Project


Design


Embodied Storytelling

In its early days of development, film was a cheaper way to provide theater performance to the masses. It took several decades for filmmakers to discover the capacity of manipulating and distorting time and space. New techniques, such the close-up shot, perspective changes and dream-time, were developed to dramatically create a new sense of continuity and simultaneity. Film has now become a new way to tell stories that are not achievable in traditional theater.

Now with Virtual Reality, we have the possibility of creating experiences that are much more participatory and immersive. What if we could let audiences directly experience empathy for problems that are outside their daily life? In other words, what if we could shift the perspective of audiences and let them be the center to live the story and feel the experience firsthand? Would that embodied experience enable audiences to relate to remote problems emotionally and immediately?

The beauty of allowing people to be part of a story is that it makes an experience unique each person each time. In experimental theater, for example, artists blur the boundary of the reality and the play and morph viewers into actors to create the unsettling illusion of “what if I were ... ”. But the embodiment we want to achieve with TreeSense is much more visceral and immersive.

Unlike conventional storytelling mediums, such as films, that are limited to only sight and hearing, we tap into more senses to set up the mood for a immediate mindset change. The body image and body schema inside our brains are plastic and variable. Through a systematic alteration of sensory stimuli, such as vision, touch, motor control and proprioception, the brain can inhabit a body dramatically different from ours, such as a tree. And this body illusion leads to a more direct, personal and emotional connection with the new identity that we embodied.


Research

Body Ownership Illusion How can we really elicit the illusion of owning and being inside the character body in VR? Body ownership illusion (BOI) is key for building empathy and developing intuitive control of the avatar in the project. 

Body Ownership Illusion

How can we really elicit the illusion of owning and being inside the character body in VR? Body ownership illusion (BOI) is key for building empathy and developing intuitive control of the avatar in the project. 

Homuncular Flexibility To what extend can human accept body distortion and obtain a non-humanoid avatar body? People usually think the body image inside the brain is static. But many phenomenon such as phantom limb and rubber hand illusion prove the existence of this neural plasticity in nature. Lanier (2006) postulates the concept of Homuncular Flexibility that the homunculus—an approximate mapping of the human body in the cortex—is capable of inhabiting novel bodies and control them.

Homuncular Flexibility

To what extend can human accept body distortion and obtain a non-humanoid avatar body? People usually think the body image inside the brain is static. But many phenomenon such as phantom limb and rubber hand illusion prove the existence of this neural plasticity in nature. Lanier (2006) postulates the concept of Homuncular Flexibility that the homunculus—an approximate mapping of the human body in the cortex—is capable of inhabiting novel bodies and control them.

Electronic Muscle Stimulation as Visuo-tactile Feedback

In this project, we will seek to achieve BOI over non-humanoid avatar body through various tactile sensation by electronic muscle stimulation (EMS). Previous work in VR has found out that when a first person perspective(FPP) is provided over an avatar body that is not highly realistic, sole visuoproprioceptive cues are no longer sufficient. Congruent visuotactile cues are critical for eliciting a vivid ownership. In this project, we will investigate electronic muscle stimulation (EMS) as the main tactile technology and explore multiple EMS sensations such as on-skin sensation, muscle sensation, muscle control, etc, for various scenarios.