What if you could inhabit The Swan Lake?

What would you see?

What would you feel?

“A Swan’s Dream” is a gaze interaction-based immersive narrative experience I developed as part of my thesis research in user engagement in virtual reality narrative experiences.

It reimagines Tchaikovsky’s world famous ballet “The Swan Lake” from the perspective of an anonymous being of the lake .Unlike the original ballet which centres around the character of Prince Siegfried, this focuses on Odette, the female lead, and it’s told from a newfound point of view.

The 15 minute experience was developed entirely by myself through the entire pipeline: Narration development, AI voicing, 3D modelling, motion capture, VFX, and both coding and programming

This short video of “The Entrance of the Black Swan” in Act III showcases all of these elements working in synch:

3D modelling

  • Ballerina's hair using NURB curves

  • Odette and Little Swan's headpiece

  • Odette and Odile's tutu

  • Odile's headpiece

  • Tutu's rhinestones

    Rhinestones in the tutu

Motion capture

Using the Neuron Perception suit and Axis Studio software, each choreography was divided into 20 second sections that were later put together in the Unity animator.

Entrance of the Little Swans

White Swan variation

VFX

My favorite part of the process was creating the visual effects that truly made the experience come alive and put forwards the world-making capabilities of VR. With that in mind, I introduced effects in the Oculus controller that the users can trigger at any point of the experience.

I used two main tools within Unity for this:

Disintegration of Odette into feathers

VFX graph

Particle systems

A key aspect to make VFX look stylized and believable is to have an animated colour Alpha over the lifetime of the particles. When the particles begin with a complete transparency and end with the same property, it creates a seamless effect which works towards maintaining the suspension of disbelief.

This was particularly relevant for the smoke-like particles as it behaves in that specific way, but it was also necessary with the other VFX available to the user.

Fog effect in the A button

Using the Skinned Mesh Renderer to bind the transform to the models achieves the effect of a trail that follows their movements. The dramatic effect was achieved using a turbulence effect, while a downward curve for multiplication and life allowed for the soft fading of the particles.

The same principle was used for Odette’s disintegration into feathers, as well as Prince Siegfried’s lightning bolts, albeit with modifications in spawn rate, turbulence and different use of the transforms.

Movement trails

Lightning bolt trails

Antagonist’s smoke effect

Lightning bolt effect in Index Trigger Button

Sparkle rain effect in the B button

Interaction logic

The story progressing interactions, which were triggered by gazing at magic circles, worked by the interactions of four scripts:

A “Gaze Interactor” script was attached to the centre eye of the OVR camera, which spawns a raycast.  When the ray cast hits something that has the “Gaze Interactable” script attached to it, it triggers the “On Gaze” function on it.

 “Gaze Interactable” is an abstract class which was used as a base for all of the magic circles. Once it was hit, it triggered a function on a “Lerper” script, which softly lerps the colour of the material in the magic circle to completely transparent.

 “Gaze Interactable” is an abstract class which was used as a base for all of the magic circles. Once it was hit, it triggered a function on a “Lerper” script, which softly lerps the colour of the material in the magic circle to completely transparent.

 Once this function is triggered, a specific flag is set to true in the “Sequencer” script, which is in charge of creating a succession of events through a centralized coroutine, which relies heavily on the “Wait Until” function, to properly time the events. The Sequencer played music, narrations, triggered animations, and overall managed the flow of the story.

Anterior

An afternoon