Red Sun | A multisensory experience chamber with VR, ambient sound and scent
May 11, 2019
Fireflies
Enchanted forest concept for AI-based embodied interaction design.
DESIGN BRIEF & CONTEXT:
The AI-company Grazper, which does 3D-visualizations using AI, needed a use case design and visual concept for their technology.
Grazper's technology is able to track up to 5 people at a time via video cameras placed around a large floor space, which their AI-system then translates to anonymized skeletal rigs showing real-time movements of users.
The company needed a showstopping interactive experience to accurately demonstrate and sell the features of their tech.
My Roles: PRODUCT DESIGN • UX DESIGN • ART DIRECTION • CONCEPT DEVELOPMENT • PRODUCTION ROADMAP incl. project pipeline in Unreal Engine, production plan for physical installation, team roles, timeline, budget, recruiting plan.
Imagine that a large, 6m wide screen in front of you
is actually a glass window to an old, peaceful forest.
On the other side of the "glass barrier"
are supernatural creatures made of shining dust,
floating in the air idly, scattered like fireflies.
Reference images for the environment on the screen
As you come nearer to the "glass barrier",
the Fireflies mimic your movements
and turn into humanoid shapes:
lifting their limbs as you do,
walking as you walk,
turning as you turn.
Reference images for the Fireflies
...But they also have "free will" and agency:
making eye-contact and smiling at you,
reacting to how quickly and slowly you move,
looking over their shoulder,
getting distracted by their environment.
Concept Art with Firefly creature
INTERACTIVE, SCREEN-BASED, PHYSICAL INSTALLATION: Based on the design brief, I created a use case design using 2 or more large screens for single or multiple users.
The screens are 6m wide, placed in a convex angle to form a corner, horizontally or vertically.
On a 6x6m floor space in front of the screens, up to 5 users at a time can interact with a real-time virtual environment running on Unreal Engine.
"Fireflies" was one of many possible visual concepts in the use case design.
View of the physical design of the installation:
2 screens placed at an angle, showing an "alternate world" with which users can interact.
Top-down view of the design showing the 2 screens.
The users are represented as dark circles
ENCHANTED AVATARS: The tracked persons are translated to on-screen avatars, which are not 1:1 representations of the users, but instead represent autonomous creatures "meeting" the users. The illusion is that the screen is actually transparent glass, and on the other side are living, supernatural beings who can “choose” to interact with the users by mimicking their movements, gestures and position. However, they can also break out of mimicry and move, express and act seemingly autonomously.
I created this live environment in Unreal Engine with Fireflies (particle effects) as an initial prototype proof:
USER FRIENDLY: Up to 5 Firefly Creatures can be generated at the same time. The visual representations are designed to be vague enough to accommodate the limitations of the tracking system, and to mask technical issues. The interaction does not require any tutorial or monitoring by staff as the creatures are triggered automatically when users walk into the floor space.
FLEXIBLE VISUAL DESIGN: The "Fireflies" concept is deliberately designed to be interchangeable with other representations of users with relative ease. Instead of fireflies, other types and styles of particle effects or even animated 3D meshes could be used. It is not necessary to use humanoids.
The digital environment around the creatures could change from a forest, to any imaginable setting: underwater, in deep space, at a particular location indoors, outdoors, completely imaginary or even a real-time video feed.
The same Unreal Engine scene without Fireflies, and different lighting to test aesthetic variation.
