Magic and Storytelling is an interactive presentation about the history of storytelling augmented with virtual alchemy in real-time. It was produced and created by New York-based cyber illusionist Marco Tempest in collaboration with onformative and checksum5. When Marco asked us to work on the art direction for Magic and Storytelling we were fascinated by the challenge of combining not only several new technologies, but also giving the whole presentation a homogenous and magical look. In order to achieve this, we developed a variety of different particle systems.
A combination of three different systems were used to compose the visual effects of each scene. The ‘magic global dust’, based on and controlled by the Kinect depth image, the ‘magic hand dust’, for which we placed particle emitters via hand-tracking onto Marcos hands to emphasize his gestures, and finally the ‘custom magic dust’, which allowed us to visualize very different elements, for example the appearance of party guests, typography or flying fireballs and bullets. The main task was to bring together a long list of demanding interactive effects in real-time – such as hand-tracking, face-tracking, face substitution, particle systems, elements in 3-dimensional space – within the course of exactly six minutes, which was timed exactly by the given soundtrack.
The aim in Magic and Storytelling was to find a magical and engaging way to enhance a live performance with interactive graphics. The live component could be anything from a lecture to a performance. The augmentation layer is not pre-rendered. It is generated by a data flow system in real time and in reaction to the live elements. I think that this type of magic can help us to map out what that part of the future might feel like, where 3D projection and gestural sensing are intelligent and ubiquitous. The collaboration with onformative and checksum5 created the perfect synergies to explore my vision. vvvv proved to be the ideal tool to develop these graphics and computational intensive app. Magic and illusion are a good sandbox for creativity. Adding the element of illusion to an existing technology allows us to create things that are otherwise not possible. An illusion takes a technology beyond its natural boundaries. Magic is a unique tool for pre-visualization.
By that I mean the illusion created is so convincing that the experience is almost indistinguishable from the reality. In time, the advances in technology will turn the illusion into reality. But for now the illusion gives us a very good idea of what that technology might accomplish. Looking to the very near future, when every laptop or device has a 3D sensor built into it, imagine the entertainment and educational possibilities of a system like this, designed for face-to-face communication. Psychology and cognition, chemical and engineering, design and computer. These are all disciplines that are employed in magic. They are the tools of illusion but they are not liberally interfaced with other fields of research and that strikes me as a shame. For me the most progress is going to be made by embracing current and future technologies in the best way possible, by sharing and collaborating, and inventing new and seemingly impossible things.
