Constellations is a collaboration with Open Stage Theater and the Discovery Museum. The play, written by Nick Payne, is performed by actors in the Discovery Museum Digital Dome Theater. Full dome animation accompanies the actors as they explore ideas of theoretical physics through the lens of a love story.
Check out this great review from SALT magazine.
Software: Adobe After Effects, Trapcode Suite, Maxon Cinema 4D, Resolume Avenue
Scroll down to see the process from initial sketch thumbnails all the way to final video mixing and live performance cues.
COMING SOON. I'm planning to cut together a highlight reel of behind the scenes and audience footage, once we get to resume the remaining shows that were postponed due to COVID-19.
Working with animation and live action is tricky. We wanted to pre-design the entire show and pre-render animation while still allowing the acted scene animation to be colorized at the dome so that we could work together with the director to ensure her vision was met. This required pre-rendered looping scene clips with high contrast black and white that could be later colorized in the VJ software. I also created a series of 30 second interstitial animations to be played in between scenes. Unlike the scene animations which are intended to take a back seat to the actors, the interstitials are meant to be the focus of the audience. During these insterstitials is when the actors moved props, changed costumes, or changed positions on the darkened stage. Animated scene clips were pre-rendered one of two ways:
Rendered out of After Effects using the VR Converter
Rendered out of Cinema 4D using the spherical camera
After rendering, animated clips were converted in Alley for use with Resolume. A resolume master file was created to layer, colorized, and add cues to the animation so that it could be pre-designed yet live cued with QLab based on the timing of the actor performances each night. In some instances, the spherical render positions were animated directly in resolume allowing us to take the audience on a journey through the spherical animated world. Below are some partial clips of those used in the show.
Working with animation in the dome is pretty different than the standard 2D animation. Some key findings:
Being "in" the animation at the dome makes it feel very fast. Most clips need to be rendered much slower.
Like AR/VR, you need to be very careful with fast movement to avoid causing motion sickness in the audience.
While 4K looks amazing, it slowed the servers in the dome and we had to downrez much of the hemispherical renders to 2K for performance.
The front of the dome is the bottom of the video.
If the video looks right on a flat screen, it will look warped in the dome and vice versa.
This might be me being picky, but the animation controls in resolume leave a lot to be desired!
Color plays a significant role in the visuals. For flexibility, we will separate the colors from the 3D imagery, allowing us to mix and layer in the projection software (Resolume Avenue) as we work to develop and fine tune the visuals. The goal is to develop a palette that is both expansive and cohesive. Layering of colors will enhance the meaning for the audience and reflect the actors' emotive performance.
I have established three color sets to allow flexibility. The traditional 5 colors palette, the larger 16 colors palette, and an even larger 32; though we will likely be fine with 16 or less. Palettes were inspired by the Hubble image below.
To maintain flexibility, 3D will be rendered with a white texture, lighting, ambient occlusion, alpha channel (transparency), etc. so that we have quality 3D videos with good contrast. We can then overlay colors in the video mixing application and pre-program these color/video combo sequences to be cued with the live show.
The following image is collected reference (Pinterest, etc.) with processed color overlays to preview what this will look like both in renders (white) and single color layered sequences. Note we may also combine alpha matte layers for multiple color layers in a single sequence, for example one foreground color/layer, different middle ground color/layer, different background color/layer as long as we create renders with alpha channels for each area where we anticipate this need.
Director Natalie Scarlett divided the script into thematic sections. I expanded on those sections to explore an aesthetic directive for each dome animation.
Key: Layered Animations in BLUE, Interstitials in GREEN, X Room in PURPLE.
After the initial planning it's time for... sketching! I'm a lover of good old fashioned ink on paper. I love to work in After Effects, Animate, Maya, and Cinema4D; but on every project my first step is "quick and dirty" little thumbnail sketches. There's no faster way to get my ideas on paper. I can spend 15-60 minutes and come up with several pages full of thumbnails trying out many different ideas. Jumping in to the software, it might take that long to try out just one single idea.
From my rough thumbnails, I created some key frames in rough black and white to nail down the content and vale range. This step can help to communicate the vision without putting in too much time on detail and color. Note many frames are still in progress.
While this was a high stress project due to the large amount of animation required from one animator, it was a really great experience. As a team, we all worked well together and managed to pull off everything we set out to do. Every single member of the production and the theater company does what they do because they love what they do. It's hard to find this level of dedication in the commercial world. Working with Open Stage has been a gift and I am ever so grateful to have had the opportunity to work with such tremendous professionals.
This project could never have happened without this fabulous team of professionals. I am so grateful to have had the chance work with such a dedicated and highly skilled team.
Of course, we can't forget the talented stars of the show:
Ariel Greenspoon as Marianne
Keegan Bockhorst as Roland
And our fabulous production advisors: Sydney Parks Smith, Denise Burson Freestone, and Bruce K. Freestone.