3D Post-production with Mistika
After becoming the first stereographic production to win a BAFTA, the documentary Flying Monsters 3D with David Attenborough has also received the Special Prize at the IBC2011 Innovation Awards for the application of new technological developments in post-production, carried out by Onsight using the Mistika programme (creation of the Spanish company SGO). Cameraman travelled to the London offices of Onsight and had an interesting conversation with Rik Tester (Chief Engineer of the British company) and Javier Moreno (SGO´s Technical Director) about the successful project.
An investment in the spectacular
Produced by Atlantic Productions for television (Sky 3D channel) and IMAX, the film recreates the history of pterosaurs, real flying giants who circled the skies 220 million years ago, when dinosaurs ruled the earth. The producer decided to choose Onsight, one of the pioneers in the use of 3D in the UK: "Atlantic and David Attenborough wanted to show the prehistoric world in three dimensions," says Rik Tester, "using a mixture of live action and CGI, and came to us for the entire production process, so we provided the camera material, technicians specialised in stereoscopics, workflow design, cutting-edge management, image processing, online corrections... all the 3D services, from the shoot to the creation of the final master."
The documentary was filmed with two Red One cameras mounted on an Element Technica Quasar™ 3D Rig, with a beam-splitter configuration. "Atlantic chose Red One from the options we presented them, because at that time [the shooting began in March 2010] it was the model that offered the highest resolution (4K), which was key for screening at IMAX cinemas" explained Tester. For the big screen version, the 16:9 image from camera was cropped to fit the 4:3 ratio, the sequences created entirely by CGI were worked from the beginning in 4:3 (scaling up to 16:9 for television), in order to maintain the highest resolution possible in IMAX.
To deal with such an ambitious and complex project, once again Onsight entrusted SGO and its powerful software Mistika. "We are very pleased about our collaboration with SGO" says the English engineer, "thanks to both its excellent technology (Mistika is currently unmatched, it is undoubtedly the best available post-production platform) and the excellent technical skills of its workforce and their high quality standards." Besides the usual support, SGO provided new Mistika developments tailored to the project: "For this work the first technology tool, Mistika Equalize, was developed" said Javier Moreno, "the pixel by pixel Match Colour, to repair discrepancies between left and right eyes, which can cause dizziness and headaches for the viewer; subsequently new functions within Equalize were added to achieve automated correction of differences both geometric (position, rotation and scale, keystoning ) and colour, the stereographer then deals with more creative adjustments, but these automatic touch-ups greatly streamline the whole process. We have also developed a tool to correct the filage differences between the two cameras and apply this motion blur to the CGI images, and hence obtain coherent integration; all in real time (this was previously only possible in still shots)."
One of the technical challenges presented by the documentary was the sheer volume of data involved when working with 4K stereo (twenty times the data of a single HD channel). "Onsight asked us to extend its storage," continued Javier, "we went from using two machines to five. We used a complex infrastructure, render farm (a cluster of several machines for performing various tasks at the same time, or for carrying out the same task more powerfully), which also had to be updated while working with it. We use redundant arrays of hard disks (over 100), which at a given time switched to the new system; it was all very fast (there were only seven seconds of downtime), something that the client always appreciates."
Another of the challenges Onsight faced was the handling of the Red One RAW format, due to its complex decoding and the large amount of metadata it contains. "The Red format is not easily editable," says Tester, "it does not offer a pixeled image, and for visual effect all data must be stored, everything has to be catalogued, copied to multiple locations, verified, checked in QC and 3D, be prepared for editing offline, special care must be taken with file names to keep track of them throughout the process and conduct incident-free conforming."
Rik Tester also mentioned, the integration of the large number of visual effects, as one of the difficulties of this project: "Before the online work with the effects might actually start, we had to send what we called "plate effects" that were actual image plates to which we had to add geometric corrections to export them to the visual equipment so that there was barely any loss in quality. It was also necessary to utilize the Mistika composition functions to match the images captured by both cameras (for example, if brightness had been recorded on one of the "eyes" and not in the other, it was easier to copy it onto both than to try to remove it). To speed up the process, we had multiple networked Mistika systems, which strengthened collaboration and enabled file sharing to conduct the online work between multiple machines; we were able to concentrate the creative work that formed our "base camp" on the Mistika, while performing the "hard work" on effects and stereo corrections on other machines." Javier Moreno was keen to highlight the importance of the HP Z800 workstations on achieving the efficiency so highly praised by their customers: "We have been collaborating with HP for about 4-5 years, it´s a company with extensive experience in high performance computing (HPC), which also has an excellent technical service, so essential for a small company like SGO, as it allows us to operate throughout the world and be able to meet the maximum period of 4 hours required by the production and post-production companies for the repair/replacement of equipment. HP workstations combine massive scalability at high speed, they are also based on the Linux operating system that lets us make changes, does not enslave us and supports all types of codecs."
The complexity of all these challenges was increased by the tight crossed deadlines to which Onsight had committed. "The premiere on Sky3D was scheduled for December," says Rik, "and there were numerous vendors involved in the project, so we relied, for example, on receiving the latest versions of the complete visual effects, and each of these processes takes time, but Mistika, integrated with HP technology was reliable, fast, never leaving us in the lurch and allowing us to have everything finished on schedule."
Step by step
In addition to assuming the technical supervision of the film´s post-production process, Rik Tester was commissioned to design the workflow to be followed:
-"RAW images captured by the Red One in different locations around the world were sent to our internal data laboratory, after making multiple backups, the material was classified, resulting in a database that allowed all access to the information to be registered.
-Once this was done, files were prepared for offline and we received feedback on what plates we had to correct and export so that visual effects were able to be worked online.
-When editing was complete, we were sent the final assembly as a sequence, we then would gather the relevant material from the Network and pass it onto the main machines along with the visual effect we had received to begin the online.
-We carried out the conforming. Mistika can conform from the configuration that it has memorized, so that this phase benefits from the direct access to the RAW original of the camera files (material from shooting must be unzipped, but then work is conducted without compression) and the corrections carried out before and after the start of the editing session.
-Once the conforming has been carried out, two "technical screenings," take place, one geometric and the other colour, to correct the discrepancies between the right eye and the left eye. When shooting with a beam-splitter rig, one of the cameras has a slightly different exposure due to loss of light because it is reflected from the mirror, and there will also be some colorimetric difference, because the glass has a slight coloration. It's very subtle, but in 3D great care must be taken to provide a comfortable experience.
-With the colour-match finished, grading is then carried out and the most creative depth adjustments."
The traditional workflow (camera material- offline EDL-post-production) is thus obsolete when working on stereographic projects, as Javier Moreno explained: "If you keep the same workflow, the editors would make creative decisions based on images they are not viewing correctly, some post-production is required before the offline, an initial basic adjustment. Mistika enables this process to be started in parallel with the offline edition, because all the VFX, colour correction and stereo adjustments can be re-conformed from the EDLs automatically. Previously, many of these tasks had to wait until the offline EDL was complete, which meant tedious manual work. This new workflow offers much more interactivity and feedback. For example, it is customary to introduce smooth depth transitions when switching shots where the point of convergence changes, but if the stereographer does not know what the editing is, they will not be able to have them ready... In addition to loading the new changes automatically, Mistika avoids any loss of information during the process, as rendering is not required, and the camera settings may even be changed a posteriori."
Widespread application of stereoscopy?
Although, today, filming in 3D is much more cumbersome, slow and expensive than 2D filming or recordings, it seems certain to provide "something more" to audiovisual projects; as with any film, it will be the makers who must weigh up in every situation, whether it use is worthwhile or not. In this case, Rik Tester is quite clear: "Stereoscopy has allowed us to observe the vanished world of pterosaurs in a way never seen before, 3D technology makes a huge difference compared to 2D, the same way as colour does compared to black and white. This is especially noteworthy in documentaries, because stereo makes it easier to understand the subject matter. Of course, I think there are many projects that will never really benefit from 3D, such as news or other creations that require a very small team, but I cannot imagine any reason why not to use this technology to add something extra to most productions, in the same way that today very few people choose to shoot in black and white instead of colour. “Adrienne Desmeules, Onsight marketing manager, wanted to add that "for us, it is not just about using 3D, but offering high quality 3D, always offering cutting-edge technology that meets the needs of our customers. Onsight provides materials, technical personnel and facilities which helps you centralize the whole process in a single company (something unusual, at least in the UK), which is more convenient and economical for all concerned." "This is to ensure that projects are carried out according to stringent quality standards," qualifies Tester, "to safeguard the customer technically with regards any difficulty that we may experience."
IBC Special Award 2011
Because of the importance which both SGO and Onsight attach to the application of advanced technologies, the teams from both companies are especially pleased and proud about receiving the award in the IBC Innovation Awards this year. "The award is given to a project that has employed a new technical development, not the new technology itself," explains Javier Moreno. "It is awarded to both a production company and a technological innovation company." The technical director at SGO also shared with us that he considers this to have been the most important technical innovations for the audiovisual industry in recent times: "Included in this list are the Alexa Arri and Red Epic cameras (very light), because they represent a quantum leap over earlier models due to their resolution, latitude and colour reproduction; stereoscopy, which has required changes in the workflow, resulting in a more interactive process, and the new speed standard which filmmakers such as Peter Jackson, have attempted to establish at 48fps, to reduce or eliminate motion blur that causes so many headaches in stereographic post-production, and achieves a enhanced definition of the moving image . The truth is that Peter Jackson looks set to mark a before and after in the history of cinema with The Hobbit, at least from a technical standpoint, with 15 pairs of Red Epic cameras rolling continuously during each day. Of course, this is a pile of material to be processed every day!" Jackson was also sure that he was going to use Mistika in his last film, so we hope to give more details on such a mammoth shoot in forthcoming issues of Cameraman.
Script: David Attenborough
Director: Matthew Dyas
Director of photography: Tim Cragg
Artistic Director: Humphrey Bangham
Editing: Peter Miller
Colour grading: Andrew Lee, Dado Valentic
Music: Joel Douek
Sound: George Hapig, Andrew Motti, Adam Prescod, John Rogerson
Production: Anthony Geffen, Chris Howard, Celia Taylor, Martin Williams, Sias Wilson
Cameraman magazine October 2011
(Translated from Spanish)