Point Grey Grasshopper cameras used for photorealistic 3D video capture
Everyone knows it took far more than a day to build Rome. When director Fabrice Hourlier of Indigènes Productions, a company specializing in 3D computer graphics, special effects and green screen filming, set out to bring a series of epic historical battles to European television screens, he knew that this ambitious project would require a lot of careful shooting and attention to visual details. For the film production of “The Destiny of Rome,” Indigènes Productions teamed up with co-producer Docside Productions to recreate Roman cities and battle scenes with up to 200,000 soldiers engaged in battle. To achieve unprecedented visual realism, Indigènes Productions partnered with 4D View Solutions, a 3D video capture technology startup located in a hi-tech valley of the French Alps near Grenoble. 4D View Solutions built a custom 16-camera HD capture rig to film hundreds of sequences of Roman & Egyptian soldiers and citizens in 3D. The integration of these 3D sequences into the film allowed Indigenes Productions to combine the realism of actors with detailed CGI environments and virtual camera tools.
Shooting The Story
Shot primarily in green screen studios, “The Destiny of Rome” combines classic two-dimensional HD shots of primary actors with 3D video capture of supporting characters. All of these elements are carefully integrated into CGI environments that recreate the period's cities, interiors and battlefields in incredible detail. The final images are the result of a complex multi-layered compositing process. Getting stunning final images for “The Destiny of Rome” depended on being able to integrate each layer successfully with the others, and on how well each element related to the virtual CGI camera. For this project, Hourlier wanted to combine photorealistic 3D Video capture of the actors with the CGI environment tools in order to get the most out of virtual camera movements in the exterior scenes. He turned to 4D View Solutions to provide more than 260 3D sequences that helped achieve his goal.
A Flexible 3D Video Capture Implementation
4D Views’ capture solutions are implemented using a layered architecture principle. The first layer consists of managing synchronized 2D multi-camera video recording with anywhere from 2 to 80 cameras from a centralized control point. A second layer can be added to the first providing step-by-step offline 3D Video sequence production. The third layer is an optimized real-time architecture that allows real-time 3D video sequence output streaming. "This project really benefited from all three complementary layers", said Richard Broadbridge, President of 4D View Solutions. "The base for any project is always ensuring reliable, high-quality, multi-camera 2D capture. Given the project's HD requirements, we knew that accurate, quality images were the key to success. We constructed a custom sixteen-camera rig based on the Grasshopper camera by Point Grey Research to guarantee those images. With that as a foundation, the offline suite was used to process all the actor sequences into 3D Video sequences in post-production. The real-time 3D component allowed us to give the director an instant 3D preview of the end result, helping him focus on the essential question of whether the scene would render out in the final sequence the way he had envisioned. Overall it was a very comfortable working environment. Our team focused on all the 3D capture details while the director focused on actor performances and evaluating the real-time 3D results."
How the 3D Capture Works
4D Views' system uses anywhere from 2 to 80 video cameras to film a central capture area. When creating 3D sequences, each camera is used to capture the geometry of the filmed scene or the surface texture information of the visible objects, or both. "Camera placement really has a lot to do with the quality of the 3D images you get out of the system," says Broadbridge. "For this project we knew that the actors were going to need a 360-degree freedom of movement and expression within the capture area. We chose to use all the cameras for both geometry and texture, and their placement was like a three-layered cake with four cameras on top, five on a middle level and the rest much lower down. When there are multiple actors and they are fighting with swords, you really need an even coverage to get the best possible mix of accurate body posture geometries and good facial and body expression textures."
Once in position, the cameras are calibrated to sub-pixel accuracy in a few minutes and the capture area is filmed briefly in a process known as "Background Learning." With this information stored to disk, the system is ready to make 3D Video. Actors were introduced into the capture area and were filmed by the 16 synchronized Grasshopper 2.0 MP cameras at 25 fps in sequences varying in length from three minutes per scene to single frame T-pose shots. To achieve synchronization, all 16 cameras were triggered simultaneously by an external trigger. Internally the system compares each set of 16 incoming images with the learned background information previously stored. Each image presents a unique silhouette of the actor that can be combined with the other silhouettes of the set to generate a visual hull description of the outer surface of the actor. "Once the visual hull or 3D mesh description of the actors is available we map a blend of the captured texture information back onto the mesh surface," says 4D Views' CTO Florian Geffray. "In the real-time preview we use a dynamic color assignment approach to help the director study the effect of virtual camera positioning on color and subsequent relighting possibilities. For the final post-production sequences for “The Destiny of Rome,” Fabrice wanted to go with weighted single textures for rendering simplicity."
Great Cameras for Great Images
In order to get great input images, 4D Views built their rig using sixteen 2.0 MP Grasshopper color cameras, each equipped with a Sony ICX274 CCD sensor with a 1624x1224 resolution running at 25 FPS. "We selected the Grasshopper model for its great image quality and its very competitive pricing," said Clément Ménier, Head of R&D at 4D Views. "Our clients are often research labs that are looking for reliable, high-quality image data at an affordable price. We've used Grasshopper technology repeatedly and have found it way ahead of the competitors. For this project we needed uncompressed HD images at 25 frames per second. The Grasshopper was the obvious choice."
For the first half of the “Rome” project, 4D Views' camera rig used four acquisition servers to manage the 16 cameras. For the second half of the project, to be shot in the fall, the company is planning to increase the number of cameras in the rig. "There's always a tradeoff between increased quality with more cameras and the resulting data volume," continued Clément. "Right now we're working on streamlining our data management techniques throughout the production process to give people that increased quality in a way that is easy to manage."
“The Destiny of Rome” Hits the Market
With the release this month of the trailer for “The Destiny of Rome” at France's MIPTV International Entertainment Content Market expo show in Cannes, the first part of Fabrice's dream has hit the market. The film is scheduled to be shown this fall by ARTE in both France and Germany. Given the international success of Hourlier's earlier production of “Trafalgar,” televised in over 60 countries, “The Destiny of Rome” will likely come to a television set near you in 2010 or 2011. Meanwhile, editing of this first installment is in full swing and the planning of the next shoot is gearing up. To conclude the series, Hourlier will be bringing us two more epic battles, this time focusing on the Greeks and the Persians. It will certainly be a rough ride for the losing side, but through the magic of photorealistic 3D video capture, we're sure to get a front row seat.