Imagine trying on the latest fashions without having to undo a single button. That’s the idea behind Virtual Mirror, a series of augmented reality (AR) visualization applications developed by researchers at the Fraunhofer Heinrich Hertz Institute for Telecommunications, and recently displayed at SIGGRAPH 2009 in New Orleans. Instead of viewing yourself in a real mirror to check how you look in new clothes or new shoes, Virtual Mirror uses highly sophisticated 3D image processing techniques to visualize the look of new garments without any need to actually put them on. Virtual Mirror allows users to move freely in front of the mirror without additional aids like glasses to view themselves wearing the augmented clothes.
Tracking and Retexturing of Garments for Virtual Clothing
The Virtual Clothing application uses a Flea2 0.8 megapixel (1032 x 776 pixels) camera from Point Grey to capture images of a moving person wearing a piece of clothing, such as a t-shirt or tie. Images are streamed from the camera over IEEE 1394b (FireWire-b) to the host system, where image processing software replaces the clothing’s original texture with a correctly deformed and shaded texture and displays the augmented image of the person on an LCD monitor. Image capture, processing and display is done in real-time. The result is a combination of real video and the new augmented texture yielding a realistic impression of the virtual article. Image processing techniques estimate the elastic deformation of the garment at its main creases and folds, which allows the person to adjust the clothing without any impact to the augmented texture. Correct shading and illumination of the scene is also estimated to enhance the realistic appearance of the augmented garment even in its smaller creases and folds. “We selected the Flea2 for image capture for a number of reasons,” says Jürgen Rurainsky, Head of Image Processing Marketing at Fraunhofer. “It’s extremely affordable and very small, which are critical factors in minimizing the cost of the system as a whole.”
FIGURE 1. Virtual Mirror, shown at SIGGRAPH 2009, exchanges color and partial texture.
Virtual Mirror for Shoe Configuration
The Virtual Shoe application uses a Flea camera from Point Grey with 1024 x 768 resolution to capture the customer wearing footwear with a standard design. An LCD monitor replaces a real mirror and displays the horizontally flipped camera image. The display is mounted so that the person appears in the same position where she would expect to see herself when looking into a real mirror. To enhance the virtual feeling of the framework, the background is segmented and replaced by a synthetic environment. A novel 3D motion tracker estimates the position and orientation for each foot using a highly robust model-based approach that can easily be adapted to new shoe models. Once the exact foot position in 3D space is known, the computer graphics models, configured and colored to the customer's own wishes, are rendered and integrated in the video stream so that the real shoes are replaced by the virtual ones.
FIGURE 2. Top: Virtual Mirror installation in the adidas store, Champs Elysées, Paris.
Bottom: Upper row: camera input. Lower row: output of Virtual Mirror with personalized shoes augmented into real scene.
The Fraunhofer Institute, a global leader in the development of mobile/fixed communication networks, first used this technology in 2007 when they developed a different type of virtual mirror for the Adidas flagship store in Paris. Customers there designed their own shoes, stepped in front of the mirror and saw them superimposed on their feet. Unlike shoes, textiles have elastic qualities and their structures aren't always uniform, so that creates a challenge for the virtual mirror. “The software algorithm creates a two-dimensional model of the image that is used to predict any changes,” says Mr. Rurainsky. “The system also knows the directions in which the fabric is capable of stretching or flowing.”
The “Virtual Mirror” has been a huge draw at various trade shows in the past year, such as Siggraph and CeBIT. Click here to watch the Virtual Mirror video from Siggraph 2009.