Lemony Snicket (2004) - Special Effects [Behind the Scenes]
Feb 16, 2021 2:57:40 GMT -5
the panopticountolaf and Optimism is my Phil-osophy like this
Post by gothicarchiesfan on Feb 16, 2021 2:57:40 GMT -5
As I've mentioned previously, I recently acquired an old issue of the special effects magazine CINEFEX which contains a very detailed article about the film. I've scanned and transcribed the entire article and uploaded it here.
Both the transcribed and scanned version are presented below, albeit behind spoiler boxes because of the amount of text and high-resolution images.
ILM created a CG Sunny for gags - such as catching a spindle in her mouth – that live babies could not be expected to perform. In lieu of scanning, ILM upgraded its image-based modeling system to build the digital Sunny model, photographing the twin babies cast in the part with an array of high resolution digital video cameras, capturing geometry and textures simultaneously. From the digital footage, modelers built a library of facial expressions in motion. For final body animation, ILM motion captured toddlers performing specific actions, then employed XSI software to edit and massage the data. ILM refined its rendering tools, which included improved subsurface scattering and hair shaders, to impart a photoreal look to the digital baby.
With their parents' untimely demise, the Baudelaire orphans – Violet, Klaus and baby Sunny - are delivered into the hands of their new guardian, Count Olaf, an actor of questionable talent and motives, played by Jim Carrey. The scheming Count, out to rob the children of their inheritance, dons a series of disguises in the course of the picture. All of the makeups were designed and implemented by makeup supervisor Bill Corso, who had previously worked with Carrey on The Majestic and Bruce Almighty.
Modelmaker Tom Proost adds finishing touches to the Aunt Josephine house miniature. The model shop carefully measured a 1/48-scale maquette provided by the production art department, then input those measurements into the computer, and laser-cut components. The final miniature was fabricated at 1/8 scale, out of redwood.
Model shop supervisor Steve Gawley attends to a section of the house miniature on stage. For some scenes requiring digital imagery, crews shot stills of the miniature, which were then photomapped onto CG geometry of the structure.
Industrial Light & Magic supplied 505 shots to the show, including a wide view of the flame-ravaged Baudelaire Mansion - a composite of live footage and a digital matte painting constructed from photos of a scaled mansion model. ILM also augmented real emergency vehicles shot on a street set with digital fire trucks.
Arranging for the children's untimely deaths, Count Olaf locks them inside a car parked on a railroad crossing. Live-action elements were shot on a forced perspective stage set, which was extended digitally. ILM animated a CG train, using lattices to contort the locomotive and match it to the forced perspective of the set as it moved through frame. The same compression was applied to the CG particle smoke rising from the engine smokestack. Digital car windows and windshield, and atmospheric haze to heighten the forced-perspective illusion, were added in compositing.
Though the Uncle Monty house set was filled with real snakes and lizards, ILM animated the harmless and inappropriately named 'Incredibly Deadly Viper,' which figures prominently in the story. For a scene in which Sunny playfully wrestles with the serpent, the crew shot one of the twins on set, wrapped in a rubber snake, to capture lighting reference, then added the CG baby to a second, clean plate. In shots featuring the real baby, ILM's rotoscoping crew reconstructed plates, removing the rubber snake to make way for the 3D-animated version.
After exploring a number of Stephano looks, Silberling and Corso opted to maintain the basic Olaf facial structure, altering his appearance by darkening the wig, styling it into a comb-over, shaving the eyebrows and adding a pencil-thin mustache.
For Captain Sham, Corso exchanged the Olaf goatee with a graying beard, and the black unibrow with bushy gray eyebrows. On the set, Carrey wore a peg-leg appliance – his real leg bent behind him - which ILM finessed digitally in post-production.
Production built a huge water tank specifically for the lakeside sequence, erecting a partial house-and-cliff setting and surrounding it with a painted backdrop. ILM extended the set for wide shots, using miniatures as the basis for a digital house and cliff, and replaced the backdrop with a matte-painted view.
CG supervisor Philippe Rebours checks out a stage setup of a cliffface model carved out of urethane foam. The miniature cliff set was used for a sequence in which Aunt Josephine's house blows apart and tumbles from its clifftop perch in a hurricane. In addition to being used for the cliff-face element itself, the model was laser-scanned and duplicated in the computer to create 3D models of surrounding cliffs on the lake, which allowed Brad Silberling complete freedom in moving his camera through the environment. Still photographs of the model provided textures for the CG terrain.
"We laser-scanned that portion of the cliff,” added Rebours, "and also took plenty of photos of it and used those to apply the texture. We were able to use that model by duplicating it and cutting it, et cetera, to give us enough cliff for all the shots. We also had some still camera shots. In those cases, we took one photo of the miniature, and if we could just use that, we used that.
For shots of one room after another shearing away in the gale-force winds – leaving the children stranded atop a swaying platform – Brad Silberling and his production crew photographed the child actors against bluescreen, on partial house sets. Full-size house interiors were constructed atop a gimbal so that they could be made to shake, rattle and roll with the force of the storm.
ILM integrated the bluescreen set elements with a 3D environment made up of the cliff-face set and CG stormy waters below, created through geometry and shaders. In post, the stage platform shot against bluescreen had to be recreated digitally to accommodate a change in camera movement. Final shots in the sequence, in which the house tumbles down the cliff and into the lake, made use of breakaway house miniatures dropped from a crane.
Model builder Robbie Edwards constructs the 1/8-scale Olaf house. The model shop based the miniature on photographs from the set, architectural drawings, and painted pieces provided by art director John Dexter. Model shop crew members input the drawings into ILM's computers, then built the house miniature from the CAD files, ensuring that every feature of the replica was precisely scaled to the full-size set.
Materials used on the miniature for Olaf's house included shingles, slate, copper and brick. Crew members weathered the model with washes of paint and covered it in scaled vines. Modelmaker Randy Ottenberg applies final set dressing to the miniature.
Both the transcribed and scanned version are presented below, albeit behind spoiler boxes because of the amount of text and high-resolution images.
A Series of Persnickety Effects
Article by Jody Duncan
In the opening moments of Lemony Snicket's A Series of Unfortunate Events, the unseen narrator warns audience members that what they are about to see is not a typical happy children's story, but rather a grim tale of the orphaned Baudelaire children – teenagers Violet and Klaus, and toddler Sunny – who suffer a series of most terrible and unfortunate events as they are passed from guardian to guardian after the tragic and untimely death by fire of their wealthy and loving parents.
Screenwriter Robert Gordon translated the first three volumes of Daniel Handler's popular children's book series - supposedly penned by the fictional 'Lemony Snicket' - into a screenplay for the Paramount Pictures and DreamWorks production, which was filmed almost entirely on stages at Paramount and at the former Lockheed facility in Downey. The completed movie featured 505 effects shots created by Industrial Light & Magic, with visual effects supervisor Stefen Fangmeier helming the project.
Months prior to the start of principal photography, director Brad Silberling went to ILM with one overriding concern: the feasibility of creating the baby Sunny as a 3D-animated character for scenes in which the child would perform stunts and behaviors that no real child actor could manage. At this early stage, Silberling even toyed with the idea of doing all of the Sunny shots digitally. "If we'd done it entirely with a CG character," said Stefen Fangmeier, "we would have never had to intercut with a real baby, and that would have been an advantage. But, on the other hand, we wouldn't have had reference of a real baby to guide us. It was a tradeoff; and, ultimately, we decided to go with intercutting back-to-back with a real baby."
ILM created a CG Sunny for gags - such as catching a spindle in her mouth – that live babies could not be expected to perform. In lieu of scanning, ILM upgraded its image-based modeling system to build the digital Sunny model, photographing the twin babies cast in the part with an array of high resolution digital video cameras, capturing geometry and textures simultaneously. From the digital footage, modelers built a library of facial expressions in motion. For final body animation, ILM motion captured toddlers performing specific actions, then employed XSI software to edit and massage the data. ILM refined its rendering tools, which included improved subsurface scattering and hair shaders, to impart a photoreal look to the digital baby.
ILM began developing a test shot of a digital baby, using Silberling's two-year-old daughter, Charlotte, as the test subject. Among the advancements needed to create a photorealistic baby were more sophisticated rendering tools - specifically, improved raytracing, subsurface scattering and global illumination techniques that would simulate the smooth, translucent quality of baby skin as it would look in the lighting scheme Silberling intended to employ on his stage sets. "They were planning to use big light sources," explained research and development lead engineer Christophe Hery, who spearheaded the effort to refine ILM's rendering tools, "which would produce very diffuse and soft light. We had to be able to simulate that look in the lighting on the baby."
As the R&D team addressed rendering issues for the test, animation supervisor Colin Brady explored animation techniques. "Doing a fully CG baby was about as scary as it gets," said Brady. "Baby motion is very specific. They have a kind of wobbly motion that people recognize, and the physics of how big their heads are compared to their bodies also comes into play. Not only is the motion very specific, they have to look cute and appealing. And having to match this baby to a particular baby in the film, cutting back to back, made it even more challenging."
ILM initiated test animation by conducting a series of motion capture sessions with performers ranging from an eight-month-old baby, to Brady's three-year-old daughter, to a five-year-old, to an adult acting out the parts of the baby. The motion capture data was then applied to a digital model of Charlotte Silberling. "It was a pretty successful test," Brady stated. “It still wasn't all the way there, but we could see where we were and where there were still flaws."
Reassured by the test, Silberling and his crew launched into principal photography. With production designer Rick Heinrichs and director of photography Emmanuel Lubezki, Silberling fashioned a hyper-theatrical look for the film, surrounding stylized stage sets with painted, almost impressionistic backdrops, and shooting most of it locked-off, with wide lenses.
With their parents' untimely demise, the Baudelaire orphans – Violet, Klaus and baby Sunny - are delivered into the hands of their new guardian, Count Olaf, an actor of questionable talent and motives, played by Jim Carrey. The scheming Count, out to rob the children of their inheritance, dons a series of disguises in the course of the picture. All of the makeups were designed and implemented by makeup supervisor Bill Corso, who had previously worked with Carrey on The Majestic and Bruce Almighty.
Sunny's scenes were slated early in the shoot, with a set of triplets initially portraying the character. When the temperaments of the triplets proved ill-suited to filming, they were replaced with 18-month-old twins Kara and Shelby Hoffman. "The twins worked out very well,” recalled ILM visual effects producer Jeff Olson.
“They were amazingly cooperative and happy, and they didn't freak out and cry during the shoot." The twins were such good performers, in fact, Silberling began to realize they would be able to perform gags he had previously scheduled for the digital Sunny. As a result, the original slate of 75 to 100 digital Sunny shots was reduced to only 25. "These girls just did a phenomenal job, and that lightened our digital Sunny workload. Also, at the beginning, Brad had written more dialogue and more specific actions for Sunny to do; but in the course of the shoot, he realized they could get away with a little less of Sunny. So her part became smaller than it was originally conceived, and that meant fewer shots for us."
"Fewer shots was fine with me," added Stefen Fangmeier, "because it was going to be such a challenge to make each digital Sunny shot work. I preferred to do fewer of those moments, and do them really well."
On the set, crews shot reference plates with the real baby for all of the digital Sunny scenes, providing real-world grounding for lighting and animation. “We always had the little girls for reference,” Fangmeier said. "Even if the action was something that the real baby couldn't possibly do, I wanted her there, in the lighting, so that we could see all the details of the skin and eyes and hair. It was difficult on production, especially since the babies could only work for short periods of time. I'm sure they were thinking, 'This is going to be an effects shot anyway – why do we have to go to the effort of bringing in the baby?' But having that reference was essential for us. We always shot a clean plate, as well, so we wouldn't have to take the real baby out of shots."
In addition to capturing reference footage in all the stage setups, ILM conducted an extensive digital video shoot with the twins on a nearby stage, gathering images that would serve as the basis for the digital Sunny model, in lieu of scanning. “We couldn't scan the babies for our model,” explained Fangmeier, “because they wouldn't be able to sit still for the 20 seconds, or whatever it takes, to do the scan. So, instead, we defined the surface features of the baby by upgrading ILM's proprietary image-based modeling system to capture geometry and texture data together instantaneously. R&D engineers Colin Davidson, Max Chen and Francesco Callari improved the solvers and tools to reconstruct precise surfaces from multiple photographs. It was great because we could also get expressions, which you don't necessarily get with typical scanning. We could have her smile, for example, capture that moment, and get a very accurate representation of that in the digital model.”
The effects team had initially considered doing facial capture as a means of capturing expressions, a technique that would have involved placing hundreds of reflective dots on the babies' faces, capturing the data, and remapping it onto the digital model. Ultimately, the technique was rejected, not only because it was still in the research and development phase, but also because the babies were not as 'directable' as grown actors, which would have led to very long and often unproductive facial capture sessions. “Babies just kind of do what they do,” said Colin Brady, “and so you would have to capture hours and hours of facial performance to get everything you needed. So, instead, we developed this 'poor man's facial capture technique, lining up six high-resolution digital video cameras on the babies' faces and shooting hours of them sitting there makina different expressions, biting things, talking, crying. Through that, we were able to build a library of expressions that we could use in any of the shots."
Modelmaker Tom Proost adds finishing touches to the Aunt Josephine house miniature. The model shop carefully measured a 1/48-scale maquette provided by the production art department, then input those measurements into the computer, and laser-cut components. The final miniature was fabricated at 1/8 scale, out of redwood.
ILM stabilized the footage – essentially locking the baby's face center screen – then gave it to digital modeler Martin Murphy. “Martin was really the Most Valuable Player on the show," commented Brady. “He went through, frame by frame, and re-created these facial expressions in motion. Instead of creating generic expressions for one still frame, he delivered to us three to five seconds of continuous motion that followed the video footage exactly. These chunks of expressions in motion — the baby puckering its mouth, crying, smiling - became libraries that our animators could use as they animated the baby. They were done as Caricature files, so they were in a form the animators could understand. It enabled them to tweak things and edit things - hold a smile longer or take out a blink or whatever."
Armed with the production footage and all of the research and development that had been implemented during the test phase, ILM jumped into its work on final Sunny shots. For final body animation, the crew went back to the motion capture stage to record specific actions required in the now cut live action scenes. "I feel as if I've worked with the best animators in the world at Pixar and here at ILM,” said Brady, “but I don't think any animator would be up to animating a baby from scratch and making it convincing. So we motion captured my daughter again, and the daughter of another TD here. It wasn't practical to motion capture the twins, mainly because we were doing our motion capture up here while they were still shooting down in Los Angeles. But we felt that generic baby motion capture would work, as long as we could tweak it afterwards." To enable extensive tweaking, the animators used XSI software, which has excellent user-friendly motion capture editing tools. "You can cut-and-paste chunks of motion capture and blend them from one to another, and really massage it. Since we were dealing with young children, and we couldn't get exactly the actions we needed in one long take, we just had to capture quantity, then create the appropriate action by taking one piece from one thing and another piece from something else and mixing them together."
Rendering of the final Sunny shots took advantage of new technologies for lighting reference that ILM had shot on the set. ILM matchmovers modeled the rooms and environments Sunny would be seen in, then digital artists applied texture maps derived from stereo photography to that geometry. “From the onset data capture they were doing,” explained Christophe Hery, “the matchmovers were giving us the real placement of the lights. So we'd put these texture maps on the furniture or the walls, and we would be lighting the environment just as a DP would. The light would be absorbed by the walls or a piece of furniture and radiate back out a different color. And that colored light is what would reach the baby. We could put the character close to a wall, for example, and the wall would bleed color onto the subject - and vice versa. It gave the character a richness, interacting within its environment. It wasn't 100 percent accurate, but it was a really good approximation - probably 95 percent there. And it was still manageable in terms of rendering speeds. All of this was possible because of advancements in RenderMan's ray-tracing capabilities."
Model shop supervisor Steve Gawley attends to a section of the house miniature on stage. For some scenes requiring digital imagery, crews shot stills of the miniature, which were then photomapped onto CG geometry of the structure.
The new approach was so much closer to lighting the way a real DP would, it required some retraining on the part of the technical directors. “The TDs have always had to cheat light," said Hery, "because there were so many limitations in CG before. They always had to fake bounce light, for example. With this, we told them: 'Don't cheat anymore. Let the light bounce around and behave like it's supposed to. At first, some of them still had a tendency to cheat it. They would set lights by hand to bounce around; but then the system would add bounce, and we'd wind up with too much. It took a bit of time to retrain everybody."
Improved shaders created the fine hairs on Sunny's head and the barely discernible peach fuzz on her face. "It was a matter of so much detail,” said CG supervisor Gerald Gutschmidt. “We had to create a very specific specular reaction on the hair to get its soft sheen; but, at the same time, individual hairs had very broken-up, pointy glints. Another issue was color variation. The hair may be brown, but there are hundreds of browns! And we found that if we didn't get just the right color, the hair didn't look realistic. The amount of clumping hair versus wispy hair was another crucial factor in getting the hair to look real — especially in silhouette. Also, the shadows - how soft should we go? It took a great deal of time to find exactly the right levels. We had to keep tweaking the material to get all of those details in it. We were fortunate in that we had the time on this project to really make the hair perfect.” Technical director Jeff Hatchel took the lead in the hair look development.
By the time the project moved officially into post production, the CG Sunny shots were coming together and exceeding all expectations. “We were really ahead of the game at this point," noted Stefen Fangmeier, "because of all the work we'd done in the test phase. As it turned out, the thing we'd thought would be the most difficult — the digital baby – actually went very smoothly. And the things we hadn't thought too much about, such as 2D and 3D environments, became the bulk of our work on the show."
Sunny and her siblings are first revealed on Briny Beach near the Baudelaire Mansion, where the youngsters contentedly wile away the hours prior to learning of their parents' demise. In a series of flashbacks, the scene establishes Violet (Emily Browning) as the young and brilliant inventor, Klaus (Liam Aiken) as the bookworm, and Sunny as the 'biter' with sharp, serrated teeth. "That is where you find the most remarkable digital Sunny shots," stated Jeff Olson. “They establish that she likes to bite things, and they show her biting a wooden spindle her brother throws for her — like a dog catching a Frisbee. We really establish her character in this scene."
In the hero spindle-catching shot, Sunny jumps from right of frame, catches the object, then turns toward camera. “Her face is full-frame in the shot," said Fangmeier, “so it didn't matter how long it took to do it — that shot had to be absolutely perfect. It had to work." To gather raw animation for the shot, ILM motion captured its real child performers jumping. "I told them to pretend that they were frogs jumping up to catch flies in their mouths," Colin Brady recalled. "Once they got into that mindset, they were great. Of course, a real baby couldn't catch that spindle in its mouth; but at least the action was physically based on real kids jumping. Because of that, you buy it as a real action."
In another flashback shot, Sunny hangs from the edge of a dining table by her teeth. To capture the action, ILM built a bar on the motion capture stage, and had the child performers bite the bar as crew members held them up underneath their armpits. “It wasn't as daunting as I thought it would be," recalled Brady, "because that kind of biting came very naturally to the kids. As we held them up, we would swing them gently, which gave us some beautiful motion. Of course, in the motion capture data, you could tell that the baby was being lifted and helped a bit underneath her armpits; but we were able to tweak that and correct it through our great motion capture editing tools."
After their parents' death, the children inspect the charred remains of their home, Baudelaire Mansion, views of which were realized with a combination of live-action footage shot on a full-size street set, location photography, and matte paintings derived from a miniature. "We received the footprint drawing of a real house in Boston that Brad had picked for the location shoot,” said model shop supervisor Steve Gawley. "We took that footprint and the concept art that was given to us by Rick Heinrichs, and built a quick 1/24-scale foam model of the burned-out house that measured about six foot square and four feet tall.” The model shop delivered the foam model to digital matte painter Paul Huston, who dressed it, then photographed it, using those images to construct a matte-painted view of the charred mansion.
Huston also added fire trucks - augmenting the real emergency vehicles that had been shot on the street set and smoke.
Compositing supervisor Marshall Krasser oversaw the assembly of the live-action and matte-painted elements. For the end of the aftermath sequence, Sabre artists took the running footage of the scene, froze it and turned it black-and-white. “The camera pulls back," stated Krasser, "and you realize that Lemony Snicket (the voice of Jude Law) is actually holding a photograph of the event, as if he is telling a story. It's a very nice transition.”
Industrial Light & Magic supplied 505 shots to the show, including a wide view of the flame-ravaged Baudelaire Mansion - a composite of live footage and a digital matte painting constructed from photos of a scaled mansion model. ILM also augmented real emergency vehicles shot on a street set with digital fire trucks.
The orphaned children are transported by car to the decrepit home of Count Olaf (Jim Carrey), a distant relative who schemes to rob them of their inheritance. Traveling car shots throughout the film were photographed on a bluescreen stage, with digital environments created by ILM art director Wilson Tang and artists Dan Slavin and Jeff Grebe providing the views outside the windows. “Wilson's team created some inexpensive 3D environments for us to travel through," said Jeff Olson, "which was a big innovation for us. These car shots serve as transitions from one scene to the next. There are 20 or 30 of them with the kids in the car, being taken to their next guardian and next 'unfortunate event’.”
City views, as seen from the car windows on the drive to Count Olaf's residence, suggested a period look. “The city looks a bit like Boston or 19th-century Paris," said Wilson Tang, "but it was really different time periods all mixed together to create this fantasy world. We referred to a lot of paintings and photographs to create the look. On set there were paintings of the city on giant backdrops, and we took inspiration from those, as well. Rick Heinrichs also gave us a really good scrapbook that we used as a quide." ILM added CG traffic in the traveling shots — particularly in the early part of the drive – to establish the illusion of a populated city street.
Count Olaf makes a dramatic entrance shortly after the children's arrival. Makeup designer Bill Corso, who had worked with Jim Carrey on The Majestic and Bruce Almighty, worked with the actor to design a basic Olaf makeup that would serve as the foundation for two additional disguises the character would don in the course of the film. “All of these characters were vividly described in the books," related Corso. "And in the books, his look changes drastically from one story to the next. The problem for us in doing three books in one film was that it wasn't realistic to do such drastic changes in appearance. Count Olaf couldn't have a shaved head in one scene, for example, and then his own hair grown back in the next scene. So we came at it as if he is a kind of Mr. Potato Head - a basic canvas on which we would build these other disguises. So, first, we had to create the look of Olaf as Olaf. Then we could create the look of that same guy in disguises."
Corso began playing with looks on the computer to generate Olaf makeup designs. “Working in Photoshop,” Corso said, “I did every disguise in the book on Jim's face. Jim is actually very computer-savvy, and it so happened that he was getting into Photoshop at the time. So the two of us spent a lot of time together at the computer, experimenting with looks.” Corso then took the best of the Photoshop designs and manufactured pieces to apply to Carrey for a test several months before the start of filming. “We sat Jim in front of a mirror, and I provided him with noses and chins and eyebrows and wigs and hairpieces. For two days we just stuck different stuff on him. We finally came up with the definitive Count Olaf look, which was very menacing and creepy."
The makeup featured a large hooked nose, a prominent brow bone, heavy eyebrows, a goatee, discolored teeth veneers, and a wig, which was applied over Carrey's shaved head. Once Corso and Carrey got the hang of the routine, makeup application times came down to an hour and a half. “Jim would come in and get his head shaved, which took about 20 minutes. Then he'd come over to me for makeup, and that was about an hour and a half. The hardest part about it was blocking out his hair, covering the blue shadow from his shaved head. To paint that and not have it look like makeup took a long time. Then he would go back to hair, and his stylist, Ann Morgan, would pop the wig on. He also grew his nails so they looked kind of creepy; but they broke off, so we ended up popping on fake nails - constantly."
Bill Corso applies the Olaf makeup to Jim Carrey. Months prior to the start of principal photography, Corso designed a variety of Count Olaf concepts in Photoshop, then manufactured prosthetic pieces for a test. The final makeup - which took 90 minutes to apply – consisted of a large hooked nose, a prominent brow bone, a heavy ‘unibrow,' a goatee, and discolored dental veneers. Throughout production, Carrey had his head shaved each morning to aid in the fitting of a balding Count Olaf wig.
Olaf's abode is as unappealing as the Count himself. Rick Heinrichs built the front and back exteriors of the house on stage at Paramount; but, due to insufficient vertical space, the set could be built only up to a height of 35 feet, leaving the uppermost 45 feet of the house to be created and tracked into the live-action by ILM. A 1/8-scale miniature provided the house extension, and also solved on-stage lighting problems. "The set was built all the way up to the lights on stage," explained Stefen Fangmeier, "and there wasn't enough distance between the set and the lights. As a result, the top of the set would be lit hotter than the bottom. Also, the stage lights could only approximate what a real sun and sky would do. There was still more key light than you would get outside. So we decided to use the miniature house for establishing shots and other purposes, because we could exercise a lot more control over the lighting on a miniature."
In addition to extending the top of the live-action set house and providing establishing views, the miniature gave Heinrichs a second shot at perfecting the look of the Count Olaf house. "Production art director John Dexter art-directed the miniature as we were building it," said Steve Gawley,"providing for an exact match. We also had extensive pictures from the set, 200 architectural drawings, and painted pieces provided by John Dexter." Lead modelmaker Bryan Dewe input the drawings into ILM's computers; then Gawley and Dewe, along with modelmakers Robbie Edwards, Tom Proost and Michael Lynch, built the house miniature from the CAD files. "That meant that our front door was exactly 1/8th the size of the big door. The overhang was exactly 1/8th the size of the full-scale set. All together, the house measured about nine feet square and nine feet tall. We nicknamed it the 'Home Depot' house, because it had every building material you could think of in it - shingles, slate, copper and brick, beautifully weathered and covered with vines by lead painter Peggy Hrastar, along with Randy Ottenberg and Victoria Lewis."
The children arrive at Count Olaf's ominous abode. Adhering to a stylistic decision to shoot most of the film on stage, production designer Rick Heinrichs built the house exterior set as high as stage dimensions would allow. ILM then completed the uppermost regions of the structure by tracking in photography elements of a scaled house miniature, constructed by the model crew. Ultimately, the house miniature was used for wide establishing shots, as well, due to the greater control over lighting that could be achieved on the model photography stage.
Life at Count Olaf's is not pleasant. The children work as slaves all day, and sleep in a cold, comfortless room at night. In mutiny, Sunny lashes out at the Count verbally, speaking a gibberish that is heard throughout the movie, translated for the audience in subtitles. The intensity of the argument required that the moment be played by the 3D animated Sunny. “Initially, we overplayed Sunny's encounter with Olaf,” recalled Colin Brady. “All of a sudden, she became too intelligent and combative — an expert debater, instead of a baby. So we pulled that back, articulating the mouth less." For some shots, the team decided to replace only the bottom portion of the live baby's face, leaving her very expressive eyes intact. “The real baby's eyes were unfocused and looking around, and they looked more believable than anything we could animate. So there were several shots where we did ‘muzzle replacements, just replacing her cheeks, nose and mouth. We didn't want to just replace her mouth, because it had that cereal-commercial-talking-animal look. Replacing the entire muzzle seemed to be a good compromise." In some instances, animators tweaked the eye area of the real baby, manipulating the brows ever so slighting in 2D to give her a more angry expression. The blending of the digital muzzle to the liveaction face was done in the compositing department, with 2D warping sometimes required to perfect the blend area.
Orchestrating an ‘unfortunate event’ for the children that will make him heir to their fortune,
Count Olaf parks his car on the tracks at a railway crossing, locks the children inside, and walks away. Production shot the train sequence on a huge forced-perspective set consisting of tracks and some surrounding terrain. “It was a big circular set,” said Fangmeier, "with full-size railroad tracks, a little convenience store that Olaf goes into, and a painted cyc all the way around it. As the train tracks and telephone poles extended out toward the painted backing, they got smaller and smaller to force the perspective. So if you walked along this track, you would suddenly look like a giant. It was beautifully done."
ILM took the live-action plates shot on the forced perspective set and added a digital train, along with smoke and reflections in the car windows for a total of two dozen shots. “The most interesting challenge for us in this sequence was to put our CG train into that forced-perspective set," said Fangmeier. “We had to force the same perspective change onto our CG train, making those transitions, starting from the backing and all the way toward the camera. The problem is that forced-perspective sets are never entirely accurate, because they're done by eye, essentially. But in CG, we had to reverse-engineer that and figure out a numerical approach."
The digital train was put through a kind of virtual 'squeeze box' to distort it and match the forced perspective of the live set. "Creature developer Andrew Anderson came up with a combination of a wire deformer and two lattices,” Gerald Gutschmidt explained, “one for the individual cars and one global lattice on top of that. The wire deformer was there to follow the track, and the lattices were there so we could dial in the forced perspective of the individual cars. The global lattice allowed us to override what we had done on the individual cars, if the effect wasn't quite right.”
The same compression had to be done on the particle smoke coming out of the engine smokestack. “We had special rules to force the perspective of the smoke,” Gutschmidt commented. “The smoke was a simulation done in Maya; and then we did volumetric rendering of the particles. We also had a custom RenderMan shader that we wrote, made up of several layers — a core and wispy layers that broke up and dissolved into the background sky.”
Atmospheric haze added in the compositing stage further heightened the forced-perspective set illusion. Compositors also added digital car windows and windshield – a post chore made necessary by on-set reflections of silks that had been hung over the stage lights. “Several compositors and I did some testing to see if we could lessen the reflections of the silks on the windows,” recalled Marshall Krasser. “But, after testing for a bit, we realized we were going to have a problem. So they pulled out the windows and windshields of the cars, and we had to put those in digitally. In some of the shots, it's really just a hint of a windshield, because Brad didn't want to obscure the characters inside the car. So we purposely kept it a bit clear.” Pat Brennan served as compositing sequence supervisor for the scene.
ILM's art department extended the train crossroads set digitally. All of ILM's 2D and 3D environments for the show had to look appropriately realistic, while also matching the theatrical style of the painted backdrops used on stage. “They had half a million square feet of backing painted for all of the sets,” said Jeff Olson. “While that worked for a lot of shots, often we had to extend it. We had to find the balance between a backing that was very stylized and a more photoreal look. It was very challenging, artistically."
“We knew it would be jarring to go from these very stylized backdrops to our 3D environments if we went for a normal photoreal look," added Wilson Tang. “The contrast would be apparent to the viewer. So our goal was to create very painterly, impressionistic environments.” ILM took digital photographs of the painted stage backings, and used those stills as starting points for their matte-painted backgrounds. “When it was necessary, we could use those to blend the starting point of our matte paintings with the end point of their painted backings.”
Arranging for the children's untimely deaths, Count Olaf locks them inside a car parked on a railroad crossing. Live-action elements were shot on a forced perspective stage set, which was extended digitally. ILM animated a CG train, using lattices to contort the locomotive and match it to the forced perspective of the set as it moved through frame. The same compression was applied to the CG particle smoke rising from the engine smokestack. Digital car windows and windshield, and atmospheric haze to heighten the forced-perspective illusion, were added in compositing.
Using one of Violet's brilliant and hastily assembled inventions, the children manage to escape a train collision, and are next transported to the home of Uncle Monty (Billy Connolly), which sits in a marshy area on ‘Lousy Lane.' Lousy Lane exteriors were created digitally, inspired by photographs Tang and other members of the crew took in a marshy area north of San Francisco. The exterior environment was created in 3D, from scratch, and was used in 10 shots of the car driving through the marshlands. Locked off establishing shots of Uncle Monty's house were created with 2D matte paintings.
Uncle Monty is a kind but eccentric herpetologist with a house full of snakes and lizards - most of which were real, shot live on the set. Exceptions included a two-headed cobra and the ‘Incredibly Deadly Viper,' both of which ILM created through computer animation. “The whole Uncle Monty set was filled with real snakes and huge lizards,” recalled Stefen Fangmeier, "So we had a lot of reference of real snakes to work from.” An animatronic viper built by the production prop department provided some additional reference for the 3D team. “The animatronic was about two thirds of the snake, from the head back, operated by four puppeteers. We also had a rubber dummy one that we could use for posing. But, essentially, they were just place-holders, and none of the animatronic work remained in the movie. Snake movement is very hard to animate, and even harder to mechanize in an animatronic. In fact, we had more trouble getting that snake animation right than we had doing Sunny. When you really look at what is going on in a snake's movement, it is really tricky.”
"Snakes are surprisingly difficult,” Colin Brady agreed. "They're among the most deceiving models to build in the computer because they're not just big tubes. Snakes never have consistent centers of their bodies, and their heads tend to be somewhat gyroscopic. So we had to give the snake multiple centers; and when we were animating it, we had to constantly switch from one center point to another.” Kagi Yamaguchi modeled the viper, then rigged it for animation in a way that would facilitate multiple-center movement. “He came up with a multiple hierarchy idea, and he was able to modify the rig for every shot - which was great, because every shot had its own problems. We never really came up with the ideal rig that worked for all scenes. Usually, I would just call up Kagi, tell him what we needed for a shot, and by that afternoon he would have created a custom rig."
For a pivotal sequence in which the digital Sunny wrestles with the viper, the actions of the baby had to be determined before the snake animation could be nailed down. For reference purposes, the production crew shot one of the twins on the set, wrapped gently in the prop snake. “That was valuable just to get the true lighting cues,” commented Brady. “We wrapped the giant rubber snake around her, and did a few shots like that. Then we took her out, shot a clean plate, and added our CG baby to that.” Animation of the CG baby evolved from broad actions to more gentle playing. "Once we decided what human performance worked the best, we were able to nail down the snake performance and get it to interact with that human performance.”
Though the Uncle Monty house set was filled with real snakes and lizards, ILM animated the harmless and inappropriately named 'Incredibly Deadly Viper,' which figures prominently in the story. For a scene in which Sunny playfully wrestles with the serpent, the crew shot one of the twins on set, wrapped in a rubber snake, to capture lighting reference, then added the CG baby to a second, clean plate. In shots featuring the real baby, ILM's rotoscoping crew reconstructed plates, removing the rubber snake to make way for the 3D-animated version.
The animation of the snake in the scene needed to look believable, but not so horrific that it would appear as if the baby was in real danger. “The gag is that people assume this baby is being attacked by the viper," said Brady, "but, actually, she is just playing with it. The viper, as it turns out, is not deadly at all. It is like a big, playful puppy dog. At first we played it that way — like a big, playful puppy — but that looked way too cartoony. So we found reference of giant snakes attacking things, and we moved the viper much more in that direction.”
ILM's roto department was responsible for plate reconstruction, removing the prop snake from the takes Silberling wanted to use for the final shots. “This snake was going over the kid's face in a lot of shots,” said Marshall Krasser, "so there was a lot of roto involved. It was incredibly hard to do, but most people won't even think about it."
Uncle Monty and the children are about to ship off on a science expedition to Peru when a replacement research assistant, Stephano – actually, Count Olaf in disguise — appears on the scene. For the Stephano makeup, Bill Corso originally took his lead from Daniel Handler's book, which described the character as a bald man with no eyebrows and a fake beard. “I did Photoshop illustrations of that idea on Jim,” said Corso, “but he looked like an evil German scientist. We were worried that it looked too scary. So Jim came up with a character that was more of a swashbuckling, very boisterous Italian – very 'Antonio Banderas,' with long dark hair and a pencil mustache. But then they hired Billy Connolly, who is boisterous and has a big personality, to play Uncle Monty – and that character was a bit like what Jim was doing for Stephano. So he and Brad thought maybe that wasn't the way to go, and we started from scratch – only a week before we started shooting."
Back at the makeup trailer, Corso and Carrey began with the basic Count Olaf makeup. "We started with the Olaf brow ridge and nose,” recalled Corso. "Brad decided that the nose was such a defining look for Olaf, we would keep that in all of the disguises as the one unifying element. Then we said, 'Okay, here's Count Olaf, and we know what he looks like — what would he do to disguise himself?' We came up with something very simple: sleeking his ‘Olaf' hair into a bad comb-over, darkening it, then shaving off his sideburns and eyebrows. We kept a little bit of a mustache, and we put glasses on him. When we got all of this stuff on him, Jim instantly became this character and started talking in a funny voice and improvising. This character just came out of thin air.”
After exploring a number of Stephano looks, Silberling and Corso opted to maintain the basic Olaf facial structure, altering his appearance by darkening the wig, styling it into a comb-over, shaving the eyebrows and adding a pencil-thin mustache.
After the demise of Uncle Monty at the hands of Stephano/Count Olaf, the children are delivered to the home of Aunt Josephine (Meryl Streep), a structure that clings precariously to the side of a cliff overlooking a lake. Production built sets for the lakeside sequence in a water tank at the Downey facility. “There wasn't an existing tank that had enough surface area," said Fangmeier, “so the production built this one, and I think it's now the biggest indoor tank in North America. They constructed a beautiful forced perspective village set right there on the tank. Briny Beach was also there. They subdivided the tank and put in different backings, the beach on one side, and the village with the dock area on the other side.”
It is in the village that Josephine and the children meet ‘Captain Sham,' Olaf's final disguise. “He's a crusty old sea captain," observed Corso. “Jim found a picture in all the research we'd done of an old sailor from Newfoundland, who had a little white beard and big Wilford Brimley eyebrows — and Jim said, “That's him!'” A bit of sunburnt look to the skin and a week's worth of hair growth, bleached white, completed the character.
For Captain Sham, Corso exchanged the Olaf goatee with a graying beard, and the black unibrow with bushy gray eyebrows. On the set, Carrey wore a peg-leg appliance – his real leg bent behind him - which ILM finessed digitally in post-production.
In addition to the village, the Downey sets included the entryway to Aunt Josephine's house, and a small section of cliff surrounding it. ILM extended the sets and painted backings for wide views, using miniatures as the basis for the synthetic house and cliff. "Rick Heinrichs and his team had built a pretty accurate maquette of the house,” said Steve Gawley. “They shipped that up to us, and that's what we actually built from. We took measurements off the maquette, and input those into the computer, so we could cut out components using our laser cutter. The maquette was 1/48 scale, and we ended up building it at 1/8 scale, mainly out of redwood." CG crews photographed stills of the house miniature, which were then photomapped onto a digital version. “The house had to be put up on a cliff far away, and for us to do that with this large a model would have been impractical. It was better to do that as a CG thing." The model shop built the supporting scaffolding beneath the house, as well, which was also photographed and applied to digital geometry.
Shortly after the children's arrival at Aunt Josephine's, the house is destroyed in a hurricane. Fullsize house interiors had been constructed atop a gimbal so that they could be made to shake and roll with the force of the storm. As the hurricane advances, parts of the house break away and fall down the cliff. "The library is the first part to break away,” said Fangmeier, "and that was completely CG. We had to do a huge rigid body simulation to get books and chairs and everything flying around the room. Just as we had done for the house exteriors, we took a lot of photos of the interior of the full-scale set to give us texture maps for our interior shots.”
Production built a huge water tank specifically for the lakeside sequence, erecting a partial house-and-cliff setting and surrounding it with a painted backdrop. ILM extended the set for wide shots, using miniatures as the basis for a digital house and cliff, and replaced the backdrop with a matte-painted view.
The library drops away as the kids run into another room, a sequence that was realized by shooting the child actors against bluescreen, then compositing the falling CG library into the background. When that second room, too, blows away in the wind, the children are left standing on a small platform atop the pilings – all that remains of the house. "The set for this was basically a platform,” said CG supervisor Philippe Rebours, “shot bluescreen.” That bluescreen element provided the starting point for a very long pullback. “They pulled back on the bluescreen,” said Marshall Krasser, “but, of course, they couldn't pull back as far as we are pulling back in the shot. So we used that bluescreen as far as we could, and then basically flew that 2D element spatially through our 3D environment. We did a set extension to the right of it, extending the cliffs, with the stormy seas below.” Jay Cooper was compositing supervisor for the hurricane sequence.
The model shop built a urethane foam section of cliff face at 1/24 scale. “The skilled hands of Lorne Peterson wielding a big machete knife led the way on that,” noted Gawley. "Lorne has become the Machete King here at ILM. He is excellent at carving organic shapes — like stone and things like that — out of urethane foam. In this case, the sculpt imitated a slatelike texture. It was approximately 14 feet tall, and about 24 feet long at the base. Lorne, Carl Assmus, Richard Miller and their crew sculpted and painted this big cliff face in about two and a half weeks. We had to do it quickly because it had to appear in a trailer that was coming out shortly thereafter. In addition to being used for the cliff face element itself, this model was used to put texture mapping on the surrounding cliffs on the lake.”
CG supervisor Philippe Rebours checks out a stage setup of a cliffface model carved out of urethane foam. The miniature cliff set was used for a sequence in which Aunt Josephine's house blows apart and tumbles from its clifftop perch in a hurricane. In addition to being used for the cliff-face element itself, the model was laser-scanned and duplicated in the computer to create 3D models of surrounding cliffs on the lake, which allowed Brad Silberling complete freedom in moving his camera through the environment. Still photographs of the model provided textures for the CG terrain.
"We laser-scanned that portion of the cliff,” added Rebours, "and also took plenty of photos of it and used those to apply the texture. We were able to use that model by duplicating it and cutting it, et cetera, to give us enough cliff for all the shots. We also had some still camera shots. In those cases, we took one photo of the miniature, and if we could just use that, we used that.
But for the moving camera shots, like the big pullback, we had to apply the miniature textures to a 3D version of the cliff. That allowed us to do whatever movement Brad wanted on it.” As the shot continues to pull back, the village enters frame in the background. To accommodate the move, the village stage set was re-created digitally, with a matte painting serving as the far background. The platform the children stood on against bluescreen also had to be re-created to accommodate a change in the camera movement. “We had to re-create it in CG so we could change the perspective slightly from the movement they had shot on the bluescreen stage. On the set, it is very difficult to get the speed that you want, and we had to get our move to link up to the move they did on that set. For all these reasons, it was easier to replace the original platform to get the changing camera speed and perspectives.” Stormy CG lake water that could be seen at the bottom of the cliff was created through shaders and geometry.
For shots of one room after another shearing away in the gale-force winds – leaving the children stranded atop a swaying platform – Brad Silberling and his production crew photographed the child actors against bluescreen, on partial house sets. Full-size house interiors were constructed atop a gimbal so that they could be made to shake, rattle and roll with the force of the storm.
A down-shot at the end of the pullback reveals the house structure breaking apart as it tumbles down the cliffside and into the water. ILM crews dropped breakaway house miniatures from a height of 40 feet, filming three takes. “We did some tests for that,” stated Steve Gawley, "then gave the miniatures to Geoff Heron and his effects crew, who rigged them to break apart on the way down. We were up very high on three big Condor cranes, and we shot them falling highspeed.” Separately, ILM shot house-shaped objects dropping into a 40-by-40-foot pool to capture water interaction of the house falling into the lake.
ILM integrated the bluescreen set elements with a 3D environment made up of the cliff-face set and CG stormy waters below, created through geometry and shaders. In post, the stage platform shot against bluescreen had to be recreated digitally to accommodate a change in camera movement. Final shots in the sequence, in which the house tumbles down the cliff and into the lake, made use of breakaway house miniatures dropped from a crane.
The resourceful children make their way off the swaying platform, back onto solid ground, then commandeer a sailboat to go in search of the now missing Aunt Josephine. Live-action for the scene was shot in the middle of the water tank in Downey. "That portion had a very dramatically lit backing, with golden sky and everything,” said Stefen Fangmeier. “But, of course, it looked like a painted backing and a tank. There was also a horizon painted into the backing - but with different camera angles and everything, it didn't work. So we had to extend all of that and create a new horizon and background.”
"You could see where the tank water stopped and the painting began,” added Marshall Krasser. "So we did some CG water to transition across that area and give it the feeling of live water going out to the horizon.” Compositing sequence supervisor Tia Marshall finessed tank shots for this and the following scene in which the boat is attacked by giant leeches – seen initially only as ripples in the surface of the lake water. ILM created 30 shots for the leech attack, building upon some practical rippling water effects that had been executed live, in the tank, and replacing practical leeches that had been designed and built by the production prop department. "The practical stuff wasn't exciting enough," commented Fangmeier, "and it didn't look like enough leeches. So we replaced a lot of the water, creating CG water with ripples in it to make it look as if hundreds of leeches were there. Interestingly, when you are shooting in a tank, the water looks extremely CG; so it wasn't that hard to match that water."
ILM animated its CG leeches, implementing a real-time physical simulation simultaneously. "As we were animating the overall movement of these leeches," related Colin Brady, "the simulation would make their tails wiggle and overlap as they were whipping around. Instead of the animator doing his animation and handing it off to a technical person to put all the overlap physical simulation stuff, we had that built into the program in real time; so the animator got all of that for free, instantly. That was the first time we'd done that."
The children are rescued by Olaf and taken, protesting, back to his house. Once there, he stages a play that is, in reality, a marriage ceremony, convincing Violet to marry him by holding Sunny hostage in a cage dangling from a house-top tower, rigged to drop if Violet refuses him. Live-action for the marriage sequence, and a subsequent scene in which Sunny escapes the cage, had been shot on the full-scale Olaf house and yard set on stage at Paramount; but, again, ILM tracked in the top-level portions, created from the house miniature. "All of the skies were created, as well," said Jeff Olson, “because they were shooting off the top of the stage at Paramount. The entire environment was a very big 2D composite, with miniatures and matte paintings. Marshall Krasser took the lead in putting all those pieces together."
Model builder Robbie Edwards constructs the 1/8-scale Olaf house. The model shop based the miniature on photographs from the set, architectural drawings, and painted pieces provided by art director John Dexter. Model shop crew members input the drawings into ILM's computers, then built the house miniature from the CAD files, ensuring that every feature of the replica was precisely scaled to the full-size set.
Compositors also finessed a shot of Count Olaf entering on wires in the course of the wedding scene. “They actually had him on wires on the set," commented Krasser, "but the wires didn't look funny enough. This was supposed to be a cheesy Count Olaf production, and these wires just weren't cheesy enough. So we ended up shooting bigger, thicker rope, and we put those over the existing wires. It was kind of weird for us to actually have to add thick wires to a shot!"
Sunny escapes, Olaf's marriage scheme is dashed, and the kids return to the remains of Baudelaire Mansion with a better understanding of the circumstances surrounding their parents' deaths. Via the power of their imaginations, the house returns to its pristine and happy state. "The camera pans around the children,” Stefen Fangmeier related, "following them up the stairs, coming around and seeing their reaction to the house transforming right before their eyes. So there were two live sets – one was the pristine house set and the other was the ruined house set - and we shot them both motion control. It was difficult to get them to line up, because we weren't just shooting passes on the same set; we had to actually move the motion control tracks and everything from the ruins set to the pristine set."
For the actual transformation from one set to the next, ILM attempted a look that was more three-dimensional and organic than a simple cross-dissolve. “We wanted it to look like there was invisible fire eating away at the house and burning it," said Fangmeier, "reducing its volume in a very interesting, three-dimensional way. It is almost as if you are watching the house burn away in timelapse. We used a particle approach, but with a heavy-duty RenderMan shader. It's a very magical shot, but it required a lot of R&D."
"This was such a challenging, unique shot," added Gerald Gutschmidt, "we worked on it up to the last minute. It started off with an ultra-precise matchmove set. We did Maya simulations on that, so instead of a traveling matte or 2D mattes, we had an actual 3D reveal happening. It is literally as if an arsonist had gone around and burnt the set in several places. So we have places where the fire crawls up along the wall, but in a very natural way. The camera is moving and the lighting is changing throughout the shot, from the ideal pristine lighting to the burned ruins look." Ultimately, Brad Silberling simplified the shot somewhat. The transition was still based on very precise matchmoved and photo-modeled geometry, but it used a dozen lights with a special flicker noise shader to produce 3D mattes. The transition was processed further in the composite, enhanced with 2D displacement effects and 3D particle dissolves.
Materials used on the miniature for Olaf's house included shingles, slate, copper and brick. Crew members weathered the model with washes of paint and covered it in scaled vines. Modelmaker Randy Ottenberg applies final set dressing to the miniature.
As Lemony Snicket warns throughout the film, the story does not end on a 'happily-ever-after' note; but, instead, on a note of ambiguity as the Baudelaire children face their uncertain future. "What's great about these books, and this movie," concluded Jeff Olson, "is that, unlike most children's books, they have a dark feeling to them. Things don't always work out so well. Daniel Handler has quite a dark and wry sense of humor, and that really comes out in this movie. Lemony Snicket is like 'Harry Potter meets Edgar Allan Poe."
The film's dark and subtle tone was sustained in the effects, many of which were virtually invisible to the audience. In that way, Lemony Snicket's A Series of Unfortunate Events recalls a more classic type of effects movie - one in which the effects do not call attention to themselves, but are there only to further the story. "For this show," commented Colin Brady, "we had to make our effects – especially the Sunny animation effects – as invisible as possible. That was the true challenge. We'll know we did our job right if everybody who sees this movie just thinks Brad Silberling found the best acting baby in the world!"
Lemony Snicket's A Series of Unfortunate Events photographs copyright © 2004 by Paramount Pictures and DreamWorks. All rights reserved. Production still photography by François Duhamel. Industrial Light & Magic photos by Sean Casey. Special thanks to Ellen Pasternack, Stephen Kenneally, Miles Perkins, Lori Petrini, Megan Corbet and Karl Williams.