A method to the madness

Woah, woah, woah. A third entry in just as many days. You'd think I wasn't working on a show to be delivered in less than a month. In the Open Call entry from last year I asked people to give suggestions on what they wanted to read about. One of the suggestions was the following:

Could you give us a rundown of the things you encounter most often while working on a complex script? A shot that has multiple cg elements integrated? I find the questions I primarily have now deal with pipeline issues. Of course some of the tools and nodes used are proprietary, and I know those can't be mentioned. It would be very nice to hear what the main passes are that you typically get from the 3D dept. I'd love to hear about any of the more modern ways of approaching a shot. Such as rendering flat with a normals pass and a couple lighting passes, doing the final lighting in post.

So here's an answer. I'll try to keep it as simple as possible, and not be specific to a particular company. When I say we, it refers to the compositing departments of the studios that I've worked for. Feel free to ask questions in the comments section.

Over the span of a decade, lots of things have changed. We've gotten faster, better machines, more intuitive software packages, and the knowledge that people have gained over the years. From the dinosaurs in Jurassic Park, to Davy Jones in Pirates of the Caribbean, the detail of the characters and environments have increased substantially. We are only limited by our imagination now. As such, the elements we are required to assemble vary in complexity. From the well known color channels such as red, green,blue, alpha, and z depth, to more specific ones such as ambient occlusion, global illumination, specular, diffuse, environment, subsurface, normals, uv maps and more! All of these rendered elements are not always necessary. Depending on the complexity of the final shot, we're tasked with often using the least amount of rendered elements (layers) to accomplish the shot. At the very least, we need RGBA.

But just using RGBA doesn't always give us the level of control that we compositors need. So, we ask for more render layers. Often these layers are just things that our Lighting TD can give us. Most of the time they're very good about giving us all the correct layers that we need to final our shots. Other times we approach them and ask for extras. The communication between the compositing department and lighting department is vital to the completion of our shots. How do we decide what we need? Well, there are a couple ways. Ask a lead. Comp and Lighting leads are leads for a reason. They've had the experience to know what render passes are necessary to get the look of a shot. Other times, experience will lead the way. I've reached a point where I know what passes I need from lighting artists, and if it's not rendered out and I absolutely need it, I'll let them know. Only time and experience will let you know what you need when compositing a final shot. Most of the time we receive the following passes. R,G,B and A of course, as well as Z depth. For environmental comps, we might get a Y depth pass. Almost always a diffuse and specular pass is present, as well as normals. There are also certain plugins and tools that we use in our day to day compositing tasks, from normals remapping and lighting in 3D, to lens distortion and uv projection. All these help us to final a shot. These are some of the basic passes we get. Depending on the timeframe and complexity of a shot, we might get more render intensive passes such as ambient occlusion and subsurface scattering. Often, the final shot might look like it has a certain cg pass, but in reality, we hacked it together using a number of different tricks and methods.

In constructing a complex shot, it's vital to break down the major bits and pieces into managable chunks. A bunch of tanks on a beach, for example, in Flags of our Fathers, requires different approaches for the shot. The tanks themselves may have required all the passes I mentioned, from diffuse, reflection, specular, environment, ambient occlusion, z depth, and so forth. The beach, being CG, might require a different selection of passes, from just diffuse and specular, to z depth. The men on the beach, even different passes, a beauty pass, diffuse, specular, and shadow. Then you have the CG water, and the background matte painting of sky and clouds. As you see, the complexity of this shot can grow enormously. By breaking the shot up into specific pieces, it's easier to delegate the responsibility of the completion of that piece to a department. The water and smoke and clouds, to the fx department. The tanks, modelling and lighting. The CG people, animation and lighting. Assembling everything, the comp department. In compositing we don't really light a shot. We might fix or add small nuances that a supervisor might ask for, but by and large, we don't relight anything. As we build a shot up like this, we frequently get new renders from all the departments, and it's our job to put them all together, document what goes where, and what changes from the previous version there might be. Approaching such a shot in steps is necessary for sanity. Some of us are more insane than others. As illustrated below.

How in the hell can we manage such complex and messy scripts? Well, if you're working on a shot for the better part of two or more weeks, you'll know your script inside and out, especially if you start it from scratch. Taking over a shot, like the one on the left, is just asking for horror. The one on the right is a little bit more manageable, and actually has some notes. On Pirates of the Caribbean I took over some shots from other artists, and let me tell you, it's often easier to go ahead and start over from scratch for certain parts of the tree than try and figure out why your fellow comper decided to do something one way, and you know another way to do it. Our minds are wired certain ways, and it shows when constructing and deconstructing shots such as these.

I hope this post was informative for you! Please let me know if I answered your questions to completion, and how we approach complex shots. Don't be shy.