Main Media Filmmaking
Certificate in Collaborative Filmmaking at Maine Media College

Tricks for Working with Troubled Green Screen

Share this post

Tricks for Working with Troubled Green Screen

Written By Lee Lanier

Working with green screen is a common task when it comes to VFX compositing. Yet, many times you are handed a shot where the green screen is quite poor. This is not only the case with student or independent productions but may also be true for $100,000,000 blockbusters. Sometimes, production limitations do not allow for a careful green screen preparation, especially when shooting on location. So, what pitfalls may you encounter with such shots? Here are a few common problems:

(1.) Irregularly lit screen with wrinkles and/or shadows.

(2.) Objects overlapping the screen, such as light equipment and/or tracking tape marks.

(3.) Subject too close to the screen, so there is heavy green spill (reflected green light).

(4.) Reflective elements on the subject that show the green or greenish clothing, labels, and the like.

What can be done about such problems when it comes to compositing? Well, there are a number of steps you can take to make your life a little easier:

(1.) Create a garbage or outside mask around the subject to reduce the amount of green you’re trying to remove. This also allows you to remove unwanted objects such as rigging or tracking marks.

(2.) Create an inside or core mask to restore the interiors of the subject (which otherwise might wind up with transparent holes). Many chroma key tools or keyers, designed to remove green screen, support such masks.

(3.) Examine the alpha channel as you adjust your keyer. You can’t really tell the quality of your keyed footage unless you look at this channel, which determines what part of the frame is opaque, semi-transparent, or transparent.

VFX Compositing. Tricks for Working with Troubled Green Screen.

VFX Compositing. Tricks for Working with Troubled Green Screen.

VFX Compositing. Tricks for Working with Troubled Green Screen.

There are many compositing software packages you can use and many keyers to boot. Nevertheless, let’s take a look at one common workflow: using the Keylight plug-in effect inside Adobe After Effects. Here’s an example green screen shot (Figure 1). It suffers from irregular lighting, wrinkles, tracking marks, and heavy green spill in the singer’s blonde hair.

The first step, after setting up a composition, is to draw an inner and outer mask using the Pen tool (Figure 2). Because our singer is moving, the masks require some rough animation (that is, rotoscoping). The masks only need to be loosely drawn but should not cut off any part of the singer or include the tracking marks.

Using Keylight, you can expand the Inside Mask and Outside Mask sections and set the Mask menus to the corresponding masks (Figure 3). Note that you will need to turn on the Invert checkbox for the Outside Mask within Keylight. Also, to make this work, you will need to set each mask to None in the layer’s Masks section (Figure 4). In my example, the inner mask is Mask 1, and the outer mask is Mask 2.

VFX Compositing. Tricks for Working with Troubled Green Screen.

VFX Compositing. Tricks for Working with Troubled Green Screen.

VFX Compositing. Tricks for Working with Troubled Green Screen.

VFX Compositing. Tricks for Working with Troubled Green Screen.

After you’ve completed the masking, you can set the other important Keylight options. Here is my preferred workflow:

(1.) Set the Screen Colour. Try to choose an average green color close to the subject. Try to avoid selecting green within shadows or dark areas of wrinkles.

(2.) Switch the View menu to Combined Matte so you can see the resulting alpha channel. Odds are there will be some gray in the foreground (the part that should be opaque) and gray in the background (the part that should be transparent).

(3.) Expand the Screen Matte section. Raise the Clip Black value until the background is black as can be without eroding too far into the foreground. Lower the Clip White value until the foreground is white as can be without introducing gray into the background. The goal is to produce a pure white foreground and pure black background with some gray on the edges of the subject, like along the hair (Figure 5).

(4.) Return the View menu to Final Result. In the Screen Matte section, set the Replace Method to Source (Figure 6). This prevents Keylight from aggressively removing green spill from the footage, which often causes extra noise. If you need to remove green spill, use a separate effect. For example, add the Advanced Spill Suppressor effect and set its Method menu to Ultra.

That’s basically it (Figure 7). Now, there are many other Keylight options you can adjust to improve the result; however, if you follow these basic steps, you will have a solid start. If you’d like to delve deeper into chroma key work and Keylight, there are many fine tutorials available (some of which are mine) at LinkedIn Learning,, and similar sites. Good luck!

Lee Lanier has worked as a professional animator and VFX artist for over 25 years, having spent time at Walt Disney Studios and DreamWorks. He’s also written a dozen books on the topic and has recorded video tutorials for various companies that have generated close to 1 million views. You can see his work at and

Recommend0 recommendationsPublished in How-To Be A Filmmaker Articles, Post Production