States Expense _ Visual Effects Sequence

Length: 00:01:57:12

Number of VFX shots: 14

Frame Rate: 24
Export Resolution: 1920 x 810
Shooting Resolution: 4096 x 2160
Codec: ProRes HQ 444
Camera: Ursa Mini Pro 4.6K

Lenses: 24mm, 35mm: Samyang Cine Primes 

Role(s):

• Director of Visual Effects 
• Compositor 
• Roto
• Keying 
• Tracking 
• Colour Grading (VFX Sequence) 
• BTS Photography

Software Used:

NukeX (Keying, Tracking, Composite work)
Adobe After Effects (Particles, Composite work, Grade)


The Project 

Short Film

Creating a full green screen composite of an ex-soldier stuck at death's door after overdosing on drugs during a rough time (As shown in the full cut) He finds his way to the light, with his friends voice amongst the darkness that surrounds him flashing through audio memories and visuals from his previous time in the army, making the reassuring voice of a passing friend sound like home (The Light).

The Visuals 

An empty space, dark full of mist and smoke out of this current work a visible yet grungy floor. White silhouettes for lights, rays, smoke and particles passing by as he struggles to see through to the light.
---- Pre Visuals can be found here: https://www.youtube.com/watch?v=zYPeVZmm228

the light revealed behind him, Rays pass through his body as he is absorbed by it.

The Technical 

Using on-set lighting to light my subject from above to give the effect of him being alone in a dark space unable to see past his reach.

My Research and using industry techniques 

The Industry Techniques I followed.

On set practice following schedules  

Lighting with a BBC Camera operator

Working with a Director and team

Following the post production team through the full edit

Receiving Edits in premier pro and creating workflows between - Editor, Grading and post production

Creating a file management system

following delivery methods

Why I chose to shoot the scene on a green screen using practical lighting and a 4K Resolution.
"On any major feature film a common problem can be shooting VFX exteriors elements such as the actors on a soundstage in front of green screen. Ideally one would shot the green screen outside with the correct matching sunlight direction but this is not always possible" "The solution has virtually no additional cost, but relies on the much higher shooting resolution that modern cameras such as the Red Camera provide. While a Red camera can shoot up to 8K, VFX shots are still normally mastered at 2K. This is a brilliant use of that extra resolution.  The technique works for films, television, commercials and documentary production. It is not a 'VFX post process", in the sense that it does not move the problem to post and add cost or time to your budget."              @Mike Seymour, March 2016.
https://www.fxguide.com/featured/lighting-vfx-master-trick-from-chris-godfrey/

Why I used Element 3D to render out my 3D assets and textures vs Arnold which is a more photo-real for renders.
"CPU rendering vs GPU renderingOn the CPU a more complicated scene will just take longer to render, but on the GPU, if the scene or memory requirements are too high, the GPU will crash so it is important to manage your GPU’s resources by closing unnecessary GPU programs and watching texture memory usage. A graphics card with 2GBs of VRAM will allow you to create more complex scenes compared to a card with only 512MBs."       @Andrew Kramer, December 2015
https://www.videocopilot.net/docs/element/render_settings/optimizing_render_settings_performance/

Keying in nuke and creating my own workflow based off of the three pass method.
The Three-Pass Method by Don Starnes Initially, I did my keying and compositing in Final Cut Pro (7 and X), After Effects or Resolve. All of these have fine keying tools; some of them have automated tools that practically make the key for you. No matter what software you use, though, some part of the image is often stubbornly resistant to the algorithms’ best efforts. A solid key of the body makes an artificially hard edge. Adjustments often don’t help much in getting a good overall key. Even if the green screen is perfectly even, different parts of the foreground image have different properties: therefore one-pass keying is almost always insufficient.Friends don’t let friends use one-pass keying. Following treatises by Mark Christiansen and others, I started making three basic, overlapping mattes for each.

• edge, which is focused on preserving the sometimes semi-transparent pixels of the delicate edge 
• the core, which hammers out a solid body for the object but excludes the edge
• a light wrap, which wraps a bit of the background around the edges of the foreground (in the way that light refracts around objects in real life)

In addition, I divided any problem areas of the image into separate mattes. For example, I’d make a matte for the hair portion of the foreground image if it required different keying than the body.

Although my skills were, shall we say, developing, I was able to get significantly better composites using these professional methods. Still, I was dissatisfied: despite my best efforts, the edges were a bit noisy and I couldn’t preserve as much hair detail as I’d wished.

      @Don Starnes February 15, 2016

https://www.provideocoalition.com/keying-and-compositing-in-nukex/



The Visuals - 

Although I didn't get much creative input in the VFX sequence I had inspiration from the TV show 'Stranger Things' I tried to incorporate the visuals from the dark scene (LINK BELOW) into my scene as described to me by the creative director, so I had a run through the TV show and used the colour pallet in my project to base my work from. 

~The Dark Room 

https://www.youtube.com/watch?v=c-_KRSo54wE


Colour Grading 

For the grading of the pallet I wanted a way to split up my grading workflow add more detail and depth into the image in nuke for the intro scene, after following a tutorial I found from a professional Grading artist and compositor.


~ The Bloom Technique 
https://www.youtube.com/watch?v=GjiQjscs1LY&t=1685s



Keying, Roto and Tracking

All my keying a roto work was completed inside NukeX, and I have devised a few of my own pipelines workflows to speed up my keying which worked for a select few shots (Examples Below).

The workflow is based around a small multi-key just around the edge of the body using a grade node to get almost clean edges around the subject then keying them using key light and pulling out the and merging it over the original footage and re-adding the motion blur a green spill remove script.  I only used this workflow for the handheld fast moving shots as the motion blur hides any mistakes in the key and the lighting in the composite fades them all together giving the exact same effect of a perfect key.

The hardest shot was the first shot of the sequence. The lighting wasn't even to make for a good key and tracking markers placed throughout the shot covered by the subject, In order to composite this shot I had to create a clean plate, here's how I did it.

Tracked the shot in Nuke using the camera tracker (Masked out the subject and increased contrast for a better track, also de-grained)

In and out key over the subject roto then piped back in, replaced the shodow with a constant node masked around the subjects feet, added grain to match the original. Cleaned all the tracking markers up and piped the key into a grade node to blend the shadow and the footage together, added a pre-lightwrap node to blend then floor texture with the shadow, added a 3D scene based of the 3D camera I tracked earlier, linked the 8K floor texture to a 3D card system with a light positioned in the same place as the on set light, added motion blur the renderer (Same as we shot in 140 shutter)



 ---- Before final Composite in AE



 Due to the reflective surface on the subjects head I had to go back and roto the top of it for the first 20 frames and then later it swipes past a tracking cone which I also had to roto.







Only 5 out of 14 shots could be tracked the rest of them were shot handheld which was actually a bad decision on my end as I had to manually create cameras based of the lighting I could see on the green screen in order for the particle movement to be believable. as for the bullet tracers flying by, I manually created them using mask tools then added glow, vector blur and motion blur and animated them in different directions in sync with the light.







































Compositing







Using Fractal Noise and Particles I will create an atmosphere to surround the subject and sell the effect of being lost in darkness. (linked to a 3D Camera with Z-depth enabled for realistic depth)

- Update I have used light rays on a separate composition merged over the top of the main one using the subject as a light emission to spread the effect of the fog. 

I will repeat this process for the main sequence of shots where the subject is in fast motion and looking around in panic.

-Update I have added particle emission from the subject's alpha to produce realistic particles coming off the body almost giving it a vivid effect of not being real and to sell it to the viewer
.

Adding fake depth.

For one of the shots (2nd Shot in), I added a fake depth of field rack focus to show to bullet tracers falling past the subject as he turns around the camera changes focus to then reveal his reaction afterwards back in focus  (I used on set lighting to glow the right side of his cheek)
 

 During the dark scene, the subject has flashback like visuals so I have added a destroyed city behind him with debris falling towards the camera for a split second as it suddenly bounces back into the dark visuals as the calming voice of a friend starts to echo through the scene.


This scene was created in Maya and then exported as an OBJ into Element 3D (A 3D plugin for after effects) I did this to speed up my composite times as it was for a small shot and is only used briefly. I added a fake camera move, FOG, 2D Assets and light emission from the passing tracers above to give the feel of him being in a war zone.


I used this technique again in the final scene where the light absorbs the subject. The floor is a full 3D floor with 8K textures placed onto it (The same as the first shot and the city shot for continuity) I then added a depth fog and some Fractle noise to some 3D plains and adjusted them and animated them to pull in and out as the light gets brighter. For the light source, I used a Spotlight and then linked a Knoll light to it that was animated in and out alongside a light rays effect so it would feel as though the mist is pulling in around the subject.
 



Nuke keying and compositing

shadow diffuse

adding bloom

tracking marker removal

keying workflow

3D card system

Working with a production team and following industry practice conclusion.

During the production of the short I attended all the shoots to take BTS photos as I was requested to do so, this helped and gave me an open eye on the full project down to each role of everyone on set. This was the first full scale production I have worked as a VFX director and following industry practice such as, file structure working alongside editors and grading teams to get the correct files and footage to work in my programs, working to a deadline and delivery quality for the project, sitting alongside the director and running through shots and creating Pre-vis on set and post vis which then lead into the final look and delivery. During my time on set, I also had a talk with a camera operator from the BBC who gave me some tips on lighting for the green-screen he learnt through years of practice, I now have that knowledge to use on my future shoots.

Overall I believe this project went very well, it opened my eyes to industry practice and made me realize how creative input is limited on set when working for somebody else to get a certain look, I believe this shot will help me in my future depths into the VFX industry and has given me a taste for more work.