Matthew Clark Shares Virtual Production Techniques Behind HBO Comedy Thriller

The creative team behind HBO’s Run pioneered a virtual production technique with potentially far-reaching impact for how future episodic shows are made. Created by Vicky Jones and co-executive produced Phoebe Waller-Bridge (Fleabag), the show’s deceptively simple premise has two former college sweethearts (Merritt Wever and Domhnall Gleeson) drop everything in their lives to reunite after 15 years apart. The characters spend most of their time travelling across the United States on a train but cast and crew never planned to leave Toronto.

Cinematographer Matthew Clark (Late Night) explains, “In my first meeting with Director Kate Dennis, Production Designer Denise Pizzini, and the producers, the discussion was all about how we were going to shoot a story which takes place largely on a train hurtling across the country. We ran the gamut of ideas from photographing onboard a real train; partial real, partial studio shoot; or using green screen.”

Stargate Studios in South Pasadena gave them a new direction. It demonstrated how a mocked-up train carriage surrounded by LED screens could display video to simulate a photoreal location.

“What was key is that the display was tracked to the camera movement by a wireless sensor so, as I stood up and shot out of the window, it looked like I was shooting down at the track and if I knelt down I was shooting more of the sky,” Clark explains. “Since we had planned to shoot a lot of handheld the ability to change perspective in real time sold us on the possibility. We then had to figure out if it would work practically.”

On a soundstage in Toronto, the production built two cars outfitted to resemble an Amtrak carriage. These rested on airbags which could be shaken to simulate movement. Instead of LEDs, a series of 81-inch 4K TV monitors were mounted on a truss outside each train window. The train’s windows were made a little smaller than real life to avoid camera seeing the edge of the TV screens.

“It’s a smaller scale and less expensive version of Lucasfilm’s production of The Mandalorian but the principal is the same,” says Clark. “It effectively brings the location to production rather than move an entire production to often hard to access locations.”

At all times Clark wanted to recreate the feeling of being onboard an actual train. “I wanted to make the camera feel as experiential as possible – to put the audience with the characters,” he says. “That meant that any light that played on the actor’s faces or on surfaces had to be synchronized to the illumination outside the windows otherwise the effect wouldn’t work.

“It was important to line up the picture so when you’re standing in the car your perspective of the lines of train track and power lines has to be realistic and continuous. If the angle of the TV screen is off by just a few degrees, then suddenly the wires of a telegraph pole would be askew. The monitors are on a truss so that when we needed to turn the car around to shoot from another angle the grips could flip all the monitors around to the exact angle.”

For this endeavor, he selected Panavision’s DXL2 with the RED MONSTRO 8K sensor in order to work with large-format lenses.

“I wanted to create the sense of shallow depth of field with a large field of view so when we’re inside the train the emphasis of the scene is on the actor not on what’s going on outside,” Clark says. “It is supposed to be overexposed in places to give it a bit of lo-fi realism.

“Sometimes we do focus on the view outside the train which is—let’s not forget—a TV screen that is flat and displays video that is completely in focus. To avoid the whole shot feeling like that, I wanted to control depth of field.

“The camera and lens package helps creates an intimacy akin to shooting anamorphic. Crucially, the way this works with the RED sensor is that the image is much more rectilinear, giving a more defined straight edge to the frame and no aberrations on either side of the wider lenses. That’s an important consideration given that we are confined to such a small space.”

Opening the MONSTRO sensor up full enabled him to use large-format Panavision Ultra Speed lenses typically with speeds from T2 to T2.8 and close focus distances from 2 to 5 feet.

“The Ultra Speeds did allow for the T1.1 a few times when we wanted to get ‘in the character’s head’ by using razor thin depth of field—one eye sharp or a super soft shot that the character then walks into focus. The fun part of lens/sensor choice was the field of view with relation to the depth of field – perfect for our story.”

Lighting with electrics and CG

In pre-production, Stargate’s Sam Nicholson, ASC and Bryan Binder fixed four RED 8K cameras and two Sony VENICE 6K cameras to the outside of a train car and captured footage from Amtrak lines crisscrossing the States, including from Vermont to New York, NYC to Chicago, onward to Los Angeles, and a route from San Francisco back to Denver.

The footage was stitched together at Stargate into a 180-degree bubble for either side of the train “so you could literally see the cars turning in front,” Clark explains.

“My gaffer, Randy Brown, and I would go through the script and figure out how to light each scene in a fairly conventional way. Then we’d find a spot in the VFX footage that I liked as a foundation and shared this with Stargate so they could pixel map the light to match the level on the live-action picture.”

Ambient exterior lighting fixed to the adjustable truss just above the monitors outside the car’s windows was supplied by the SkyPanels S30, S60 and S120 series, augmented on the ground with ARRI L7/L5/L10 LED Fresnals. These were tied to a control board enabling Clark, an operator and the VFX team to tune the lighting. The system was able to deliver a photorealistic moving image, displayed through the train windows, with animated lighting to match the plate, tracked to and composited with the shot in real time.

“For example, if the train is passing in and out of trees then the light would fluctuate or if it travels at night past a red light, that red light would be tracked down the outside of the car. Everything was pixel mapped to the plate footage by Jon Craig at Stargate. We could adjust the color temp, intensity and even the placement of the effect by a combination of traditional dimmer board and VFX map communication.

“In the pilot, there was no baseline there for me so to some extent we were using our best judgement to get it right, but once in the series Jon, Randy, Andrew Read – our fantastic dimmer board operator – and I were able to work out some more detail, making our lighting control much more repeatable.”

For even greater realism, Pizzini included reflective surfaces in props and materials which flicker and shift with the lighting design from super dark to bright and modern.

“Any glass on the set, even water glasses, reflect the effect of light and picture from the outside world. The characters even take a couple of shots on an iPhone out of the window—which are in fact of the monitor. We could create these layers of actuality because we are able to work with the image right there, something that has always been troubling with greenscreen.”

While some scenes were lit just using the monitors, it proved challenging to augment the window illumination, especially for shots framed close up.

“Being able to get an additional light in to augment the feeling of sunshine coming through a window was a struggle,” Clark says. “It was problematic to move a light near enough so that it wouldn’t interfere with the view of the monitor through the camera.”

Considering the volume of scenes featuring the train in the seven-episode series, it is remarkable how few shots needed additional treatment in post. Clark estimates that just 10 percent of scenes required additional greenscreen as coverage when a character moved quickly through a carriage.

“Usually doing VFX takes a lot of time and productions typically don’t grant the DP much time to work it out in post, but on Run I could set exposure based on what I saw there. Sometimes I wanted to see every detail of the trees, other times I wanted it blown out. I felt this approach maintained a sense of story for Kate, myself and the actors without having to revisit the scene and manipulate the photography later.”

Naturalism was also the tonal key for Clark’s color palette which is predominantly of desaturated blue and silver in keeping with Amtrak livery with shadow greens for landscape.

“We didn’t want to tip our hand and make it super colorful which is the ‘go to’ for comedy. There are moments of gravity and mystery which is why a more muted palette is suitable, especially in the pilot to set the mood. As the story proceeds, the looks evolve but all the while we’re aiming for naturalism and not to push too far in one direction.”

As processors get faster and high-resolution displays become more affordable, the real-time combination of photographic and CG assets will become much more prevalent.

“I do think virtual production will change the way a lot of shows are made,” Clark says. “For me, it’s important that when you make a choice to build a set and ring it with LED screens or monitors you don’t lose the naturalism you get for the location. That is what you are striving for. You don’t want the technology to get in the way of the storytelling.”

Photo credit: Ken Woroner/HBO