Blockbuster Scanning: Using LiDAR for Joker & John Wick

Author: John Ashby

John Ashby is LiDAR supervisor at Aura FX, a company providing on-set data acquisition services for the film and gaming industries. Ashby has worked on Hollywood blockbusters including Joker and John Wick: Chapter 3 – Parabellum, capturing 3D scans of film sets and locations. The captured scan data are used in many ways during film production, especially for creating VFX (visual effects).

With time pressure one of the main challenges of the job, Ashby uses a Leica ScanStation P50 long-range 3D laser scanner for high-resolution, fast scanning. And with lots of data to juggle, he uses Leica Cyclone software for easy processing of large numbers of scans.

The benefits of reality capture in the film industry

Using LiDAR and other laser scanning technologies is helping the film industry to increase efficiency in production and post-production, and at the same time, create more realistic visual effects. A scan of a film set or location provides an accurate record. which can be used to help match camera angles. This ensures continuity and sets the foundation on which to build any virtual assets, for example, a digital set extension.

Using LiDAR to improve the realism of VFX

Having a scan as a reference point helps VFX teams create more seamless effects. Previously, footage of a film set would be used to make a virtual recreation in 3D. To add realistic-looking effects, the technicians would need to match the same camera angle and motion as in the original shot (‘match moving’). Without laser scanning crews would take detailed measurements but couldn’t achieve 100% accuracy. By capturing the entire 3D world of any shot, LiDAR and laser scanning combined offer a precise map from which to work.

Ashby gives an example from his work on Joker. “I scanned a lot of sets, including Arthur’s apartment, and the TV set for the Murray Franklin talk show, which I scanned because there was all this CG blood that had to get added in post and this allowed them to get a perfect camera track of those sets.”

Fast scanning speeds are required for scanning in a frantic environment

Speed is critical for scanning on set; with large crews and huge running costs, film production can’t be delayed by data collection. : “It’s all about what’s the most data you can get in the fastest amount of time without holding up the production,” Ashby explains.

When working on John Wick 3: The Continental, Ashby had only four hours to scan and photograph a major location. “The building for the shoot-up of the Continental was One Hanson Place in Brooklyn which is huge. It was just a pure race against time.” Ashby needs to get as much data as possible and to be prepared to stop and start scanning to fit around filming, sometimes even grabbing scans between shots.

Ashby explains his process of LiDAR scanning on set or location, “I tend to start with a quick blitz of the particular environment to get a ‘registration’ of it. This is just lots of fast low-resolution scans; it’s just so I know in the worst-case scenario, I’ve got a full registration of the environment. Then I’ll go around and do my luxury scans from different ‘stations’...upping the resolution and sampling quality in order to get the more distant and reflective surfaces.” This is where Ashby uses the ScanStation P50, which he finds twice as fast as the other laser scanners he uses for shorter range work.

“The way I have the Leica Geosystems scanner setup is at a resolution of 1 point per 0.6 centimetres at 10 metres...you’re capturing more data in a lot less stations,” Explains Ashby. “The data from the ScanStation P50 is unbelievably clean – it’s really fast.”

Using LiDAR scanning to improve continuity

By creating a virtual 3D version of the filming environment, LiDAR scanning makes it easier to see the position of an object from different angles, and to make sure that things are where they are supposed to be. “What LiDAR really helps you do is, when you’re working on multiple shots from different camera angles, you can track everything to the LiDAR data and it snaps all the cameras into the same world space,” Ashby explains. “It means that if you put something in 3D space in one shot, it will be in the same 3D space in all the other shots.”

Using an example of a scene from Joker, Ashby talks about a scene with a lot of cuts. The scan helped the technicians to match CG blood from different angles, “What it means is that the blood in shot A lines up perfectly with the blood in shot B.”

Easy-to-use software is essential for checking high numbers of scans

The need to stop and start scanning can complicate the process of registering scans together. “All your scans can get really jumbled around. So a good logging system is key and then it’s much easier when you have to manually align your scans after,” clarifies Ashby.

Ashby uses Cyclone software for post-processing and likes the amount of control that it allows for manual alignment. “The Cyclone software package is very fast for that. You just overlay them [scans] on top, you quickly just line them up and that’s pretty much it…You can get through about 80 scans in about a couple of hours.”

In other industries like construction or manufacturing, a scan would be used to create a highly detailed 3D model of the environment, however, in the film industry an OBJ (polygonal mesh) is often the preferred output, containing all the detail that’s required; “OBJs are pretty unbreakable as a format and generally the most requested deliverable.”

Ashby’s job is then complete as he delivers the scans to others working on the production, “I typically hand off everything to production or a visual effects studio.” Having packed his laser scanners into flight cases, the LiDAR supervisor is ready to head off and use his Leica Geosystems laser scanner on the next set.

Based on: Lidar scanning for Joker and John Wick