StudentFilmmakers Magazine: Can you tell us a little bit about the evolution of sound design for VR and 360 filmmaking?
Michael Filimowicz: For 360 video, there’s the kind that you would play on a head mounted display (i.e. ‘goggles’), but there’s also immersive spaces like you find with micro theater spaces such as the Igloo Vision system, where, say, 5 projectors create an immersive space and the screen is a cylinder you’re standing inside of.
With the former, virtual audio tends to rely on head related transfer functions (HRTFs) which emulate the delay times of sounds reaching each ear at different parts of the wavefront’s phase. The problem with these algorithms is that everyone’s head is a different size and shape, and so you can get front/back confusions, or sounds feel like they’re inside your head. There are newer active headphone designs which take measurements of the listener’s head and try to customize the noise cancellation and virtual audio effects so that the response is more tuned to each listener’s head dimensions.
With the latter cylindrical micro theater 360 video spaces, there still tends to be a 5.1 surround sound approach where the speakers will be ringed around the screen and actually you can see the speakers which are usually placed high up toward the top. The spatial mapping effect isn’t that great in these kinds of systems, as sounds tend to feel ‘smeared’ across too large and area and so the sound/image associations are pretty weak.
My own research is based on trying to address these kinds of limitations by vibrating the screen and using software to place the sounds where the objects and events are in the displayed content. I call this ‘audiovisual colocation’ and I have a Udemy course where I teach people how they can build their own collocative displays in a kind of DIY manner.
Much of the sound design for 360 will be similar to regular films (e.g. diegetic vs non-diegetic sound, or on-screen vs. off-screen sound, etc.), only you have much more emphasis on attending to the spatial audio aspects and localizing sound in space becomes trickier and more a component of the overall aesthetic.
StudentFilmmakers Magazine: Is there an industry standard when it comes to capturing panoramic three-dimensional sound – like specific outdoor and studio microphones, recording devices and software?
Michael Filimowicz: Surround comes in many channel ‘flavors,’ e.g. 5.1, 7.1, even 22.2! That’s a Japanese standard for ultra-high definition television. But classically, in film ‘stereo’ means basically 4 channel audio, with the idea of a listener in the center. Thus, you find surround sound field recording equipment with capsule designs that capture 4 channels of the sound field around the mic. Since dialogue is usually panned center, a surround ambience recording might place the ambient audio in front left, front right, rear left, and rear right channels. So, just go with any of the 4 channel mic designs, which are sometimes something like two stereo mics at angles to each other but where you can get 4 individual audio tracks from the same mic.
I use a Soundfield SPS200 4 channel microphone with a 4 channel Edirol field recorder. There are a lot of smaller recorders though where the mic and recorder are integrated into a small package, e.g. the Zoom H2N has 5 different mics inside its mic mesh and it can record 4 channel surround sound in a palm sized unit.
For software, any modern DAW can do surround sound without a problem. For music I use FL Studio and for video mixing I use the Fairlight side of Davinci Resolve. I do more music these days but of course I used ProTools for, like, almost 20 years before I just got tired of Avid changing their licensing and subscription model constantly. Good riddance to my iLok USB dongle!
StudentFilmmakers Magazine: How is VR changing the film industry?
Michael Filimowicz: I think people are still generally trying to figure out what to do with it. The film industry is based on cramming hundreds of people into rows of seats in a theater. But VR is based on giving individuals a large spatial footprint to themselves for exploring a virtual space. VR runs completely counter to film’s business model and economic scene. Studios have done things like have VR booths in theater lobbies that give a kind of added experience to movie going, but it’s really not scalable and thus you can’t make serious money on that if it doesn’t scale.
I think you have to look at the home viewing space for where VR and filmmaking can really synergize, but that would mean you should be able to plug your head mounted display into a smart tv (or a streaming box, for instance I stream via an Apple TV), and Netflix, HBO or Amazon Prime or what have you would have to stream VR content to your living room and that hasn’t happened yet.
But, if we’re going to have VR filmmaking ‘for the masses’ it will have to be for homes and streaming boxes because it doesn’t make any sense in theaters. There are other kinds of VR spaces, like VR Arcades and in Seoul, South Korea, there’s even a whole mall of VR Arcades called VR Square, but that’s more for multi-player gaming and not so much for film.
StudentFilmmakers Magazine: If you could share your Top 3 VR Tips for aspiring filmmakers, what would they be?
(#1) The headsets you might pick for games aren’t the same as for video, so do some research and pick one with the right resolution specs for video. The models are constantly changing so ask people who really know the ins and outs of headsets for their advice. The Pico Goblin G24k is a great budget choice, up the price scale would be something like the Vive Pro Eye.
(#2) Same with cameras, the GoPro Max or Insta 360 have great quality, but a lot depends on your budget.
(#3) Most high-quality video NLEs can do 360 post-production, e.g. Davinci Resolve or Premiere.
These 3 tips are more on the technical side, but I think probably the most important tip is to have a concept where 360 makes sense. Doing 360 just for the sake of it probably isn’t a great idea. I would also recommend producing the same film BOTH for regular 2D and 360. Just like when you create mixes of a soundtrack that come in mono, stereo, 5.1 versions, or where there’s a Music and Effects track etc., think about concepts where you can produce a VR immersive version AND regular screen-based version.
The reason for this is that VR is still very niche and there’s just not a lot of audience there. You might get very disappointed at the very low numbers of eventual viewers. So, consider projects where you can still reach regular audiences with 2D and specialized audiences with your VR version, similar to how films produce IMAX versions for that specialized format. I wouldn’t do VR-only films unless it’s just a kind of calling card piece to show what you can do, OR if you have a paying client, e.g. in the corporate media space, maybe some large company will want a bespoke VR film and are willing to pay you for it. Right now the audience for VR film is too small so I’d suggest having a VR offering in tandem with a regular film format.
With VR games, it’s common to create a regular 2D screen version for the same reason– the market for VR is a lot smaller, so games are often made for both VR and regular display to capture both markets, with the latter one much larger.
Interview conducted by Jody Michelle Solis. Associate Publisher for StudentFilmmakers Magazine (www.studentfilmmakers.com), HD Pro Guide Magazine (www.hdproguide.com), and Sports Video Tech (www.sportsvideotech.com) Magazine. “Lifelines, not deadlines. Motion Arts. Fusion Everything.” If you’re in Dallas, sign up for Jody’s Yoga Class.