[sdiy] Ray tracing hardware for audio simulation

[Mike Bryant]

About to upgrade to a new PC.  This might have influenced me to go nVidia, even though I never play games.


-----Original Message-----
From: Synth-diy [mailto:synth-diy-bounces at synth-diy.org] On Behalf Of cheater cheater via Synth-diy
Sent: 30 July 2022 16:48
To: synth-diy
Subject: [sdiy] Ray tracing hardware for audio simulation

Hi all,
there was recently an AMA with ray tracing experts at nvidia and I asked about uses for audio. I thought the possibilities could be interesting for some people here. Below is the link to the original thread as well as a copy of the question and answer.

Cheers


https://forums.developer.nvidia.com/t/meet-the-ray-tracing-gems-team-live-ama-july-28-2022/217920

-------------------------------------------------------------------

Welcome to our first Connect with Experts - AMA, an exclusive benefit of the NVIDIA Developer Program 12 The team behind the popular Ray Tracing Gems 23 will be here live on July 28, 2022 at 10am (PDT) to 11am

Eric Haines, Adam Marrs, Peter Shirley and Ingo Wald,

Post questions - watch the discussion - ask the team anything.

-------------------------------------------------------------------

Could you please talk about how ray tracing and RTX could help realistic sound in sound-oriented games, in the context of the gameplay of the original Thief game, to give a concrete example?
Effects such as realistic reverberation, sound occlusion, the carrying of sound by continuous surfaces, and others are all of benefit here.
Are there current approaches to such problems which will still allow ray tracing the graphics at the same time?

Here is an example of gameplay: https://youtu.be/4rWfb7ZtSPc?t=8855
(Thief Gold | 1080p60 | Longplay Full Game Walkthrough No Commentary -
YouTube)

-------------------------------------------------------------------

Answer from Ingo Wald (iwald):

First off: “IRL” sound is important to judge directions - you can absolutely hear what direction something is coming from - so simulating that better in a game should help make this game “feel”
more realistic, and better. On the technical side, sound transport and light transport are - conceptually - actually very similar; though there’s differences in “how” things reflect you still need frequent “line of sight” computations, which are exactly what ray tracing does
- so yes, having fast ray tracing should help in making sound simulation better/more accurate.

Eric notes two things: VRWorks - Audio | NVIDIA Developer is from NVIDIA and may be just what the poster wants. For more on research in the area, a good place to start might be “Guided Multiview Ray Tracing for Fast Auralization” by Micah Taylor, Anish Chandak, Qi Mo, Christian Lauterbach, Carl Schissler, and Dinesh Manocha, 2012

Response from Tony Scudiero:

There’s a good history of ray tracing in audio: there are a number of commercial products that use ray methods for generating synthetic room impulse response filters. RTX technology is actually very good for acoustic simulations, as the material interactions of sounds are usually modeled at a coarser granularity than interactions of material and light. Acoustic simulations tend to have simple shaders, making their performance fundamentally a function of ray-scene queries, which RTX accelerates quite well!

One of the fundamental challenges of ray tracing acoustic energy is that the wavelengths in question are about 1 million times longer than visible light. Wavelengths can be on the order of a meter, which is the same order of magnitude as many objects. The consequence is that many effects must be treated over a cross-sectional area of the
wavefront: the interaction of sound energy with a surface cannot be accurately modeled only at an infinitesimal point. That said, there has been some research on how these effects can be treated using ray tracing techniques. The ‘right’ approach usually depends on your
goals: accuracy or speed.

>From a technological perspective, there’s absolutely nothing standing in the way of writing a real-time acoustics simulation using ray tracing graphics APIs like DXR or VkRay to do sound propagation simulation in tandem with ray tracing graphics. The available ray-tracing power of current-generation GPUs should be able to handle a moderately complex acoustic simulation in tandem with graphics.
Depending on how the graphics rendering engine is designed, primary rays could be used for both purposes, further economizing the simulation. While this is perfectly possible, I’m not aware of anyone that has actually done this in one of the graphics APIs.

NVIDIA’s VRWorks Audio, which is a relatively simple acoustics simulation intended for interactive experiences, uses OptiX. Version
2.0 of that SDK can make use of RTX hardware when available.

_______________________________________________
Synth-diy mailing list
Synth-diy at synth-diy.org
http://synth-diy.org/mailman/listinfo/synth-diy
Selling or trading? Use marketplace at synth-diy.org