Audio Support in Daedalus R9

Last week I said that I'd talk a bit about the audio support I've been working on. I want to describe how it was implemented, and continue with what to expect from it in R9 and how it affects performance.

I'd been putting off implementing audio support for a number of reasons. Firstly, I just didn't think that the emulator was running fast enough to spare any extra time doing other work. Secondly, because it wasn't running fast enough, I knew the resulting audio would sound horrible and choppy. Finally, I knew that writing an audio list processor from scratch was going to be very time consuming and error prone.

Maybe it's worth going into a little more detail about that last point.

Most audio and graphics processing on the N64 is handled through audio lists and display lists (very similar to an OpenGL display list). These lists are sequences of simple instructions which are usually constructed by the CPU as the game is running. I used the word 'usually', because it's possible to generate the lists offline, and just execute them at runtime as required.

One way of thinking about the audio and display lists is as a very simple scripting language. Each instruction in the list performs a trivial operation, like copying a block of sample data to another region of memory, interpolating samples from two buffers and storing the result in another and so on. The format of the instructions in the lists has been very carefully designed so that they can be efficiently operated on by the N64's coprocessor, the RSP. The RSP is a very fast processor with SIMD instructions (i.e. instructions which operate on multiple pieces of data - much like SSE on x86 based processors), and it's ideally suited to performing the kind of jobs required in processing audio data or transforming and lighting geometry. The code which executes on the RSP is often called 'microcode'.

The nice thing about audio and display lists is that they can be executed asynchronously by the RSP while the CPU is off doing other work. The CPU just needs to copy a small block of data (describing the task at hand) to the RSP's local memory, and then kick it off. The CPU then goes off doing some other work (updating the game logic for instance.) At some point, the RSP finishes processing the audio or display list, and signals the CPU that it's finished its work through an interrupt. The CPU can then queue up the next available job to the RSP for processing.

There are two ways in which an emulator can handle audio or display lists. The first method is to emulate the RSP at a very low level. This means emulating the RSP as a separate processor, emulating the microcode which has been written to decode and process the audio and display lists. This is what's known as LLE (low-level emulation). The other approach is HLE (high-level emulation). This involves interpreting and processing the audio and display lists manually, ignoring the emulation of the RSP and the microcode entirely.

The HLE approach is typically much, much faster than emulating the task at a low level. This speed comes as a price though, as rather than 'just' writing code to emulate the RSP core, a separate implementation must be written for every task that you want to emulate. This would be fairly simple if there was a single format for audio lists, and a single format for display lists. Unfortunately though, over the years Nintendo released a number of different versions of the microcode controlling these tasks, and many of these changes resulted in different formats for the audio lists and display lists.

Last time I checked, there were 3 major audio list formats, and 5 (6?) major display list formats. Some features of the list formats are similar between versions, but typically they're all very different, and large portions must be reverse engineered from scratch.

So, back to audio support in Daedalus. I had come to the point with implementing various dynarec improvements that I felt it was worth looking into audio to get an idea how how well it would work, and how much of a performance hit there was. As I was saying, one of the reasons I'd been putting off implementing audio support was that I knew that taking a high-level approach was going to be time consuming and error prone. The alternative, low level emulation, just wasn't practical from a performance point of view.

In the end, I decided to see if I could find the source for a PC audio plugin, and adapt this for use with Daedalus. I was fortunate enough to discover that Azimer had made the source for v0.55 of his audio plugin available (Azimer - if you're reading this, please drop me a line to say hello :) My reasoning was that if I could get something up and running in a couple of evenings then I could figure out how much work would be involved in getting it polished. If it looked like it was going to be too much work then I could revert all my changes and at most I'd have wasted just a few hours.

In the end it all went incredibly well. Azimer's source was very self-contained, with just a couple of hooks to implement to get it working. There were a few small issues to solve like having to upsample the output to 44.1KHz (as the PSP output is fixed at this frequency), but nothing that took more than a couple of hours.

So, how does it sound? Well, it's pretty variable. It sounds great in places where there's a high enough framerate to keep the audio buffer full. Mario 64 is pretty good for this - it seems 25fps is a minimum to avoid any choppiness in the output. When the framerate drops lower than this the audio quality does start to suffer significantly. On the plus side, it seems to be decoding everything correctly in all the roms I've tested so for, which means that the quality is only going to improve as I make further improvements to the performance of the emulator.

Which brings me to my next point - how does enabling audio support affect the performance of the emulator? I've not had chance to measure this empirically, but it probably slows things down by about 10-20%. Given that R9 is running about 40-100% faster than R8, you should be able to run R9 with audio and still see a small speedup. In any case, I've added an option to enable or disable the audio in the front-end (this is also accessible from the pause menu while the rom is running).

So, that's just about the story so far with audio support in Daedalus. There's clearly a lot more work to do, but the good news is that audio support is in, and working, and is only going to improve with time. Later this week I'll talk a bit about future improvements, including methods for reducing the impact that audio processing has on performance.

-StrmnNrmn