It's been a while since the last update so I wanted to give some news on how things are going with work towards the next release.
I've spent a lot of time working on getting the HLE audio code working on the Media Engine. I've been making steady progress, but it's been taking longer than I initially expected. Fortunately, I'm very close to getting all of the audio processing moved over to the ME - in fact I believe I have just one significant bug left to fix.
The issue seems to be a very odd synchronisation bug which causes the emulator to lock up when running the audio processing asynchronously. As with many of these types of bugs, it's proving quite hard to track down because as soon as I change the code to debug the problem, the issue goes away. A true Heisenbug :(
What's particularly annoying is that the bug is stopping me from measuring how much of a difference running the audio code on the ME makes. Hopefully I'll be able to fix the bug over the Christmas break and be able to publish some timings over the new year.
As part of this work, I've also been writing a general-purpose 'job manager', which coordinates batches of work between the main CPU and the ME. The idea is to build on top of J.F.'s MediaEngine.prx to provide a simple interface for queing up and dispatching work asynchronously. When a job is added to the queue, a flag indicates whether the job is suitable for running on the ME, or whether it should just be run asynchronously on the main CPU instead.
Initially just the audio processing will run through the job manager on the ME, but eventually it should be possible run other pieces of work asynchronously too. I'm hoping that it will eventually be possible to move parts of the HLE graphics processing to run asynchronously too, but I need to investigate things a bit more first. That's a job for future releases however.
Anyway, that's all for now. I'm off to eat mince pies and watch The Great Escape on tv. Merry Christmas everyone :)
-StrmnNrmn
Showing posts with label audio. Show all posts
Showing posts with label audio. Show all posts
Monday, December 24, 2007
Sunday, November 11, 2007
Media Engine
Earlier I discussed my plans for getting Daedalus's audio processing working on the PSP's Media Engine.
As I mentioned in that post, it's not just a case of changing some compiler setting to get this working. I've not spent much time investigating the ME so I may be wrong on a few of these points, but here are the current issues that I think need solving.
Firstly in order to access the ME I need to be running in kernel mode. This requires either running Daedalus in kernel mode, or (preferably) creating a kernel mode PRX that encapsulates the required functionality. I think kernel mode rules out anyone running with v1.50 firmware (hence my earlier post - please respond to the poll if you haven't already done so!) Maybe one of the more savvy psp developers out there can correct me on this? If no-one is using v1.50 any more then maybe it isn't even an issue.
Another problem is that although the ME is essentially the same processor as the main core, it has a different memory map. This means that things like the VRAM is invisible to the ME, so any code ported to run on the ME would have to be written to operate on main memory. This isn't an issue for Daedalus's audio list processing, but it would cause problems if I wanted to move display list processing to the ME too.
Touching on the memory map issue, another problem is the lack of cache coherency between the two cores. I need to be careful when accessing the same areas of memory with both cores to correctly flush and invalidate the data caches. Ideally any shared memory should be kept to a minimum, but this is easier said than done when porting existing code, rather than writing new code.
For a similar reason, any code which needs to run on the ME should avoid making any calls to the runtime library, including doing any system memory allocation. System calls are also ruled out. This is fairly easy to guarantee if you're writing new code, but again, it's a lot harder if you're porting existing code.
I think that's most of the issues from the hardware side. There are also a number of issues to be solved to do with the way that Daedalus handles audio and display list processing.
On the N64, the audio and display lists are processed asynchronously by the RSP coprocessor. In Daedalus, I can identify when these tasks are queued up for the RSP, intercept them, and process them synchronously (using high-level emulation rather than simulating the RSP execution directly).
The key thing here is that as far as the emulated N64 is concerned, audio and display list processing currently happens instantaneously. As soon as it kicks off the RSP it gets a interrupt to inform it that processing has completed. The whole process is very deterministic and I'm worried that by processing these display lists asynchronously on the ME that a number of intermittent and hard-to-debug issues will crop up. On the other hand, processing these tasks asynchronously is much closer to the behaviour of a real N64, which may fix some timing-related issues. It will also allow Daedalus to exploit the inherent parallelism that N64 roms were designed to take advantage of.
My current plan for ME audio support in R14 is:
So, that's the plan; I'll keep you updated on my progress. If anyone has any experience doing this kind of thing on the ME it would be great to hear your thoughts.
-StrmnNrmn
As I mentioned in that post, it's not just a case of changing some compiler setting to get this working. I've not spent much time investigating the ME so I may be wrong on a few of these points, but here are the current issues that I think need solving.
Firstly in order to access the ME I need to be running in kernel mode. This requires either running Daedalus in kernel mode, or (preferably) creating a kernel mode PRX that encapsulates the required functionality. I think kernel mode rules out anyone running with v1.50 firmware (hence my earlier post - please respond to the poll if you haven't already done so!) Maybe one of the more savvy psp developers out there can correct me on this? If no-one is using v1.50 any more then maybe it isn't even an issue.
Another problem is that although the ME is essentially the same processor as the main core, it has a different memory map. This means that things like the VRAM is invisible to the ME, so any code ported to run on the ME would have to be written to operate on main memory. This isn't an issue for Daedalus's audio list processing, but it would cause problems if I wanted to move display list processing to the ME too.
Touching on the memory map issue, another problem is the lack of cache coherency between the two cores. I need to be careful when accessing the same areas of memory with both cores to correctly flush and invalidate the data caches. Ideally any shared memory should be kept to a minimum, but this is easier said than done when porting existing code, rather than writing new code.
For a similar reason, any code which needs to run on the ME should avoid making any calls to the runtime library, including doing any system memory allocation. System calls are also ruled out. This is fairly easy to guarantee if you're writing new code, but again, it's a lot harder if you're porting existing code.
I think that's most of the issues from the hardware side. There are also a number of issues to be solved to do with the way that Daedalus handles audio and display list processing.
On the N64, the audio and display lists are processed asynchronously by the RSP coprocessor. In Daedalus, I can identify when these tasks are queued up for the RSP, intercept them, and process them synchronously (using high-level emulation rather than simulating the RSP execution directly).
The key thing here is that as far as the emulated N64 is concerned, audio and display list processing currently happens instantaneously. As soon as it kicks off the RSP it gets a interrupt to inform it that processing has completed. The whole process is very deterministic and I'm worried that by processing these display lists asynchronously on the ME that a number of intermittent and hard-to-debug issues will crop up. On the other hand, processing these tasks asynchronously is much closer to the behaviour of a real N64, which may fix some timing-related issues. It will also allow Daedalus to exploit the inherent parallelism that N64 roms were designed to take advantage of.
My current plan for ME audio support in R14 is:
- Create a kernel mode PRX and get Daedalus successfully loading and invoking functions (under all supported firmwares). I've just about done this.
- Add the code to support initialising and running code on the ME to the PRX. Test invoking user mode functions from the main EBOOT.PBP. I'll probably be using J.F.'s great sample code as a reference for this. Thanks J.F.!
- Rewrite the audio list processing code so that it can be invoked synchronously or asynchronously as required (via some kind of configuration option). When running asynchronously it can just be run from a separate high-priority thread to start with. I can use this to test for various synchronisation issues without going through the pain of trying to do this on the ME first.
- Audit the audio list processing code to minimise any memory accesses or ensure that they are correctly synchronised with the main core/thread. Any crt or system calls need to be eliminated or abstracted away (e.g. printfs NOP when compiled to run on the ME).
- Invoke audio list processing code from the ME.
- Cross fingers.
So, that's the plan; I'll keep you updated on my progress. If anyone has any experience doing this kind of thing on the ME it would be great to hear your thoughts.
-StrmnNrmn
Wednesday, February 14, 2007
Audio Support in Daedalus R9
Last week I said that I'd talk a bit about the audio support I've been working on. I want to describe how it was implemented, and continue with what to expect from it in R9 and how it affects performance.
I'd been putting off implementing audio support for a number of reasons. Firstly, I just didn't think that the emulator was running fast enough to spare any extra time doing other work. Secondly, because it wasn't running fast enough, I knew the resulting audio would sound horrible and choppy. Finally, I knew that writing an audio list processor from scratch was going to be very time consuming and error prone.
Maybe it's worth going into a little more detail about that last point.
Most audio and graphics processing on the N64 is handled through audio lists and display lists (very similar to an OpenGL display list). These lists are sequences of simple instructions which are usually constructed by the CPU as the game is running. I used the word 'usually', because it's possible to generate the lists offline, and just execute them at runtime as required.
One way of thinking about the audio and display lists is as a very simple scripting language. Each instruction in the list performs a trivial operation, like copying a block of sample data to another region of memory, interpolating samples from two buffers and storing the result in another and so on. The format of the instructions in the lists has been very carefully designed so that they can be efficiently operated on by the N64's coprocessor, the RSP. The RSP is a very fast processor with SIMD instructions (i.e. instructions which operate on multiple pieces of data - much like SSE on x86 based processors), and it's ideally suited to performing the kind of jobs required in processing audio data or transforming and lighting geometry. The code which executes on the RSP is often called 'microcode'.
The nice thing about audio and display lists is that they can be executed asynchronously by the RSP while the CPU is off doing other work. The CPU just needs to copy a small block of data (describing the task at hand) to the RSP's local memory, and then kick it off. The CPU then goes off doing some other work (updating the game logic for instance.) At some point, the RSP finishes processing the audio or display list, and signals the CPU that it's finished its work through an interrupt. The CPU can then queue up the next available job to the RSP for processing.
There are two ways in which an emulator can handle audio or display lists. The first method is to emulate the RSP at a very low level. This means emulating the RSP as a separate processor, emulating the microcode which has been written to decode and process the audio and display lists. This is what's known as LLE (low-level emulation). The other approach is HLE (high-level emulation). This involves interpreting and processing the audio and display lists manually, ignoring the emulation of the RSP and the microcode entirely.
The HLE approach is typically much, much faster than emulating the task at a low level. This speed comes as a price though, as rather than 'just' writing code to emulate the RSP core, a separate implementation must be written for every task that you want to emulate. This would be fairly simple if there was a single format for audio lists, and a single format for display lists. Unfortunately though, over the years Nintendo released a number of different versions of the microcode controlling these tasks, and many of these changes resulted in different formats for the audio lists and display lists.
Last time I checked, there were 3 major audio list formats, and 5 (6?) major display list formats. Some features of the list formats are similar between versions, but typically they're all very different, and large portions must be reverse engineered from scratch.
So, back to audio support in Daedalus. I had come to the point with implementing various dynarec improvements that I felt it was worth looking into audio to get an idea how how well it would work, and how much of a performance hit there was. As I was saying, one of the reasons I'd been putting off implementing audio support was that I knew that taking a high-level approach was going to be time consuming and error prone. The alternative, low level emulation, just wasn't practical from a performance point of view.
In the end, I decided to see if I could find the source for a PC audio plugin, and adapt this for use with Daedalus. I was fortunate enough to discover that Azimer had made the source for v0.55 of his audio plugin available (Azimer - if you're reading this, please drop me a line to say hello :) My reasoning was that if I could get something up and running in a couple of evenings then I could figure out how much work would be involved in getting it polished. If it looked like it was going to be too much work then I could revert all my changes and at most I'd have wasted just a few hours.
In the end it all went incredibly well. Azimer's source was very self-contained, with just a couple of hooks to implement to get it working. There were a few small issues to solve like having to upsample the output to 44.1KHz (as the PSP output is fixed at this frequency), but nothing that took more than a couple of hours.
So, how does it sound? Well, it's pretty variable. It sounds great in places where there's a high enough framerate to keep the audio buffer full. Mario 64 is pretty good for this - it seems 25fps is a minimum to avoid any choppiness in the output. When the framerate drops lower than this the audio quality does start to suffer significantly. On the plus side, it seems to be decoding everything correctly in all the roms I've tested so for, which means that the quality is only going to improve as I make further improvements to the performance of the emulator.
Which brings me to my next point - how does enabling audio support affect the performance of the emulator? I've not had chance to measure this empirically, but it probably slows things down by about 10-20%. Given that R9 is running about 40-100% faster than R8, you should be able to run R9 with audio and still see a small speedup. In any case, I've added an option to enable or disable the audio in the front-end (this is also accessible from the pause menu while the rom is running).
So, that's just about the story so far with audio support in Daedalus. There's clearly a lot more work to do, but the good news is that audio support is in, and working, and is only going to improve with time. Later this week I'll talk a bit about future improvements, including methods for reducing the impact that audio processing has on performance.
-StrmnNrmn
I'd been putting off implementing audio support for a number of reasons. Firstly, I just didn't think that the emulator was running fast enough to spare any extra time doing other work. Secondly, because it wasn't running fast enough, I knew the resulting audio would sound horrible and choppy. Finally, I knew that writing an audio list processor from scratch was going to be very time consuming and error prone.
Maybe it's worth going into a little more detail about that last point.
Most audio and graphics processing on the N64 is handled through audio lists and display lists (very similar to an OpenGL display list). These lists are sequences of simple instructions which are usually constructed by the CPU as the game is running. I used the word 'usually', because it's possible to generate the lists offline, and just execute them at runtime as required.
One way of thinking about the audio and display lists is as a very simple scripting language. Each instruction in the list performs a trivial operation, like copying a block of sample data to another region of memory, interpolating samples from two buffers and storing the result in another and so on. The format of the instructions in the lists has been very carefully designed so that they can be efficiently operated on by the N64's coprocessor, the RSP. The RSP is a very fast processor with SIMD instructions (i.e. instructions which operate on multiple pieces of data - much like SSE on x86 based processors), and it's ideally suited to performing the kind of jobs required in processing audio data or transforming and lighting geometry. The code which executes on the RSP is often called 'microcode'.
The nice thing about audio and display lists is that they can be executed asynchronously by the RSP while the CPU is off doing other work. The CPU just needs to copy a small block of data (describing the task at hand) to the RSP's local memory, and then kick it off. The CPU then goes off doing some other work (updating the game logic for instance.) At some point, the RSP finishes processing the audio or display list, and signals the CPU that it's finished its work through an interrupt. The CPU can then queue up the next available job to the RSP for processing.
There are two ways in which an emulator can handle audio or display lists. The first method is to emulate the RSP at a very low level. This means emulating the RSP as a separate processor, emulating the microcode which has been written to decode and process the audio and display lists. This is what's known as LLE (low-level emulation). The other approach is HLE (high-level emulation). This involves interpreting and processing the audio and display lists manually, ignoring the emulation of the RSP and the microcode entirely.
The HLE approach is typically much, much faster than emulating the task at a low level. This speed comes as a price though, as rather than 'just' writing code to emulate the RSP core, a separate implementation must be written for every task that you want to emulate. This would be fairly simple if there was a single format for audio lists, and a single format for display lists. Unfortunately though, over the years Nintendo released a number of different versions of the microcode controlling these tasks, and many of these changes resulted in different formats for the audio lists and display lists.
Last time I checked, there were 3 major audio list formats, and 5 (6?) major display list formats. Some features of the list formats are similar between versions, but typically they're all very different, and large portions must be reverse engineered from scratch.
So, back to audio support in Daedalus. I had come to the point with implementing various dynarec improvements that I felt it was worth looking into audio to get an idea how how well it would work, and how much of a performance hit there was. As I was saying, one of the reasons I'd been putting off implementing audio support was that I knew that taking a high-level approach was going to be time consuming and error prone. The alternative, low level emulation, just wasn't practical from a performance point of view.
In the end, I decided to see if I could find the source for a PC audio plugin, and adapt this for use with Daedalus. I was fortunate enough to discover that Azimer had made the source for v0.55 of his audio plugin available (Azimer - if you're reading this, please drop me a line to say hello :) My reasoning was that if I could get something up and running in a couple of evenings then I could figure out how much work would be involved in getting it polished. If it looked like it was going to be too much work then I could revert all my changes and at most I'd have wasted just a few hours.
In the end it all went incredibly well. Azimer's source was very self-contained, with just a couple of hooks to implement to get it working. There were a few small issues to solve like having to upsample the output to 44.1KHz (as the PSP output is fixed at this frequency), but nothing that took more than a couple of hours.
So, how does it sound? Well, it's pretty variable. It sounds great in places where there's a high enough framerate to keep the audio buffer full. Mario 64 is pretty good for this - it seems 25fps is a minimum to avoid any choppiness in the output. When the framerate drops lower than this the audio quality does start to suffer significantly. On the plus side, it seems to be decoding everything correctly in all the roms I've tested so for, which means that the quality is only going to improve as I make further improvements to the performance of the emulator.
Which brings me to my next point - how does enabling audio support affect the performance of the emulator? I've not had chance to measure this empirically, but it probably slows things down by about 10-20%. Given that R9 is running about 40-100% faster than R8, you should be able to run R9 with audio and still see a small speedup. In any case, I've added an option to enable or disable the audio in the front-end (this is also accessible from the pause menu while the rom is running).
So, that's just about the story so far with audio support in Daedalus. There's clearly a lot more work to do, but the good news is that audio support is in, and working, and is only going to improve with time. Later this week I'll talk a bit about future improvements, including methods for reducing the impact that audio processing has on performance.
-StrmnNrmn
Subscribe to:
Comments (Atom)