Wednesday, February 14, 2007

Audio Support in Daedalus R9

Last week I said that I'd talk a bit about the audio support I've been working on. I want to describe how it was implemented, and continue with what to expect from it in R9 and how it affects performance.

I'd been putting off implementing audio support for a number of reasons. Firstly, I just didn't think that the emulator was running fast enough to spare any extra time doing other work. Secondly, because it wasn't running fast enough, I knew the resulting audio would sound horrible and choppy. Finally, I knew that writing an audio list processor from scratch was going to be very time consuming and error prone.

Maybe it's worth going into a little more detail about that last point.

Most audio and graphics processing on the N64 is handled through audio lists and display lists (very similar to an OpenGL display list). These lists are sequences of simple instructions which are usually constructed by the CPU as the game is running. I used the word 'usually', because it's possible to generate the lists offline, and just execute them at runtime as required.

One way of thinking about the audio and display lists is as a very simple scripting language. Each instruction in the list performs a trivial operation, like copying a block of sample data to another region of memory, interpolating samples from two buffers and storing the result in another and so on. The format of the instructions in the lists has been very carefully designed so that they can be efficiently operated on by the N64's coprocessor, the RSP. The RSP is a very fast processor with SIMD instructions (i.e. instructions which operate on multiple pieces of data - much like SSE on x86 based processors), and it's ideally suited to performing the kind of jobs required in processing audio data or transforming and lighting geometry. The code which executes on the RSP is often called 'microcode'.

The nice thing about audio and display lists is that they can be executed asynchronously by the RSP while the CPU is off doing other work. The CPU just needs to copy a small block of data (describing the task at hand) to the RSP's local memory, and then kick it off. The CPU then goes off doing some other work (updating the game logic for instance.) At some point, the RSP finishes processing the audio or display list, and signals the CPU that it's finished its work through an interrupt. The CPU can then queue up the next available job to the RSP for processing.

There are two ways in which an emulator can handle audio or display lists. The first method is to emulate the RSP at a very low level. This means emulating the RSP as a separate processor, emulating the microcode which has been written to decode and process the audio and display lists. This is what's known as LLE (low-level emulation). The other approach is HLE (high-level emulation). This involves interpreting and processing the audio and display lists manually, ignoring the emulation of the RSP and the microcode entirely.

The HLE approach is typically much, much faster than emulating the task at a low level. This speed comes as a price though, as rather than 'just' writing code to emulate the RSP core, a separate implementation must be written for every task that you want to emulate. This would be fairly simple if there was a single format for audio lists, and a single format for display lists. Unfortunately though, over the years Nintendo released a number of different versions of the microcode controlling these tasks, and many of these changes resulted in different formats for the audio lists and display lists.

Last time I checked, there were 3 major audio list formats, and 5 (6?) major display list formats. Some features of the list formats are similar between versions, but typically they're all very different, and large portions must be reverse engineered from scratch.

So, back to audio support in Daedalus. I had come to the point with implementing various dynarec improvements that I felt it was worth looking into audio to get an idea how how well it would work, and how much of a performance hit there was. As I was saying, one of the reasons I'd been putting off implementing audio support was that I knew that taking a high-level approach was going to be time consuming and error prone. The alternative, low level emulation, just wasn't practical from a performance point of view.

In the end, I decided to see if I could find the source for a PC audio plugin, and adapt this for use with Daedalus. I was fortunate enough to discover that Azimer had made the source for v0.55 of his audio plugin available (Azimer - if you're reading this, please drop me a line to say hello :) My reasoning was that if I could get something up and running in a couple of evenings then I could figure out how much work would be involved in getting it polished. If it looked like it was going to be too much work then I could revert all my changes and at most I'd have wasted just a few hours.

In the end it all went incredibly well. Azimer's source was very self-contained, with just a couple of hooks to implement to get it working. There were a few small issues to solve like having to upsample the output to 44.1KHz (as the PSP output is fixed at this frequency), but nothing that took more than a couple of hours.

So, how does it sound? Well, it's pretty variable. It sounds great in places where there's a high enough framerate to keep the audio buffer full. Mario 64 is pretty good for this - it seems 25fps is a minimum to avoid any choppiness in the output. When the framerate drops lower than this the audio quality does start to suffer significantly. On the plus side, it seems to be decoding everything correctly in all the roms I've tested so for, which means that the quality is only going to improve as I make further improvements to the performance of the emulator.

Which brings me to my next point - how does enabling audio support affect the performance of the emulator? I've not had chance to measure this empirically, but it probably slows things down by about 10-20%. Given that R9 is running about 40-100% faster than R8, you should be able to run R9 with audio and still see a small speedup. In any case, I've added an option to enable or disable the audio in the front-end (this is also accessible from the pause menu while the rom is running).

So, that's just about the story so far with audio support in Daedalus. There's clearly a lot more work to do, but the good news is that audio support is in, and working, and is only going to improve with time. Later this week I'll talk a bit about future improvements, including methods for reducing the impact that audio processing has on performance.

-StrmnNrmn

36 comments:

Ian said...

can't wait!

You are a great coder, and a logical thinker!

Oh, and don't forget to thank the coder of audio support in your credits!

wally said...

Good going mate,

How loud is the audio output on the PSP?
(In the past I have noticed some emulators have not been loud enough to hear (even with headphones on) and it kinds puts people off. I mean if we were in a bus or something wouldnt you want to annoy the other people with your game sounds :P

Just checking

Wally

BrendanL said...

I can't wait till R9!

Zack said...

Sounds great man, one step at a time. Can't wait to see what you come up with in this release.

I've been playing starfox 64 on R8. It's really fun on R8, I can't wait to see it on R9.

Keep it up! :)

David said...

Hi StrmnNrmn, well, i just wanna say that you new emulator looks really great. L love that audio support, as well the 40%-100% speedup.

Thanks for do this emulator to our PSP, really, the N64 is one of the best nintendo's consoles.

Sorry about the bad english, I'm from Mexico and im really young.

Thanks for everything!!

Zodionic said...

This looks great! I'm realy quite suprised at how well things are going. For alot of previous updates we were only seeing 2 or 3 fps speed ups, then you took a break for a while and things seemed bleak, but this is amazing!!, I relay believe you can make this emulator the best it can be on the psp. :D
IM still counting the days untill i can hear mario say "ya!, ho! , yahoo!" on my SONY psp ;P
Goodluck and best wishes mate.

Mario Kart God said...

You are still releasing it before the 25th still, right.?I would really like if you could give a real set release date to keep me from looking at this site every 2 minutes. thanks, keep up the good work!

PS: Does the audio sound OK in Mario Kart???

Nick said...

Yay! Technical post! I didn't understand most of it, but yay anyway!

Morgan said...

Sounds great (pun intended) StrmnNrmn looks like the break has done you well, I can't wiat for R9. And about the poll you do after R9 releases I hope people vote for compatability, because I think game such as Diddy Kong Racing, Goldeneye, and Super Smash Brothers add alot of prestege to your emulator if you can run them. Anyway glad to see you posting daily to keep us happy, keep up the great work!

CaptainTTK said...

cmon release it already stop teasing us

pkmaximum said...

Great work Strmmmmn I was kinda surprised you did not talk about implementing sound into the ME at all. But then again you already talked a bit about this in past comments, from other people. So I bet you can implement sound that way without getting a loss in performance as far as speed goes.

Don't answer this if its uncomfortable, but do you have any other ideas on how you are going to optomize the emulator to get significant speed ups, of maybe 3-5 more fps, because right now the speeds are great, especially for Super Mario 64, and a few more fps speed ups and the emulator will be almost full speed for those games.

I saw the speed test on Zelda Ocarina of time, (once again don't answer this if it comes off as rude or anything of the sort.) But do you think Zelda will really ever get to a playable state of at least 25 fps even w/out sound? Or any other N64 games of 32 mb in size?

Well once again great work Strmmmn and I love how you have been keeping us constantly updated lately =]

Nolan said...

storman norman cant wait for R9

Rom said...

*drum rolls* in my Head!! Ahhhh I cant take it *laughs*

Good Job Daedalus and please announce your release date >_<

Ben said...

I cant wait for this. one of the main fixed in r9 that I will love is actually the mirrored texture support. Finnally mario's shadow wont be a half circle, and the stars wont be half gold half black!

and of course, the Audio support alone is enough to make me ... I dunno.. something happy.

thanks so much for the continued work on Daedalus!

samuel_munro said...

u rock strmnnrmn!
I really cannot wait for this to come out...
You see, I'm an aspiring programmer and was wondering if you could release the source code with Daedalus R9?
It would really help me with trying to understand hoe to do C programming.
Thanks and good luck with ironing out those bugs.
E-Mail me at samuel_munro@hotmail.com

zelda_fan said...

How is the sound for zelda, because during gameplay zelda's top frame speed is 20.

Also are you using the ME for sound?

Mario Kart God said...

Sorry for me being so impatient. I just think about this emu all the time and want it so badely.

Enigma said...

Cant wait til r9!!!

This is a bit off topic, do you think there is possibly a way to get gameboy saves from a rom to load through this like on pokemon stadium? Cause it would just be awesome to play gold and silver on the psp and transfer them to this emulator. just a thought not expecting it at all really but it would be cool.

Dave said...

Glad to see that you're chronicling the trials and tribulations of trying to understand the system and adapt a fully operational (and faithful) emulator given the restrictions of the PSP. It'll be fantastic to see what you release soon. Keep it up.

Stefan said...

I just wanted to say "thank you" for putting so much time and effort in this project and I really appreciate the detailed information you give us about your current progress. I always like to know "how things work" and you're explaining all those technical terms in a way that everyone understands.

Keep up the good work!

XioN said...

Just a question: How would LLE go exactly at emulating and sound, also could LLE achieve decent speeds on the PSP?

White said...

how about if you could play your mp3s when the audio is turned off? would this work?

Nico said...

Hi StrmnNrmn!

I'm just amazed with your work. I must confess I don't know much about coding but I read your improvements on the emulator and find them interesting... but anyway I have a question

Looks like with this new release you've gained a BIG speedup with "Much better memory access handling in dynamically recompiled code" and some small speedups with some fixes. Since sound emulation seems to be going fine as long as the framerate is ok... where do you think you can gain more improvements on speedup for later releases? is there any parts of code to polish or any ideas you're leaving for later on? Just to know your planning ;)

It's great to hear from you so often lately. Keep it up!

Thanks,

Nico

king said...

I think it's really great that you took the time to blog your progress. It's a fascinating read.

Great work! Keep it up!

Poohter said...

you rock. period.

Jeroen said...

Hey StrmnNrmn,

Thanks again, this might take the dust from my psp again!.. i just can't wait and course hope to see it soon..

And if we don't hear from you, ENJOY THE HOLIDAY!

Mike said...

Incredible coder, and pretty clean writer too, apparently.

I am definately looking forward to this release. You're a boon to the PSP homebrew scene, that's for sure.

pimbong said...

strmnnrmn i have a question. will we be able to disable the adio so that the roms that dont run full speed can run a littel faster and not have horible sound quality?

P.S r9 sounds amazing i cant wait for it to come out :)

Josh said...

I am so excited!! Thanks for all the work, even though some of thought you fell off the face of the earth :P

Keep up the work, if you ever need help testing or anything, you know you have a lot of support.

Gotwake424 said...

HMM... well all i have to says is great work and i hope you release soon. Also i no that you gave us a little preview of the ingame menu, but what all can you do with it like change screen ratio, audio, rom, reboot, and other stuff like that. Oh and nice to have you back.


Peace- Nrmn

Paul said...

First, great job on the emulator, and I cant wait till it is running like the games where on the N64 (or close) or better!

I would also like to add, that having audio support makes the games twice as good, but many people would agree, playing the game at full speed, is usually better than hearing the audio (depending of course). Anyway great work so far on the emulator, cant wait to see and play R9!

Morgan said...

Nice to see all the hype back on Daedalus, the PSX stuff was gettting kind of old and then N64 pops right back in. But to my point, I've just read all the posts so far, and I have something to say. People read StrmnNrmn's post most of the simple questions you asked were IN HIS POST. It's just a little annoying to see people ask questions that have already been answered. So please guys try to read all of his posts so you don't have to stumble StrmnNrmn with questions he already has answered.

Wraggster said...

Great news. im looking forward to this

David said...

Can't wait 'till R9!!!

Neil said...

Hi man R9 will be awesome!!! my english ins't so good because i'm from venezuela but i want you to know that you're work is fantastic and we love it!

nils petter said...

how about if you work with the code of project 64!google it