Thursday, April 05, 2007

R10 Reception

I just wanted to say a quick thank you for all the feedback I've received on the R10 release. I'd been looking at it as quite a small update, but people seem to be very happy about the incremental speedup and new features such as frameskip and so on. I found releasing R10 within 4-5 weeks of R9 was really hard work, but I'm now convinced that releasing smaller updates more regularly is the right way to go. I'll be following this post with some details about the plans for R11.

Finally, it turns out that this blog had over 40,000 visits over Sunday/Monday, which is incredible - thanks for continuing to read :)

-StrmnNrmn

Monday, April 02, 2007

Daedalus R10

This is a few hours later than I'd hoped, but here's R10:

Daedalus PSP R10 for v1.00 Firmware
Daedalus PSP R10 for v1.50+ Firmware
Daedalus PSP R10 Source

This is the changelist:

[+] Added frameskip option.
[+] Added framerate limiting option.
[+] Added adjustable stick deadzone.
[+] Allow pause menu to be activated even when rom emulation has hung.
[!] Fix crash when no roms found, added explanatory message.
[!] Fix flickering when small number of roms found.
[^] Fix texture hash frequency check.
[^] Finally got code compiling with -O3 optimisation flag in GCC.
[^] Use VFPU for InvSqrt (thanks hlide!)
[^] Improved VFPU code for clipping and tnl (thanks Raphael!)
[^] Improved inlining of AddTri.
[^] Reduced time spent searching for overridden blend modes.
[~] Remove debug console in public release builds (smaller footprint and slight speedup.)


I'll post tomorrow with details on what I want to work on for R11. In the meantime, enjoy!

-StrmnNrmn

Thursday, March 29, 2007

R10 Countdown

I'm in the process of tidying up my current build so that I can release R10 over the next few days.

There are a number of features/optimisations still on my TODO list, but as I promised last month, I'd like to release R10 by the end of March. I think frequent, small updates are better than keeping everyone waiting months between releases.

Although R10 will be a smaller update than R9, there are some great improvements (most of which I've already talked about):


  • An approximate 10-15% speedup
  • Frameskip, framerate limiting and stick deadzone tuning
  • Various small bugfixes


I'm hoping to have everything ready by Sunday afternoon at the latest.

One feature which might not make it is the Expansion Pak support. I mentioned that I'd finally fixed the bug that was preventing this from working, but I've been having difficulty resolving the various memory issues that it causes when enabled. Rather than delay R10, I'd prefer to disable the Expansion Pak support for now and release R11 early - as soon as I've fixed the underlying problem. I'll keep you posted.

-StrmnNrmn

Wednesday, March 21, 2007

Frameskip

A couple of people have been commenting about the mysterious frameskip version of Daedalus R9 which appeared a short while ago. I'm not going to link to it because I can't verify where it came from. That said, I've not checked my email for a week so maybe the author has emailed me about it :)

Anyway, it just so happens that I implemented frameskip in R10 on Sunday, so expect this to be a supported feature in the next official release. I had been planning to add this to R9, but I forgot :) It's no big deal - it's about 20 lines of code.

It does give a slight speedup, but not always as much as you'd expect. For instance, skipping every other frame won't double the framerate, as not all the time is spent rendering. Paradoxically, it tends to have more of an effect on roms that are already running fairly fast. Hopefully for some roms it will make the different between them being barely playable and playable though.

There are a few other things people have been asking about which I will implement for R10 too:


  • A configurable deadzone for the stick
  • A configurable framerate limiter


-StrmnNrmn

Tuesday, March 20, 2007

A look inside Daedalus

(This is quite technical post, probably only of interest to other C++/PSP developers.)

Since I fixed the issue stopping the Expansion Pak support from working, I've been testing daedalus with the feature enabled to see if the added pressure on available RAM causes any new issues. Most roms that I've been tested have been running perfectly, but I've been experiencing occasional crashes through running out of memory. I believe this is due to a small leak somewhere, so in an effort to track it down I've been improving the tools I use to track memory usage.

In the improved tracker I override the global new and delete operators, which lets me perform some logging on every allocation/deallocation made during the course of executing the emulator (this isn't quite true, as I don't currently log calls to malloc etc, but it's good enough for my purposes). At the start of each overriden implementation, I keep track of the calling function's return address with the following snippet of code:


u32 ra;
asm volatile
(
"sw $ra, %0\n"
: "+m"(ra) : : "memory"
);


I log this return address along with the allocation size and the returned pointer for every call to new and new[]. For calls to delete and delete[], I just log the address of the memory being freed and the caller's address. This data is all logged to disk across the USB connection using PSPLink.

The logfiles look a little like this:


Allocating 36 bytes - 09c40620 - RA is 08953a94
Allocating[] 8192 bytes - 09c40a00 - RA is 0895e8bc
Allocating 12 bytes - 09c3fce0 - RA is 08964898
Allocating 12 bytes - 09c40970 - RA is 089648b0
Allocating 12 bytes - 09c409d0 - RA is 089648c8
Allocating 12 bytes - 09c408e0 - RA is 089648e0
Allocating 20 bytes - 091ceb20 - RA is 089645e8
Freeing 09c3fce0 - RA is 08962374
Freeing 09c40970 - RA is 089621fc


In order to analyse the results I've written a PC tool which scans through the logfile one line at a time 'replaying' the allocations and deallocations in the same order that they occured on the PSP. The analyser keeps track of the current state of allocations at any point in time, matching up any calls to delete with the corresponding call to new. This means that at any given time the tool has a complete list of all outstanding allocations.

I can discover any memory leaks by shutting down the emulator and continuing to record the logfile while it frees up all allocated resources. By replaying the logfile the analyser can identify any leaks, as these are (mostly) the only remaining allocations at the end of the logfile. I can then run the corresponding return addresses through psp-addr2line to discover where the leaked memory is being allocated from.

The other cool feature of the tool is that it builds up a graphical representation of the state of memory allocations at any point in time. This is really useful for figuring out where all the available RAM is going.

Here's a picture showing where most of the PSP's memory is being used while emulating Mario 64 (I've added the labels on by hand, the tool doesn't do that :)



Each pixel corresponds to 16 bytes of RAM. The smallest blocks are 64x64x16 bytes = 64KB. 1MB chunks are formed from 4x4 64KB blocks. The black space corresponds to both unallocated memory, and also memory allocated outside of the tracker (e.g. calls to malloc, memory used by PSPLink, the CRT, static data areas etc.) You can see that the "Emulated RAM" accounts for just over 8MB - this is because I enabled the Expansion Pak while testing. Dynarec currently uses about 6MB - Ideally I'd like to reduce this down to around 4MB soon.

You'll notice that almost all the memory Daedalus uses is for these big fixed-size allocations. What little dynamic allocation it does at runtime is limited to:


  • Keeping track of hot-trace hit counts in the Dynamo implementation
  • Textures and texture caching


As it turns out, the out-of-memory issues I've been having have been due to the texture cache going crazy and chewing up around 3MB of memory (typically it just uses 200-300KB or so). I've not figured out the root cause yet, but the tool has helped point me in the right direction.

All in all I think this is a pretty nifty utility as it stands, but I've been thinking about a few features that would make it even better:


  • Log the time and current frame alongside each allocation/deallocation. The analyser can then use this information to see how much 'churn' there is over time. Minimising this should help improve performance and reduce fragmentation.
  • Every memory allocation has a small housekeeping overhead (for alignment, keeping track of the allocation size etc). The tool could generate a histogram of allocation sizes to demonstrate how much memory is being wasted through tiny allocations, and give some indication of where pooling or freelists might help.


These are probably features for some point down the line however :)

That pretty much sums up the tool. If this code (either for the tracker or the logfile analyser) would be useful to anyone, let me know and I'll add it to Subversion alongside the rest of the Daedalus code with R10.

StrmnNrmn

Sunday, March 18, 2007

Daedalus R10 Optimisation Progress

I didn't mean to leave it quite so long since last weekend's update, but I've been working hard on a number of optmisations for R10. Oddly enough these are mostly new issues that I've found - most of them don't exist in the list of tasks I came up with a couple of weeks ago. I think that shows how much scope there is for optimising Daedalus!

Firstly, I finally managed to get Daedalus compiling with GCC's '-O3' setting. This flag turns on all of the optimisations that GCC provides. When I've tried to enable this flag in the past I've had numerous strange crashes and odd behaviour, so all releases of Daedalus to date have been compiled with -O1.

I updated my local installation of the PSPSDK last weekend and decided to try the -O3 setting again. I was pleased to find that Daedalus ran without crashing, but there was still some odd behaviour which I eventually tracked down to my use of the famous InvSqrt function. You can read a bit more about my findings on the pspdev forums.

Enabling -O3 tends to slightly increase the code size (the EBOOT.PBP has increased from around 850KB to 900KB), but the speedup is quite noticable - my estimate is that Daedalus runs around 5% faster with -O3 over -O1.

As a result of the thread I started on the pspdev forums, hlide and Raphael both came up with some great suggestions for how I could optimise my use of the VFPU.

When I originally wrote the VFPU code for TnL and clipping there were still many undocumented/unsupported functions. A few months down the line and hlide and co have discovered a couple of instructions which are perfect for my needs - namely vuc2i and vc2i. These two functions take a 32-bit value comprising of 4 (un)signed 8-bit chars and unpack them into a vector of 4 32-bit fixed point numbers. It turns out that these instructions are perfect for converting the N64's packed colour and normal values into a format I can use in the VFPU code.

The various VFPU tweaks I've made have given Daedalus another 5% or so speedup.

The final set of changes I've been working on this week have been to do with how I handle certain blend modes. Some of the N64 blend modes are too complex for the PSP to deal with precisely, so I have a large table of 'override' blend modes which allow me to make as good an approximation of the N64 mode as possible. It turned out that looking up these blend modes was very expensive, so I've rewritten how this is handled to make it more efficient. The end result is another small speedup.

Overall these three changes give a combined 10-15% speedup on the various games I've tested, although there are roms that lie outside this range (some show an even greater speedup while others are more or less unaffected by the changes).

There's still quite a lot more in the way of optimisations that I want to get in for Daedalus R10 (mostly stuff I mentioned earlier) so hopefully these numbers will improve even further over the next couple of weeks.

-StrmnNrmn

Saturday, March 10, 2007

Weekend update

It's been a busy week at work, what with catching up after my week off and GDC, so I've not managed to post as many updates as I'd have liked.

On Daedalus I've been starting to take a look at the list of potential optimisations I listed and working out what to tackle first. To help me do this my first job is to do some work on Daedalus's profiler, to try and figure out where the biggest wins are going to come from. Hopefully I'll be able to report back with some interesting findings this weekend.

On a related note, I've spent the morning looking at converting the source control I'm using at sourceforge from CVS to Subversion. I've been meaning to do this for some time. I've never really been a fan of CVS, and as I'm using Subversion for other projects at work and at home I thought it made sense to migrate Daedalus over too.

So you can now access the latest Daedalus source* through Subversion:


svn co https://daedalus-n64.svn.sourceforge.net/svnroot/daedalus-n64/trunk daedalus-n64


With CVS I usually only updated the source alongside every release. Ideally the repository would contain an up-to-date copy of my local build, but I've had problems in the past where people have distributed 'intermediate' builds of Daedalus PSP, bugs and all. I only ever release new builds when I think there are enough new features and its stable enough to make it worthwhile for people to download and install; updating a source a bit less frequently gives me a bit more control and helps prevent everyone's time being wasted with intermediate builds. I think that I'm going to continue with this policy for the time being. We'll see how it goes.

-StrmnNrmn

*This is still just the R8 source which I lifted from CVS today. I'm in the process of testing whether this compiles OK, then I'll refresh the repository with all the changes from R9. I'll update this post when the R9 source is available.

Edit: R9 source commited to Subversion, all seems to be compiling OK.

Tuesday, March 06, 2007

R10 Plan of Action

Before I went away on holiday I asked you what you thought I should look at working on for the next release of Daedalus. Over 200 of you replied, and I've greatly enjoyed reading what you've had to say. There were some brilliant suggestions, so many thanks for your contributions.

It seems pretty clear to me that speed is the single biggest issue that most people want to see addressed. Many people also mentioned compatibility and savestate or save game support, but in nowhere near the same kind of numbers as those wanting speed improvements.

Based on your feedback my current plan is to release Daedalus R10 at the end of March, focusing mostly on speed improvements. If I can fit in any easy compatibility fixes, I'll do this too*.

Several people have asked what possibilities remain for optimisation. Here's a short list of things I know need more work:
  • In many games, a lot of the time spent executing dynamically recompiled code is doing things which can potentially be emulated at a high level. For instance, over 5% of the time spent executing dynarec code in Mario64 is just converting matrices from floating point to fixed point format. Another 4-5% of the time is spent in a loop invalidating areas of the data cache (which is irrelevent in an emulator.)
  • Some of the most expensive fragments are those which branch to themselves (i.e. those doing many loops). I can optimise for this to avoid loading and flushing cached registers on each iteration through the loop.
  • I can implement a frameskip option (I had intended to implement this for R9, but forgot!)
  • I can make use of the Media Engine (as Exophase suggested in conversation, as the ME can't access VRAM, it might make more sense to execute Audio and Display Lists on the main CPU, and run the N64 CPU emulation on the PSP ME)
  • There are certain situations where I fail to create fragments in the dynamic recompiler - for instance if the code being recompiled writes to a hardware register, this triggers an interrupt and causes fragment generation to be aborted. I should be able to deal with situations such as this more gracefully.
  • The fragment generator can do a lot more to improve register caching, and eliminating redundant 64-bit operations.
  • There are many situations where N64 roms busy wait. I detect very simple occurances of this, but not all of them. If I manually identify more complex examples I can have the fragment generator optimise them away.
  • Some roms are causing the dynarec fragment cache to be repeatedly dumped and recreated (I think Banjo Kazooie is one example of this). Fixing this may just involve tweaking a couple of magic numbers.
  • I currently optimise memory accesses under the assumption that most accesses are in the range 0x80000000 - 0x80800000, which is incorrect in the case of roms that make heavy use of virtual memory, or access RAM through the mirrored range at 0xa0000000. I can improve the trace recorder to collect information on which range a memory access fell in, and generate code to speculatively optimise for this.
  • Now that the dynarec engine is producing much better code, the cost of display list processing is becoming more significant, and may finally be worth profiling and optimising.
That's quite a big list, so I doubt I'll be able to work on these things before the end of March, but I think it shows there's still a lot of scope for further optimisation.

-StrmnNrmn

*Just this morning, I figured out why the Expansion Pak support was broken, so Majora's Mask and a couple of other games relying on this are booting correctly now :)

Sunday, March 04, 2007

Back!

Just a quick post to say that I'm back safe and well from skiing. It was awesome! As an added bonus I made it through the week without breaking any bones :)

Lots of people posted with their thoughts on what I should work on for R10 - many thanks for all your feedback. I'll post an update with my plans shortly (probably sometime tomorrow, after I've recovered from the travelling.)

-StrmnNrmn

Thursday, February 22, 2007

Please please please

If you're referencing posts on this site on your website/forums etc, please make some effort to quote relevant parts of the post rather than copying the entire text verbatim onto your site.

I've seen the entire body of the previous post quoted in its entirety on a couple of forums, including the line "Please reply to this post with your opinions on what I should work on next." Visitors to those sites are getting confused and posting their suggestions on the external forum, rather than on this blog. Unfortunately it's very unlikely that I'll read or be able to respond to people's comments on other sites.

Thanks!
StrmnNrmn

Plans for R10

Now that R9 is finally available, I'm ready to start making plans for R10. Firstly, I really don't want to leave it so long between releases again. The longer I leave between releases, the more hype builds up, and the harder it is to meet people's expectations. I prefer the idea of doing fairly frequent, incremental releases rather than relasing versions with huge sets of changes periodically.

So I'm currently planning to release R10 sometime around the end of March. The question is, what should I focus on?

The areas which clearly need lots of work are speed and compatibility. Unfortunately it's a bit of a chicken-and-egg situation - if the emulator runs slowly, it doesn't matter if certains roms don't run, because they'd still be unplayable. On the other hand, there's no point in having an ultra-fast emulator if it doesn't run the games people want to play. As I see it, I have to keep working on both of these things - in the long term, it doesn't make sense to concentrate on one and not the other.

I think the best approach is to alternate on work between the two areas. R9 saw significant speed improvements, but almost no progress with compatibility, so I'm currently leaning towards trying to get a few of the more popular titles running. But you're the ones who are going to be using the emulator, so I want to hear your views on what I should work on next.

Please reply to this post with your opinions on what I should work on next. Have a good play with R9 to see what's changed - maybe some things are working better than you expected. Maybe some things are no longer working? Here are a few suggestions:

  • Speed. Make it faster generally? Is there a specific game that runs slowly which you'd like to see an improvement with?
  • Compatibility. Should I work on FlashRAM support? Is there a specific game that doesn't boot, or crashes the emulator? Does your favourite game boot, but then crash when you try to start playing?
  • Audio. The audio support is quite new. Are there certain games where it doesn't work? Should I try to improve the audio so it's less choppy?
  • Graphics. Maybe your favourite game runs, but there are significant graphical glitches?
  • Save Games. Save game support is currently 'in' for EEPROM and MemPak based saves, but very flaky. Should I work on improving this so you don't lose your hard-earned progress?
  • Savestate. Would it be better for me to implement savestate support, so you can save an resume a game at any point?
As I mentioned earlier, I'm going to be away all next week, but I'll try to post as many comments as I can before then. When I get back I'll work through the comments and announce what I'm going to concentrate on for R10

-StrmnNrmn

Daedalus R9 - Rejoice!

I've been working hard making the last few changes for Daedalus R9, and I'm pleased to say that it's now ready for download:

Daedalus PSP R9 for v1.00 Firmware
Daedalus PSP R9 for v1.50+ Firmware
Daedalus PSP R9 Extras (a few preview pictures for the UI etc)
Daedalus PSP R9 Source

I've talked a bit about what's in this release, so I'll not repeat everything here. Here's a summary though:

* Faster
* New UI
* Audio
* Various graphical fixes

Please note - as audio support is very much 'beta' at the moment, it's disabled by default. You can enable the setting either in the front-end, or from the pause menu.

Please also note: To keep the distribution size down, I've not included any 'preview' pictures for the rom selector alongside the emulator. You can download and extract the 'Extras' file above for a few sample previews. It should be easy enough for people to create their own preview pictures - see the readme.txt for more details.

That's about it for now. Please post comments with any immediate feedback. I'm going to be a bit busy for a few days (and then I'm off skiing on Sunday for a week), but I'll try to reply to as many questions as I can in the meantime. I'll also try to get the ball rolling with my initial thoughts on stuff I want to do for R10.

[Edit 1:07AM - You may need to wait a few minutes for sourceforge's mirrors to synchronise. Sorry :) ]

Enjoy,
-StrmnNrmn

Sunday, February 18, 2007

R9 Status Update

It's been a couple of days since my last post, so I wanted to give a quick update on what's left on my R9 TODO list.
  • I've been working over the weekend at fixing a number of outstanding graphical glitches, putting in place a system to automatically translate n64 blend modes to the psp. This should hopefully nearly eradicate nearly all of the purple and black textures that have shown up all over the place in previous releases. This is quite an interesting feature from a technical point of view, so I'll talk about this in a bit more depth once R9 is out.
  • I have a few glitches in the new UI to fix (e.g. audio can't be disabled once the rom has started, controller can't be remapped once the rom has started).
  • I need to hook up a few things to the new UI (e.g. 'reset to main menu', 'take screenshot' etc)
As you can see, it's quite a short list now, so I'm still on track for releasing R9 by next weekend :)

-StrmnNrmn

Wednesday, February 14, 2007

Audio Support in Daedalus R9

Last week I said that I'd talk a bit about the audio support I've been working on. I want to describe how it was implemented, and continue with what to expect from it in R9 and how it affects performance.

I'd been putting off implementing audio support for a number of reasons. Firstly, I just didn't think that the emulator was running fast enough to spare any extra time doing other work. Secondly, because it wasn't running fast enough, I knew the resulting audio would sound horrible and choppy. Finally, I knew that writing an audio list processor from scratch was going to be very time consuming and error prone.

Maybe it's worth going into a little more detail about that last point.

Most audio and graphics processing on the N64 is handled through audio lists and display lists (very similar to an OpenGL display list). These lists are sequences of simple instructions which are usually constructed by the CPU as the game is running. I used the word 'usually', because it's possible to generate the lists offline, and just execute them at runtime as required.

One way of thinking about the audio and display lists is as a very simple scripting language. Each instruction in the list performs a trivial operation, like copying a block of sample data to another region of memory, interpolating samples from two buffers and storing the result in another and so on. The format of the instructions in the lists has been very carefully designed so that they can be efficiently operated on by the N64's coprocessor, the RSP. The RSP is a very fast processor with SIMD instructions (i.e. instructions which operate on multiple pieces of data - much like SSE on x86 based processors), and it's ideally suited to performing the kind of jobs required in processing audio data or transforming and lighting geometry. The code which executes on the RSP is often called 'microcode'.

The nice thing about audio and display lists is that they can be executed asynchronously by the RSP while the CPU is off doing other work. The CPU just needs to copy a small block of data (describing the task at hand) to the RSP's local memory, and then kick it off. The CPU then goes off doing some other work (updating the game logic for instance.) At some point, the RSP finishes processing the audio or display list, and signals the CPU that it's finished its work through an interrupt. The CPU can then queue up the next available job to the RSP for processing.

There are two ways in which an emulator can handle audio or display lists. The first method is to emulate the RSP at a very low level. This means emulating the RSP as a separate processor, emulating the microcode which has been written to decode and process the audio and display lists. This is what's known as LLE (low-level emulation). The other approach is HLE (high-level emulation). This involves interpreting and processing the audio and display lists manually, ignoring the emulation of the RSP and the microcode entirely.

The HLE approach is typically much, much faster than emulating the task at a low level. This speed comes as a price though, as rather than 'just' writing code to emulate the RSP core, a separate implementation must be written for every task that you want to emulate. This would be fairly simple if there was a single format for audio lists, and a single format for display lists. Unfortunately though, over the years Nintendo released a number of different versions of the microcode controlling these tasks, and many of these changes resulted in different formats for the audio lists and display lists.

Last time I checked, there were 3 major audio list formats, and 5 (6?) major display list formats. Some features of the list formats are similar between versions, but typically they're all very different, and large portions must be reverse engineered from scratch.

So, back to audio support in Daedalus. I had come to the point with implementing various dynarec improvements that I felt it was worth looking into audio to get an idea how how well it would work, and how much of a performance hit there was. As I was saying, one of the reasons I'd been putting off implementing audio support was that I knew that taking a high-level approach was going to be time consuming and error prone. The alternative, low level emulation, just wasn't practical from a performance point of view.

In the end, I decided to see if I could find the source for a PC audio plugin, and adapt this for use with Daedalus. I was fortunate enough to discover that Azimer had made the source for v0.55 of his audio plugin available (Azimer - if you're reading this, please drop me a line to say hello :) My reasoning was that if I could get something up and running in a couple of evenings then I could figure out how much work would be involved in getting it polished. If it looked like it was going to be too much work then I could revert all my changes and at most I'd have wasted just a few hours.

In the end it all went incredibly well. Azimer's source was very self-contained, with just a couple of hooks to implement to get it working. There were a few small issues to solve like having to upsample the output to 44.1KHz (as the PSP output is fixed at this frequency), but nothing that took more than a couple of hours.

So, how does it sound? Well, it's pretty variable. It sounds great in places where there's a high enough framerate to keep the audio buffer full. Mario 64 is pretty good for this - it seems 25fps is a minimum to avoid any choppiness in the output. When the framerate drops lower than this the audio quality does start to suffer significantly. On the plus side, it seems to be decoding everything correctly in all the roms I've tested so for, which means that the quality is only going to improve as I make further improvements to the performance of the emulator.

Which brings me to my next point - how does enabling audio support affect the performance of the emulator? I've not had chance to measure this empirically, but it probably slows things down by about 10-20%. Given that R9 is running about 40-100% faster than R8, you should be able to run R9 with audio and still see a small speedup. In any case, I've added an option to enable or disable the audio in the front-end (this is also accessible from the pause menu while the rom is running).

So, that's just about the story so far with audio support in Daedalus. There's clearly a lot more work to do, but the good news is that audio support is in, and working, and is only going to improve with time. Later this week I'll talk a bit about future improvements, including methods for reducing the impact that audio processing has on performance.

-StrmnNrmn

Tuesday, February 13, 2007

Revamped UI Sneak Peek

Here's a sneak peek of the revamped UI I've been working on:



[Edit - 9:22 AM] Please don't read anything into the list of titles shown in the screenshot above! I haven't been working on compatibility in R9 (with the exception of a few PIF fixes), so it's unlikely that anything which was broken in R8 is now working. Once R9 is released I'll hold a straw poll to see if you want me to focus on compatibility in R10, and if so, which titles.

-StrmnNrmn

Monday, February 12, 2007

The rest of the time...

On Thursday I gave this short Daedalus-centric list of things I chose between when I have some free time:
  • Do some new development on Daedalus
  • Play games/watch TV/relax
  • Reply to a few emails/comments, post a new entry here


I gave a long list of stuff I've been working on that represents the first bullet point over the past 5-6 months. I thought it might be interesting to list a few of the things covered by the second bullet point in the same period of time:

There's quite a lot of Xbox 360 games on that list. Curse my inexplicable addiction to accruing gamerpoints! Most of the DS and PSP games have just been from the hours spent commuting to and from work - I don't tend to use either much around the house.

By far the most significant item on that list for me was changing jobs. I don't really want to go into too many specific details, so I'll just say that I'm working in games again now and leave it at that. It's nice to have a new platform to explore, and get paid for doing it. I have a nicer commute to work in the morning too. It takes the same time, but I'm travelling against the main flow of people so I can stretch out with my PSP/DS and have a carriage to myself rather than squeezing into an overcrowded train each morning.

Anyway, enough of the offtopic banter! I just wanted to give you some idea of the other things I've been up to in my absence from the blogowebs.

-StrmnNrmn

Sunday, February 11, 2007

A few answers

A few common questions cropped up in yesterday's post so I thought I'd deal with them here so everyone could see the responses.

Firstly, lots of people were asking about the possibility of moving the audio processing to ME. I'm really keen on doing this, but I'm not very familiar with the ME yet so I need to spend a few hours investigating it before I can figure out how much work is involved. Maybe I can do a few technical posts about the ME as I find out more about it?

A few people were asking about an ingame menu and screen size. gotwake424 said:
will there be an ingame menu? that will let you change the size of the screen and other stuff in game?

xabier:
i have just one question: you have mentioned viewport scaling/screen ratio a few months ago. is that going to be featured in Release 9 of daedalus? and i think that i heard somewhere that it should speed up the emulator a little with the new scaling, is that true?

flyinghippo:
One of your earlier posts showed your progress on implementing a 4:3 screen ratio. Would this possibly have any effect on the framerate?

reuben:
A while ago you noted that you had been working on viewport scaling. Will that be implemented in the upcoming release?


I'm currently working on the in game menu - this is the last main thing I want to squeeze into R9. With this you'll be able to change the screen size while the rom is running, as well as a few other things (toggle audio on/off, swap controls etc).
I doubt that knocking the screen size down will help the framerate all that much, as the framerate is still largely limited by the cost of CPU emulation rather than rendering.

From philip:
Do you have plans to implement frameskipping? That would greatly improve the speed, and shouldn't be too hard to add.

on the rise1:
have you ever thought or messed about putting in frameskipping in the emulator?


Now that audio support is in, I think frameskipping makes a lot more sense. I'll see if I can squeeze this in for R9 too.

yatahaze:
can Daedalus run under 300mhz? would that boost those numbers in the table even more?


Unfortunately Daedalus is already clocking everything at the maximum frequency, so there won't be an easy win there :(

imitiaz:
Are the rom files you are using zipped or unzipped? Will the extensions of the rom files even matter anymore?


I was using unzipped rom files. The extension shouldn't matter, provided it's recognised (.v64, .z64, .rom etc) Daedalus should always figure out the correct thing to do with a given rom. I suggest you use a tool like RomCenter to manage the naming of all your roms, and extract them to the roms directory on your PSP as required.

sroon said:
Will expansion pack be inplented in your blessed R9?


The expansion pak should already be supported: 'Expansion Pak: Yes' in the Rom Settings screen (just before the rom launches). If it's not working, it's a bug, and it'll be easy to fix.

nick:
Have you always been able to add audio support? Or did you hold it back because of the slow framerate?


I've been holding off because of the framerate - I just thought it would sound horrible. The framerate is still a limiting factor with audio support, I'll talk a bit more about this soon.

jeffrey said:
Is there any official way to support your work? Either through paypal donations or whatever?


No, not currently. To be honest, I'm a bit uncomfortable about accepting money for this. Financial reward has never been a motivating factor for me, I do this because it's an interesting project with lots of juicy technical problems to solve. If I was to ever accept donations, it would be for a specific goal (e.g. if I broke my development psp and it needed replacing, or to pay for hosting/bandwidth etc).

andrew said:
The best update possible for R9 is an icon for the PSP menu.


Yes, I know :) I do like the Pochi Style icons. I emailed him asking for permission to use them with Daedalus, if I get the go ahead I'll bundle them with R9 by default.

A few people were asking for videos of the work in progress. I'm not all that keen on doing this as I'd rather spend the time getting the final few features in place and releasing early rather than spending the time making videos. You'll be able to play with it yourselves soon enough :)

As for a release date, I'm going skiing for a week on the 25th February, and I'd like to have R9 out before then.

-StrmnNrmn

Saturday, February 10, 2007

Speedup Details

In my long-overdue update on Thursday I explained how some of the work I've been doing on the dynarec engine has produced a significant speedup. This post will give a few numbers to give an idea of what you can expect in R9.

The table below shows the framerate for various scenes in R8 with a column showing the framerate for the build I'm currently testing effectively R9). The final column shows the relative speedup:





















SceneR8 framerateCurrent framerateSpeedup
Mario Head812x1.5
Mario Menu3852x1.4
Peach Letter1825x1.4
Mario Ingame1824x1.3
Zelda OoT Title813x1.6
Zelda OoT Menu3955x1.4
Zelda OoT Ingame 1610x1.7
Zelda OoT Ingame 259x1.8
Starfox Intro2127x1.3
Starfox Title911x1.2
Starfox Training1723x1.4
Mariokart Title1633x2.1
Mariokart Menu1224x2.0
Mariokart Ingame1220x1.7
Spiderman Title2235x1.6
Spiderman Menu2127x1.3
Spiderman Ingame1018x1.8
Quest64 Title7590x1.2
Quest64 Ingame 12025x1.3
Quest64 Ingame 22129x1.4




Clearly there's a significant improvement in framerate. On average, games are running about 40% faster, but in many cases the framerate has almost doubled. Subjectively, the difference means that many games are feeling a lot more responsive and playable now. Anything over 20fps 'feels' pretty good, but my long-term goal is to get this up to 30fps for as many titles as possible.

I noticed a couple of interesting differences between the two builds when running the tests, and you can see some of these in the new screenshots. Firstly, notice that the shadows in Super Mario 64 are all nice and round. This is due to the 'mirrored texture support' that I mentioned in Thursday's update (this also fixes the star that opens over Mario's head in the title sequence).

Secondly, the jerky/shaky screen that was affecting Mario Kart 64 now seems to be fixed. I'm not quite sure what was causing this, but I'm glad it's fixed :)

Next, notice that the text is Quest 64 is now fixed. In R8 this was horribly corrupted. This is due to the fixing a few texture conversion bugs as I was introducing proper support for 16 bit textures.

Finally, the texture on the floor in Quest 64 is now fixed. Again, I'm not totally sure what change is responsible for this, but it's nice to see it working correctly.

One final thing to note is that these figures were obtained by running the emulator with 'optimal' settings. For the current build, this includes disabling audio output. I'll talk a bit about audio support soon, including a bit about its impact on performance.

-StrmnNrmn

Comment Moderation

You might have noticed that I've introduced a new policy of moderating comments for each post. There are a couple of reasons for this, but mainly this is because I wanted to have some control over the nature of the debate following each post.

In the past my posts have fallen into two main categories - general Daedalus updates, and more technical posts about some of the specifics. For the techincal posts I want to encourage more detailed discussion, so I'l only be approving comments which I believe are going to help stimulate this. This means I'll be rejecting comments about unrelated issues ("when is the next release?", "make it go faster please" etc). Please don't take this personally!

For general updates, the policy will be pretty much the same as it has been, i.e. anything goes - although I'll still be rejecting duplicate posts or anything totally off topic.

Cheers
-StrmnNrmn

Thursday, February 08, 2007

Behold! An update!

Wow, it's been a long time since the last update. A really long time. How did that happen?

I've always found it quite hard to find the time to update the blog. Usually when I have some free time in the evenings (that's free time spent doing things other than eating, socialising, and getting stuff ready for work), the choices I have are:
  • Do some new development on Daedalus
  • Play games/watch TV/relax
  • Reply to a few emails/comments, post a new entry here
Unfortunately over the past half year or so the first two bullet points have won out. So, apologies for neglecting the 'outside world' for so long. On the plus side, the existance of the first bullet points means that I have lots of exciting new developments to talk about over the next few days :)

I'm going to finish off this reintroduction with a broad overview of some of the stuff I've been working on. This is all stuff that will be present in R9, which I'd like to release this month.

  • Added support for RGBA 4444 and 5551 textures, saving a bunch of memory in the front end.
  • Tidied up all the texture conversion code, fixing a few bugs in the process
  • Fixed the width/height of FillRect calls in 1 and 2 cycle mode (fixed a few small graphical issues)
  • Fixed a blending bug (fixed a few small graphical issues)
  • Use 16-bit textures on the PSP to represent 16-bit N64 textures. Saves time converting, saves memory, and faster rendering
  • Added mirrored texture support (this fixes lots of small graphical glitches)
  • Fixed a LoadTile bug, allowing a couple of hacks to be removed (this also fixes various small graphical glitches)
  • Added some new blend modes for various roms
  • Fixed the Tri2 command for F3DLX microcodes
  • Fixed a bug in busy-wait detection (this wasn't working correctly with dynarec code, net result is a small speedup)
  • Fixed a few dynarec stability issues (relating to exceptions occuring mid-trace)
  • Added audio support :)
  • Added the ability to dump textures (developer builds only at the moment)
  • Fixed screenshots. Again.
  • Implemented cmp.s, cvt.s, cvt.w, mtc1, mfc1, bc1f, bc1t, j, cfc1, ctc1, daddu, trunc.w.s, bc1t, bc1f, bc1tl, bcifl, bnel, beql, blezl, bgtzl, bltzl, blezl in dynarec (this gives a decent speedup)
  • Avoid setting the branch delay flag and current PC in generated dynarec code unless absolutely necessary (this gives another small speedup)
  • Much better memory access handling in dynamically recompiled code (this gives a BIG speedup :)
  • Use a second code buffer for generated dynarec code, to avoid polluting the instruction cache (this gives another small speedup)
  • Further improve the memory access handling in generated dynarec code (another small speedup)
  • Fix register usage analysis for lwc1/swc1/mfc1/mtc1 which was preventing base registers used in these instructions from being cached (another small speedup)
  • Have compensation blocks restore nobbled registers, so on-trace code does't need to reload (another small speedup)

There's quite a lot in that list, so I highlighted the two most significant points. In summary R9 will be much faster, with audio support. I'll write a bit more about these changes in particular over the next few days (promise!)

-StrmnNrmn