As I mentioned in that post, it's not just a case of changing some compiler setting to get this working. I've not spent much time investigating the ME so I may be wrong on a few of these points, but here are the current issues that I think need solving.
Firstly in order to access the ME I need to be running in kernel mode. This requires either running Daedalus in kernel mode, or (preferably) creating a kernel mode PRX that encapsulates the required functionality. I think kernel mode rules out anyone running with v1.50 firmware (hence my earlier post - please respond to the poll if you haven't already done so!) Maybe one of the more savvy psp developers out there can correct me on this? If no-one is using v1.50 any more then maybe it isn't even an issue.
Another problem is that although the ME is essentially the same processor as the main core, it has a different memory map. This means that things like the VRAM is invisible to the ME, so any code ported to run on the ME would have to be written to operate on main memory. This isn't an issue for Daedalus's audio list processing, but it would cause problems if I wanted to move display list processing to the ME too.
Touching on the memory map issue, another problem is the lack of cache coherency between the two cores. I need to be careful when accessing the same areas of memory with both cores to correctly flush and invalidate the data caches. Ideally any shared memory should be kept to a minimum, but this is easier said than done when porting existing code, rather than writing new code.
For a similar reason, any code which needs to run on the ME should avoid making any calls to the runtime library, including doing any system memory allocation. System calls are also ruled out. This is fairly easy to guarantee if you're writing new code, but again, it's a lot harder if you're porting existing code.
I think that's most of the issues from the hardware side. There are also a number of issues to be solved to do with the way that Daedalus handles audio and display list processing.
On the N64, the audio and display lists are processed asynchronously by the RSP coprocessor. In Daedalus, I can identify when these tasks are queued up for the RSP, intercept them, and process them synchronously (using high-level emulation rather than simulating the RSP execution directly).
The key thing here is that as far as the emulated N64 is concerned, audio and display list processing currently happens instantaneously. As soon as it kicks off the RSP it gets a interrupt to inform it that processing has completed. The whole process is very deterministic and I'm worried that by processing these display lists asynchronously on the ME that a number of intermittent and hard-to-debug issues will crop up. On the other hand, processing these tasks asynchronously is much closer to the behaviour of a real N64, which may fix some timing-related issues. It will also allow Daedalus to exploit the inherent parallelism that N64 roms were designed to take advantage of.
My current plan for ME audio support in R14 is:
- Create a kernel mode PRX and get Daedalus successfully loading and invoking functions (under all supported firmwares). I've just about done this.
- Add the code to support initialising and running code on the ME to the PRX. Test invoking user mode functions from the main EBOOT.PBP. I'll probably be using J.F.'s great sample code as a reference for this. Thanks J.F.!
- Rewrite the audio list processing code so that it can be invoked synchronously or asynchronously as required (via some kind of configuration option). When running asynchronously it can just be run from a separate high-priority thread to start with. I can use this to test for various synchronisation issues without going through the pain of trying to do this on the ME first.
- Audit the audio list processing code to minimise any memory accesses or ensure that they are correctly synchronised with the main core/thread. Any crt or system calls need to be eliminated or abstracted away (e.g. printfs NOP when compiled to run on the ME).
- Invoke audio list processing code from the ME.
- Cross fingers.
So, that's the plan; I'll keep you updated on my progress. If anyone has any experience doing this kind of thing on the ME it would be great to hear your thoughts.
-StrmnNrmn
29 comments:
pls
support the extended Super Mario64 ROM
This ROM runs faster and allows custom Levels in the game^^
Just go on youtube and search for "Super Mario 64(2)"
My name there is FeYoshiParty
It would also be no problem for me to send you the ROM to help you^^
Hello im glad your going to use ME go to ps2dev.org. you will get the best help there!
I don't think there would be any problems accessing the kernel on 1.50, the SNES emulator accesses the kernel to use the ME and that works on 1.50.
i dont have a clue what the media engine is but if it can get the sound working as fast as the frame rate then kl
i luv yhoo strmnnrmn
get this to work lol
wish u luck wid the making
j.w how long do u think this will actually take
StrmnNrmn: 1.50 certaintly doesn't rule out kernel mode, in fact most 1.50 homebrew is unnecessarily in kernel mode :) Snes9x_TYL 0.4.2 ME uses the media engine in kernel mode on the 1.50 kernel, check out their source to see how it works: http://yoyofr92.free.fr/psp/snespsp.html
Chilly Willy ported this, so the media engine access works on the 3.xx kernel, using a PRX (as you mentioned). http://forums.maxconsole.net/showthread.php?t=84525 - source here.
Basically the main difference between accessing kernel functions on 1.50 and 2.XX+ is that for 2.XX+ you need to be doing it using a kernel PRX which is bootstrapped by a user mode ELF. From what your post says, you have already started doing that, and as R13 worked on the 3.71 kernel, you probably don't need to make any changes to the Daedalus main thread, just keep it in usermode :)
If you want simple but very good example of using kernel mode functions in the 3.71 kernel, see the source code for IDStorage Manager (also by Chilly Willy) - http://groups.google.com/group/chilly-willys-ice-flow/web/idstorage-manager
Good luck and thanks for all your work on Daedalus :D
@jason: Thanks - that's really helpful info.
I'm assuming that I needed to at least kick off the main thread in kernel mode in order to load my .prx (after reading TyRaNiD's post here )? Certainly I couldn't load my .prx from outside psplink with my usermode elf.
Other than a bit of head scratching over that, I've managed to get my .prx loading and a couple of simple functions executing on the ME using Chilly Willy's DisplayTest source as an example. Next step is to start going through the audio code preparing to to run on the ME. Exciting stuff!
@StrmnNrmn:
You can actually load the kernel PRX from the usermode main thread, like this.
PSP_MODULE_INFO("daedalus (or whatever you put here)", 0, 1, 1); // main thread
PSP_MAIN_THREAD_ATTR(0);
SceUID kmod = pspSdkLoadStartModule("daedalus_kernel.prx", PSP_MEMORY_PARTITION_KERNEL);
if (kmod <0) {
printf("Error 0x%08X loading/starting kernel module.\n", prxdec_module);
}
jas0nuk
if we could play daedalus with sound with the same framrate as we do now without sound it would be almost perfect!!
keep up the great work strmn, you are the main news for the psp scene nowadays! :)
@StrmnNrmn
OK, first off, amazing work so far! It's truly stunning to see how far this project has gone. (You can really see the level of improvement it by running an early build like R3 and then jumping to R13). I am really amazed (and happy) at how far N64 emulation on the PSP has come, and how much more is still to come!
Second, as some other's have mentioned, I'm pretty sure 1.50 would not become incompatible with kernel mode. In fact, I'm pretty sure it's the opposite. 1.50 used to be the "golden firmware" before custom firmwares because it was fully unlocked and could run all homebrew including kernel mode homebrews like iR Shell and the SNES9xtyl ME version. So anyway, I don't think adopting support for the Media Engine with a kernel PRX would count out 1.50.
Third, I'm really looking forward to this ME implentation, and can't wait to see how well we can run games with sound!
Lastly, I just wanted to remind you of the following list and ask how many of these optimisations have yet to be accomplished (and if they are planned for R14 or what other optimisations are planned besides ME implementation): http://strmnnrmn.blogspot.com/2007/03/r10-plan-of-action.html
Thanks again for all your hard work. The emulator speaks for itself!
@jason: I'm actually using pspSdkLoadStartModule, but when I tried running Daedalus outside of psplink the prx failed to load.
The link I gave earlier was wrong - I was refering to this post by TyRaNiD. It sounds like I need to call pspSdkInstallKernelLoadModulePatch(), but this is only available in kernel mode.
So it sounds like the best solution is to launch Daedalus in kernel mode, call pspSdkInstallKernelLoadModulePatch(), and then spawn a user mode thread to continue execution as normal. Unless I'm missing something?
@james: Cheers. I stand corrected about the v1.5 firmware, which is good news :)
@StrmnNrmn:
On 1.50 you can do everything in the one thread. My advice would be to start Daedalus in kernel mode and use it all the way through, including media engine functions. You will need the pspsdk patches to be "installed", as you mentioned, namely:
pspSdkInstallNoDeviceCheckPatch - allows modules to be loaded from memstick
pspSdkInstallNoPlainModuleCheckPatch - allows unencrypted modules to be loaded
These must only be used in 1.50, because
1) They will affect the 3.XX kernel in very odd ways
2) M33 and similar custom firmwares all apply HEN patches which automatically apply the more complex, 3.XX versions of those pspSdk patches.
For the 3.XX version of your code, have it start in usermode, and use media engine + other kernel funcs from an external PRX, which itself has kernel mode attributes (0x1006).
If you want to keep it tidy, use wrappers.
e.g. 1.50 version of your code:
int sceKernelGzipDecompress_(u8 *dest, u32 destSize, const u8 *src, void *unknown) {
return sceKernelGzipDecompress(dest, destSize, src, unknown);
}
(this goes into your main thread)
The 3.XX version of your code
int sceKernelGzipDecompress_(u8 *dest, u32 destSize, const u8 *src, void *unknown) {
int k1 = pspSdkSetK1(0);
int err = sceKernelGzipDecompress(dest, destSize, src, unknown);
pspSdkSetK1(k1);
return err;
}
(this will go in the kernel PRX)
Then you just call sceKernelGzipDecompress_ in the main code and depending on the compiled version it calls it internally or from the PRX :)
(above code from PRXdecrypter but you get the jist of it :p)
jason.
@jason: I see - it's starting to make sense now. It looks like it's probably going to be a bit more painful maintaining and testing the different configurations from now on. Fun!
Thanks again!
@jas0nuk: "'keeping' it clean"... well let's just say, this reminded me of what I have seen of daedalus:
http://xkcd.com/292/
No offense though, if you see through, strmn_nrmn, I couldn't be happier.
Keep up the great work!
Will this still work with psp slim? If you apply the ME I mean.
StrmnNrmn great to see you want to add my thumbnail idea, hope it all goes smoothly. Question though, after this conversion to the ME do you think this will add any speed to the emualtor due to different proccessing.
Just a guess, but I'm guessing we'll only see improvements with audio enabled, since Daedalus seems fairly optimized at avoiding unecessary calls with the sound disabled. If it were running part or some of the audio code even with audio disabled then we might see a framerate increase, but I doubt that it works that way now. Then again, I could open the project and take a look.
Btw, great emulator. I wish I wasn't involved with another game project to help out.
StrmnNrmn you are the man!!!
thanks for what you've done so far StrmnNrmn ^^
I really enjoyed testing my favorite games with your emu since the early builds !
Whoaa, sound through ME and keeping the same framerate would really be great ^^ some games would be enjoyable like good "old" time but anywhere not just in you living-room :)
on another topic, I've been willing to ask if you could take a very lil' quick look at this (slim's full screen tv-out)>
http://forums.ps2dev.org/viewtopic.php?t=9160
I know that as of now, you don't have a slim to test this, but do you think it is likely possible for daedalus to feature such an option ? (I know it's not with composite/s-video cable because it mean re-interlacing progressive video signal which ultimately lead to a lot of cpu consumption... but what about direct progressive out with HD cables ? (YUV or D-terminal)
I hope you can spare a bit of your time to check it out and comment a bit about it ! ^^
thanks in advance, if it can work I'm sure a lot of ppl would be interested :)
@StrmnNrmn
I don't mean to be rude, but did you miss this little part of my last comment:
"Lastly, I just wanted to remind you of the following list and ask how many of these optimisations have yet to be accomplished (and if they are planned for R14 or what other optimisations are planned besides ME implementation): http://strmnnrmn.blogspot.com/2007/03/r10-plan-of-action.html "
I'm just curious as to how much of the list you've done, and how much is left or what other optimisations you can do that are left out from the list. Thanks again for all your hard work. The emulator speaks for itself!
@strmnmrnm
Hey man i just thought of this the other day, im unsure of whether its possible or is already implemented.
I have noticed that when your not moving in any game the framerate is close to full speed or full speed. then when you move it goes down by around 4-8fps.
Which would tell me that all the graphics/textures/models for the whole map are all loaded at once?
I reckon (if you have already not done so) if you only display/load these gfx and textures when there within the screens viewable area this would provide a significant speedup.
Btw thank you so much for working on me support. You rule dude!
@mat/shin: I'll have a look, at first glance it seems very interesting.
@james: Sorry. Of that list I've done #2, #3, working on #4, #8 and a little on #10. At some point I'll add a new post with a bit more detail on what big optimisations are still left to do.
@danny: I think this isn't so much to do with the graphics rendering as the rom itself doing more processing when you're moving. On a real N64 the cpu would probably spend more time idling when you're standing still. When you start moving it has to do collision detection, extra animation blending etc. I think you see the slowdown in Daedalus because the cpu suddenly has to do a lot more work.
That reminds me of playing "Banjo Kazooi" on an older computer with PJ64 : the audio was fine, but clearly much faster than the animation behing, resulting in the music being more advanced than image.
I do better understand why now...
That reminds me of playing "Banjo Kazooi" on an older computer with PJ64 : the audio was fine, but clearly much faster than the animation behing, resulting in the music being more advanced than image.
I do better understand why now...
Kudos StrmnNrmn for this purely wonderfull piece of software, this
is simply one of the best reasons to get a PSP. your knowledge of the
N64 and its various functions amazes me to an extent that i just laugh
out loud every time you release a new build because i know it's got
nothing to do with the previous builds, it's like Christmas or a b-day
party every time i try the newer versions :). Keep up the good work and
don't forget to get some sleep and a little break everynow and then,
even if something is hella fun to do it is also sometimes hellapain to
take the aftermath of those sleepless nights :D cheers m8
The media engine has a lot more sway then we think. We all think that it just does it's own thing. It actually has influence on our entire life.
Aaron Carter | http://www.spiritdsp.com/products/voice-video-engine/
Media engines are so cool. Some still have a few kinks that need to be worked out. They have been getting better though. It's crazy how fast technology is changing.
Ron | http://www.spiritdsp.com/products/voice-video-engine/
My brother was over helping me with my ME and I had no idea what he was doing. He was trying to explain it to me, but it's like he was speaking another language! Thanks for explaining things in a much simpler way. It was much easier to understand!
Brooke Bowen | http://www.spiritdsp.com/products/voice-video-engine/
Thanks for the great information. My brother has been doing things like this, and I never knew what it was when he brought it up. I've asked him a few times, but he always gives me really complex answers. This post defiantly answered all of my questions in a much easier way.
http://www.spiritdsp.com/products/voice-video-engine/
Post a Comment