Wednesday, August 30, 2006

Display List Debugger

I've been working on a few new features which I'm eager to talk about, but I thought it would be interesting to give some details about some of the tools I use to help me debug various problems with the emulator.

Some of the hardest problems to identify and fix are various graphical issues that crop up when running certain roms. Sometimes it's unhandled combiner modes (this is what results in the purple-and-black textures seen in so many screenshots). Other times there are black or white polygons, or scrambled textures and so on. Sometimes the screen is just totally black :)

When I'm trying to figure out what's causing a particular problem my first step is to recompile Daedalus with the Display List Debugger enabled. If you've been playing with the source, this is done by setting CFLAGS = $(DEBUG_DLIST_CFLAGS) in the Makefile, and linking in Source/PSPGraphics/DisplayListDebugger.o.

The debugger is accessed by pausing emulation and hitting the right trigger button. You need to have PSPLink set up in order to use it, as I didn't want to clutter up the display with various debbuging output. When the debugger is activated, it keeps the emulator paused and replays the current display list over and over. The first thing is does is dump out a list of all the commands in the display list to a logfile which looks something like this (I've edited it somewhat to create a simple example):


[00066] 0x00214290: 01060040 802143b0 G_GBI1_MTX
Command: ModelView Load Push Length 64 Address 0x002143b0
+1.00000 +0.00000 +0.00000 +0.00000
+0.00000 +1.00000 +0.00000 +0.00000
+0.00000 +0.00000 +1.00000 +0.00000
+0.00000 +160.00000 +0.00000 +1.00000

[00067] 0x001c1118: bb000001 ffffffff G_GBI1_TEXTURE
Level: 0 Tile: 0 enabled
ScaleS: 0.999985, ScaleT: 0.999985
[00068] 0x001c1120: 04f00100 0a000000 G_GBI0_VTX
Address 0x001c1000, v0: 0, Num: 16, Length: 0x0100
#00 Flags: 0x0000 Pos: { 0.000000, 60.000000,-1.000000}
#01 Flags: 0x0000 Pos: { 80.000000, 60.000000,-1.000000}
#02 Flags: 0x0000 Pos: { 80.000000, 80.000000,-1.000000}
[00069] 0x001c1130: bf000000 00000a14 G_GBI1_TRI1
Tri: 0,1,2


What this is showing is a matrix being loaded (command 0066), texturing being enabled (0067), a bunch of vertices being loaded (0068) and then a triangle being rendered (0069). (That's quite a lot of work to display a single trinangle! In reality the n64 would load up batches of vertices and render multiple triangles at a time.)

The display list debugger also provides a number of other useful features. The first is the Combiner Explorer. This is how it looks in PSPLink:



This displays all the different combiners used in a scene, and allows them to be enabled or disabled individually. When they're disabled, it replaces the triangle with a big green and black texture, like so:





In each of these cases a single combiner was disabled, showing how different combiners are used to achieve different effects in the scene.

This tool is invaluable in debugging combiners as I add them, to ensure that they're doing what I think they're doing.

The Texture Explorer displays all the textures used in the scene. It looks like this in PSPLink:



It allows individual textures to be displayed on the PSP so I can ensure that they're being decoded correctly:




The final function I want to demonstrate is the ability to terminate the display list at a given point. That is, if a display list has 1000 commands, I can stop it after at any point during rendering to see what the scene looks like. The example pasted above shows a snippet of 4 commands from 66-69, typically there may be 3000-5000 commands in a display list - the Mario scene above is composed of 3614 commands.

Being able to stop the display list like this is really useful, because it allows me to see the exact command at which a given triangle in the scene is drawn. Let's say that we're trying to figure out why the triangles making up Mario's feet are coming out in the wrong colour. What we'd do is render the display list command by command until the specific triangle we were interested in appeared on-screen. At this point we'd be able to determine the command number, and cross-reference this with the display list log which was dumped at the beginning. Usually looking a few lines before this point in the log will reveal the source of the problem.

I hope this has provided a little insight into how I go about debugging some of Daedalus' graphical problems. I'll leave you with a sequence of shots showing how Mario 64 looks as it's rendered from start to finish:











-StrmnNrmn

103 comments:

tsurumaru said...

Thanks for the update! :)

stee said...

Thanks sounds like a big improvment to a bug which effected so many games

Sroon said...

So what doez this mean?
( O ) ( O )

BreathofFire said...

So the purple and black spots are no more?

tsurumaru said...

sroon, basically I think Strmnnrmn has noted that people seem to be quite interested in adding the missing mux data and is expanding on how he himself identifies what is causing the graphical errors (using the debug mode)and how he fixes them. He did mention he was working on some new features for the next build but I'm sure he'll expand on these in due course :).

Morgan said...

Skater I use the regular 1.5 firmware, not any custom one. Also just so we're clear the eboot you made I put that one in the place of the eboot in my __SCE__Daedalus folder correct? Because for some reason when I do this Daedalus looks like it's going to launch but then it returns with the error.

Morgan said...

Okay I deleted Daedalus off my psp and then redownloaded R8 and put it on my psp. Then I downloaded your eboot and swapped them. Now the emulator boots only problem is now the textures were still messed up in DKR. But there is good news, this time after selecting my save spot it let me choose my initals now. I'm not sure what's going on because it works on yours you say but I'm not getting the results. Has anyone else tried it?

Morgan said...

Does the rom name matter because mine is Diddy Kong Racing (v1.0)

skater9269 said...

try wallys

Urkel said...

Love hearing about your updates strmnnrmn, they keep getting better and better. cant wait for R9

BreathofFire said...

Anyone wanna try to fix Paper Mario textures and not booting after saving.

GmDude66 said...

Nice,
Thanks for the update man!

joshisposer said...

nice update and thx for keeping us up w/ info

Chaz said...

Ok most of that when right over my head but I hope you get all of the texture decoding prefected. :)

wally*won_kenobie said...

hey StrmnNrmn, do you think you could explain the whole process to us??

Skater and I got stuck on the same step mate..
Thanks

johnybon said...

this sounds like great news; a little more speed (say three to four fps), and recification of pink+black texures, and tons of games will be as playable as mario.

N64 MAD said...

well done i just hope it reaches fulll speed feel sorry for mario cart god

BreathofFire said...

Is there anyway to play Paper Mario ?

PSDroideka said...

Great News!!
I know i usually say "good news, but could you do this?" :( But instead does this mean that Super Smash Bros rendering and font rendering bugs will be fixed? Or have i just completely misunderstood....

tsurumaru said...

Hi Guys,

This particular post from Strmnnrmn doesn't mean that anything is fixed for the new release (although I'm sure that things will be), Strmnnrmn is explaining what tools are available to him to find and correct graphical errors.

He has explained to us how (if we wanted to) we could set up the debug mode of Daedalus and explore the graphical processes used by Daedalus.

Hell, if there was someone who knew what they were doing they could really help out Strmnnrmn by using the debug mode to identify the errors causing current texture problems in roms. This doesn't include the black and purple problem which is a specific result of the combiner not being accounted for (remember there are thousands of combinations possible) and can be rectified by adding the missing mux data that Wally, Morgan and others are doing a sterling job of.

However a couple of places where this would be useful is where current combiners are not displaying as anticipated, ie textures not appearing correctly in roms, (jumbled, grey, black etc) and where
adding the missing mux data causes further problems. (ie Zelda OOT)
A final (fairly obvious) point to recognise is that the import of missing mux data must be double checked to ensure that errors are not being introduced as this data is added to the source code (adding further graphical issues).

Kind regards.

Rikki-Tikki-Tavi said...

Thanks, tsurumaru, you saved me a lot of time :).

To Strmn_Nrmn: You're a hell of a coder.
Everybody I talked to about Daedalus said that. That includes my Brother, who is developing phpBB.

I also thought about making a logo for Daedalus. I thought about one including the wax wings or the labyrinth. Tell me what you would prefer (I don't think you wanna have the murder-thing in it), or don't if you think it's a pain answering fanboy comments.

Morgan said...

Skater thanks for trying and I don't know why it's not working for me but where is Wally's link to his eboot. I'll be happy to try his but where's the link?

71M said...

Looks like quite a bit of overdraw going on there. Is it possible to render the display list front to back?

Cheers,
71M

StevIEmw said...

is that a resident evil 2 screenshot i saw in the article?! sure looks like one

Morgan said...

No if you read the article he says where he can look at the objects individually to see how the textures and everything is working. He's just got a single shot of door to the castle in Super Mario 64. It's similar looking to when you enter doors in RE2 but no RE2 pics in there, sorry. Someone please tell me where Wally's eboot is so I can try it, Wally help me out!

skater9269 said...

morgan check the dcemu forums I believe he posted there and also wally lives in australia so i dont think he will be on soon but maybe wrong.

johnybon said...

Morgan, here's wally's eboot:
www.dosgames.com/~wally4000/EBOOT.PBP.zip
Try out '007 - the world is not enough' it's interesting, quite fast and only in need of a few more graphical fixes.
On that note: Wally and Skater, will Norms additions be helpful to you?

skater9269 said...

johnny wally and I have been trying to use this and we have been unsucessful sadly.I am going to try some more today. When I use putty it gives me a connection refused warning If we figure it out it should help tremendously.

LiQuiD8d said...

really interesting stuff! Thanks... hopefully you got my compat list from r8, and hopefully it helps...

joshisposer said...

off topic
ive got a mario save with all boxes if anyone wants it.
btw my aim sn is joshisposer, just make sure it is not going to my cell.
should i have said that? idk

Morgan said...

I must have the worst luck, I just tried Wally's eboot. I have tried and tried after time and again, and I can't get the emulator to start? I tryed re downloading the entire emulator before many times. I guess it's my only hope that StrmnNrmn can fix these DKR changes (for me). I know I'm putting the eboot in right I just don't know why it won't start the mu up. Any tips Wally?

wally*won_kenobie said...

Morgan:

Go to www.dcemu.co.uk click forums

click PSP-news

Find daedalus R8 spiff up.

Register if not already done,
login and download from the first page

wally*won_kenobie said...

BTW morgan

Diddy kong probably needs microcodes,

I plan on looking into this debug tool tonight, see what i can come out with :)

mrrogers said...

You are doing one hell of a job, Strmnnrmn, thank you for the hard work/coding/patience with everything, and keep up the good work!!

Morgan said...

Wally I'll try going to dcemu and downloading it from there but the link "johnybon" put up your eboot file, Wally? I tried that one and the emulator wouldn't start.

Morgan said...

Okay I got good news I went to Wally's thread and got his file. It works and everything is good but then I noticed DKR isn't in the changelist you made Wally. Wally obviously I have been talking about DKR for the past couple days and I wanted the textures/ texture fonts in that to be corrected. Wally could you possibly do an update WHENEVER you have some free time to add DKR.

Morgan said...

Oops I forgot you posted that you'll look into it tonight. Please Wally though make the 1.5 version and release it if you get anything going tonight. Because switching eboots for some reason is working/not working at random times for me. But adding the two Daedalus folders seems to work flawlessly.

skater9269 said...

morgan use pspbrew to convert it to a 1.5 kxploited eboot.

Morgan said...

I did but DKR isn't included in Wally's build.

wally*won_kenobie said...

This is Big Brother. Morgan it is ... That is all

wally*won_kenobie said...

Oh BTW i just got the thing to compile

StrmnNrmn left something out

Heres how to compile it
1. in pspmain's includes add
#include "/PSPGraphics/texturecache.h"

2. Change

OBJS = $(RELEASE_OBJS)
CFLAGS = $(RELEASE_CFLAGS)

to

OBJS = $(DEBUG_DLIST_OBJS)
CFLAGS = $(DEBUG_DLIST_CFLAGS)

it should now compile

johnybon said...

wally, can I appeal to you to look over Pilotwings? Like forsaken (which now looks great after your fix)it was mostly okay in R8, and is fast enough to be playable (it has quite a sedate pace anyways).

Morgan said...

"This is Big Brother. Morgan it is ... That is all"

Wally I'm 100% confused on what your talking about in this post. Are you trying to say DKR is included in your fix? If so it's not in the list of games you said in your thread, also when I tested your build the texture problems in DKR are still present. I know a very good amount of homebrew and I use programs such as PSPBrew to customize my homebrew and do minor editing. There is only certain areas that I'm unsure of. Anyway can someone please fix the purple/black textures in Diddy Kong Racing please! Every single time someone here says that they fixed it, I try it and I get nothing. I appreciate the work your doing guys but I mean I'm not getting the results you say your getting so I'm feeling cheated.

Sroon said...

OKAYP MORGAN! We have nothing against you okayp bro!
Would you stop bugging?
Wally iz using his (very little and limited) time to help further this project!
Show some respect!
I honestly wouldnt be surprised I Norm quit because of all your Bugging, and its not just u too, I admit I can be stuped at times but atlest I dont ask stuff all the time kk!
Your a cool guy Captian theres no doubt about tit with your "compat list" (teehee) Now Id sugest ( as beeing popular guy and a good friend) taht you go out with sum friends befor school stars ( if it hasent started yet)
So im not tgrying to be a pain in the azz okayp bro!
Just go out for a couple of days and there will be an update before you know tit! No hard feelings!

( O ) ( O )

RyanMWolfe said...

yeah i cant get wally's eboot the site goes to a site and says cannot be found any other sites have the eboot???

please help i will only ask once as i do not want to spam up the comments page

Morgan said...

Sroon I'm not here alot, I maybe spend an hour a day on the computer. I have friends and I have a job, I come here when I'm bored and have spare time. I'm not bugging about anything I'm just trying to find out why when two people say they fixed a texture bug and then when I test out the eboot the bug is still there. I'm not getting straight answers so yeah I am going to continue to ask. I'm not bugging StrmnNrmn and the texture thing I'm talking about is towards Wally & Skater not StrmnNrmn, so why would StrmnNrmn quit when what I'm talking about isn't directed towards him. Anyway I gotta go to work in a little so I hope whenever I get back on there will be something going on here. Sroon if your saying your a popular guy on some internet comments thing I think it's you that needs to get out, because my life is fine and I don't need people telling me how to live it.

Morgan said...

I swear it's like you guys don't even read all of my posts and just read parts of it and then just respond. You said show some respect, " appreciate the work your doing guys but I mean I'm not getting the results you say your getting so I'm feeling cheated." I just said I appreciate his work and I really do it's just that I'm not getting the results they say I should be getting, so I'm trying to find out why.

Morgan said...

Here's Wally's eboot ryan.

http://rapidshare.de/files/31724445/EBOOT.PBP.html

RyanMWolfe said...

Thanks Morgan, but i already got it sorry. anyway i think someone Should fix the Blck and purple textures for Aerogauge it is like F-Zero, but it goes full speed.

skater9269 said...

morgan I did have it fixed at one time but it had a dragging problem in parts and still did not work in game Wallys has more added textures than I do and so I have been looking at other parts we have not merged our combiner modes I know that I did have dkr fixed to a point but am unsure if either I uploaded the wrong eboot or what but as the ingames is still not fixed it is not incredibly urgent. Also I have not had time to work on anything this weekend as I have not been at my house but all in good time. Thanks

stee said...

OMG morgan every update you have to post about 100 loonnnggg comments

Please stop.

Morgan said...

Then don't read it. Thanks Skater for straightening it out, I know that DKR seems useless to you guys since you can't play ingame. But see the thing is after you select what memory slot and everything you start the Adenture mode with a short video of the Wizpig mountain and the Genie (elephant) flying around. The reason I want the textures fixed is because I would like to see the actual words on the menus (they're purple/black) because the short video doesn't play on Daedalus like it's supposed too. I just want to find out why this problem is happening. So I'm saying Skater if you could try to find that eboot or just whatever someone if they could try to fix the textures if possible. Wally's build didn't fix the textures for me like you say Skater, could you tell me what fonts you see, maybe take some screenshots? Alright I'm going to go roll up a dutch, so I'm out!

johnybon said...

Wally, Skater. Just curious, are you going to pass all your texture fixes onto norm? It would be shame to see the work you've both done go to waste. If indeed there are thousands of potential combiners it would be a crime to see texture fixed or texture improved games return to a worse state in R9.

wally*won_kenobie said...

Johnybon, i plan on it.

However half of the textures myself and Skater have added are duds.

wally*won_kenobie said...

Aerogauge is fine AFAIK

RyanMWolfe said...

someone needs to release an R8 with Frameskip 1 because R8 is more stable than R7 and it will proboblu help with speed.

skater9269 said...

johnny every release is realeased with source therfor we can simply overwrite the blendmodes file that is in each source realease therfore even if strmnnrmn choses not to use our blendmodes then we can still release an alternate R9 within 5 mins most likley.

joshisposer said...

megaman 64 har great speed improvements. the game is playable now but the textures are bad even with wallys eboot

Morgan said...

Hopefully StrmnNrmn will update us on some progress here shortly. Until then I'm going to go watch the race at 7pm EST.

wally*won_kenobie said...

....

Im going to re release this eboot when i get the debug list working....

Don't ask about frameskip again, it will never happen!

StrmnNrmn has said on various occasions it will just make things worse

RyanMWolfe said...

ok sorry, so wally will your new eboot be faster, or will it just fix some Graphic. Also can you fix the Chopiness in Mario Kart. there are vaarious games that don't even work? i hope strnrm will fix these over time and by R15 most games will work like mario and Mario will be full speed.

RyanMWolfe said...

Also when will your eboot be coming out Wally??? hopefully soon!

Morgan said...

Yeah people frameskip is gay, it's like fake speed trust me just let's keep the frameskip thing on the back burner. And congrats to StrmnNrmn you are officially the only active N64 coder for PSP, now that PSMonkey has left the scene for a dev job. Make us proud StrmnNrmn, you already have blown me away! Question Wally in the settings what do you like to run with the graphics update thing (disable, every frame, every 2, every 5....) I usually run every 5 but if I use every frame it will only slwo it down fractions right?

Morgan said...

Full speed Mario might be like R10 man, it's already like full speed now.

RyanMWolfe said...

i am really intersted in the Daedalus and i wish i could help is there any way i can help the Progress of Daedalus i am 15 years old and have nothing to do. so if there is any way i can help just tell me what to do.

wally*won_kenobie said...

morgan: Good question..

I leave it as default, what ever it is.

Morgan said...

Well I never have it disabled because the graphics are bad then, like for example in Mario Kart 64 the cloud guys that gives you the green light flickers if it's disabled. I tested it today and update every frame IS NOT slower FPS wise than every 5 frames update. Well it might be mayeb fractions of fps but nothing noticeable.

Chaz said...

What games have you guys been playing on Daedalus? (of course mario64)

stee said...

Full speed mario r10???

Your jokin if you you think Strminrmn will get a 100% speed increase any time soon.

Morgan said...

On Daedalus I've played over 50 games obviously when I made my compat list. But yes full speed mario 64 could be in R10. It's only like 5-7fps (depends where you are)away from full speed. Yes it doesn't have sound yet but still, Mario 64 is almost full speed go play it and look at the FPS. I'm just saying Mario is close to full speed, there are others though. I hope StrmnNrmn gives us an update soon, I bet he's got alot to talk about and some exciting changes for us!

RyanMWolfe said...

ok guys i feal useless. :( and really would like to help out Daedalus or something. naybe Wally can let me join in on theCompat list? doubt though, but anyway i feal if i keep downloading the Daedalus Releases then i should at least help in some way if anybody wants me to help just let me know

Thanks. my Email is RyanMWolfe@yahoo.com

thedman said...

SUP FOOS man did u guys hear that psmonkey left for some crappy dev job ... hey does anyone now his name so i can see if he made a game ...oh and also more off topic STEVE IRWIN DIED.........*sobs uncontrolaby* he was so kool and also good job strnum

thedman said...

dam i cant spell

skater9269 said...

ryanwolfe do you atleast have basic programing skills atleast the abillity to read source code and cygwin or a simillar compiler?

Morgan said...

No offense to ryan but I don't think he does, but maybe you could get into making icons/wallpapers for Daedalus. I mean I currently use the green deadalus icons/wallpapers that pochi makes. I think you took your time you could make some sick icons.

wally*won_kenobie said...

ryan, we have already got a team.



Contributions are welcome though

skater9269 said...

wally have you got it yet I cant get nethost to recognize

KELVIN said...

hey wally just wanted to say that ive tried your eboot and it works great, i managed to beat the first boss in zelda OOT.thanks for sharing youre fix with us. ive also noticed that the only parts that dont show are stand still pictures. when u enter a town or house where u cant change the camera angle and youre the only thing that moves then the background doesnt show. (ex: links house)

Morgan said...

Make icons/backgrounds for Daedalus Ryan, I'm sure if you put some effort into it you could make some hot icons. And think if they look really good then people all over the world would be using your icons on their psp.

RyanMWolfe said...

I will make The backgrounds and stuff but first someone give the link to a program to make backgrounds with please

salomon st said...

i wonder what his progress on R9 is

wally*won_kenobie said...

yep its all working :D

Morgan said...

Well you can make them with almost any program you just have to make sure the icons are 134 x 74 and the backgrounds are 480 x 272.

Morgan said...

I'm sure there is a program that makes the pics those sizes, but for reference those are the sizes.

thedman said...

he should so add sound as an on or off button

wally*won_kenobie said...

There is some for mac but for pC uhmm i dunno

Disturbd1 said...

for pics use The GIMP or Photoshop....

RyanMWolfe said...

Ok guys made some pics but dont know how to get them to you guys???
HELP

ps the pics are not very good:( sorry first try

omgnintendo64 said...

host the pics on photo bucket or imageshack the post the links here,of you could upload them to rapidshare and then give us the download link

Morgan said...

www.rapidshare.de

Go there click Browse and select the pics then hit upload. Then when it's done scroll down and hit forget the whatever just give me the links. Then copy and paste the links here.

RyanMWolfe said...

http://rapidshare.de/files/32200373/plain_background.png

there is the Background i used Mario 64 because it Is the most playable game on the daedalus

http://rapidshare.de/files/32200485/ICON.png

Here is the ICON N64 Pic ok tell me what you think and tell me the truth!!

RyanMWolfe said...

Sorry bad links here they are again

http://rapidshare.de/files/32200485/ICON.png

http://rapidshare.de/files/32200798/plain_background.png

RyanMWolfe said...

Why are the Links not going all o there ther is supposed to be more to the link / icon.png, but its not there HELP!

Morgan said...

Not to bad for your first try, but here go here for inspiration.

http://www.riotsgraph.jp/pochistyle/


Check out the Daedalus ones under the N64 section, it has a matching green background if you scroll down AFT YOU CLICK THE LINK I GAVE YOU.

Morgan said...

Links worked for me, the background looked okay but the icon wasn't your best work. It was just a pic of the N64 system.

RyanMWolfe said...

yeah i used Paint the worst Phot thing Out there. lol

RyanMWolfe said...

yeah i used Paint the worst Phot thing Out there. lol

RyanMWolfe said...

sorry about double post comp Glitched up

Morgan said...

The background was better than the icon, just the plain old system as the icon can be boring.

RyanMWolfe said...

What icon Should i have i was thinking if the Icon Just Said Daedalus Just plain.

Morgan said...

Like I said go to Pochi for inspiration. Look how he made the Daedalus icon.

RyanMWolfe said...

http://rapidshare.de/files/32203100/ICON.png.html

Here is a new icon.

Babsuno said...

Hi, I would like to play Mario 64
on my PSP, where and how can I get
it?