PS3 [PSP Launchers] Problems and Research

[DeViL303]

But the alternative way is by parsing and updating the database manually (the file "/dev_hdd0/mms/db/metadata_db_hdd")
Not sure if you have some code to deal with it, right now i dont know any PS3 homebrew able to do it, but in PC the other day has been published this
https://www.psx-place.com/threads/x...ce-your-games-on-console-to-catalogues.23272/
Maybe @horovo_games could help or share some source code related with that database file ?

It should be possible to update this file and view changes instantly, just like if you was to install a pkg with a different param.sfo, it would update instantly. So just need to simulate that really. Maybe can see what exactly this is doing with socat and debugging Cobra payload.

Or A low level way to see how database changes, Maybe setup database with PSP launcher as remastered, dump database, then change nothing, except update param.sfo with a pkg that changes it to mini, then dump again and compare in hex etc. then see if backup manager can make those changes reliably. I dont know, If I could code for PS3 that is how i would approach it..

 

[sandungas]

I think updating the file metadata_db_hdd manualy while you are inside an app (like managunz) is safe and is going to work nice, because at the time you exit the app the XMB reloads the edited database

Is nice because is not needed to enter recovery menu to do the "rebuild database" manually... we are rebuilding it unnofficially
Doing it with a plugin (while the whole XMB is loaded) should not work so well, because the changes are not displayed instantly (unless you find a way to "reload database" or "reload whole xmb") with the plugin inmediatly after updating it

 

[DeViL303]

I think updating the file metadata_db_hdd manualy while you are inside an app (like managunz) is safe and is going to work nice, because at the time you exit the app the XMB reloads the edited database

Is nice because is not needed to enter recovery menu to do the "rebuild database" manually... we are rebuilding it unnofficially
Doing it with a plugin (while the whole XMB is loaded) should not work so well, because the changes are not displayed instantly (unless you find a way to "reload database" or "reload whole xmb")

As well, there is that little "blink" the ps3 does after you install a package, this reloads database and also checks xml queries again as i discovered when creating the CFW Extras category mod. We just need to be able to call that system function without installing a pkg , should not be too hard in theory, it is a separate operation that happens after extracting the pkg.

 

[sandungas]

It should be possible to update this file and view changes instantly, just like if you was to install a pkg with a different param.sfo, it would update instantly. So just need to simulate that really. Maybe can see what exactly this is doing with socat and debugging Cobra payload.

Or A low level way to see how database changes, Maybe setup database with PSP launcher as remastered, dump database, then change nothing, except update param.sfo with a pkg that changes it to mini, then dump again and compare in hex etc. then see if backup manager can make those changes reliably. I dont know, If I could code for PS3 that is how i would approach it..

Im trying to separate the changes needed to be made to the database... and the changes needed to be made to the SFO to make the swap in between categories "PSP Remasters" and "PSP Minis"

To update the database manually it can be made by an app (less intrusive for the user), or with a plugin (by reloading database after updating it)

But the changes in SFO needs to be made not only in the database, but at all times, keep in mind everytime you enter in a app the info in the real SFO overriides the "old" info about that SFO in the database, so i think is really needed to modify the SFO

Also, if i remember right cobra is using some info from the SFO and EBOOT.BIN too to configure how the emulator boots. But this goes very deep into cobra, i dont understand it
Not sure what does the EBOOT.BIN either... but im guessing we need to replace it too... or could be useful to replace it

 

[Zar]

The way i imagine it is a bit twisted, but it fits pretty well with the "create shorcut PKG"
Also, i like it because i think managunz have all the required functions already working (but used for other tasks), so initially it looks it can be made by repusposing the functions that already exists

The only doubt i have is that "autoinstall" i mentioned, right now im not sure where is used in managunz, but if i remember right you was doing something ike this before in one of the managunz previous versions ?, you was creating a installation of something

That was one of the things that surprised me, because it did look like if you was updating the databases, either by using syscalls, or by rebuilding the database manually

I still dont know how you was doing it, but right now im thinking maybe is not posible to use that syscalls to update dabatase while inside an app (managunz), im guessing there is some restriction and is not posible, dunno

But the alternative way is by parsing and updating the database manually (the file "/dev_hdd0/mms/db/metadata_db_hdd")
Not sure if you have some code to deal with it, right now i dont know any PS3 homebrew able to do it, but in PC the other day has been published this
https://www.psx-place.com/threads/x...ce-your-games-on-console-to-catalogues.23272/
Maybe @horovo_games could help or share some source code related with that database file ?

ManaGunZ isn't installing any pkg, it generate the shortcut pkg then it's up to user to install it. I didn't want to work on the database.
It's a good idea to have the option "Create a shortcut on XMB".

BUT

I was looking at his tool, I think he is just reading every param.sfo from the database, which are 'viewable' without doing anything
Adding and removing an entry, it's something else...

 

[pinky]

ManaGunZ isn't installing any pkg, it generate the shortcut pkg then it's up to user to install it. I didn't want to work on the database.
It's a good idea to have the option "Create a shortcut on XMB".

BUT

I was looking at his tool, I think he is just reading every param.sfo from the database, which are 'viewable' without doing anything
Adding and removing an entry, it's something else...

I believe that's how app2usb works on the ps4.

the database is a finicky thing. I've never tried to add something to it, and I'm not sure if album is just empty (i.e allocated space).. the data for memory cards is also in there. I'm not sure if it's connected directly to profiles either. if you mess up the database, it'll most likely rebuild the database, so it's no big deal. I've actually had that happen on the vita, so if you have a lot of albums, I'd suggest backing up the file.

 

[DeViL303]

Im trying to separate the changes needed to be made to the database... and the changes needed to be made to the SFO to make the swap in between categories "PSP Remasters" and "PSP Minis"

To update the database manually it can be made by an app (less intrusive for the user), or with a plugin (by reloading database after updating it)

But the changes in SFO needs to be made not only in the database, but at all times, keep in mind everytime you enter in a app the info in the real SFO overriides the "old" info about that SFO in the database, so i think is really needed to modify the SFO

Also, if i remember right cobra is using some info from the SFO and EBOOT.BIN too to configure how the emulator boots. But this goes very deep into cobra, i dont understand it
Not sure what does the EBOOT.BIN either... but im guessing we need to replace it too... or could be useful to replace it

Yeah, Im sure the changes to param.sfo can be made just be swapping the file, or patching it on hex level.

I was looking at changes in my database when i change category on a PSP Launcher from remaster to mini, that was the only change i made, see differences in attached files.


Mini:

Remaster:

I have attached the 2 database files if you want to see the other differences.

I think it would be possible to search for the Content ID "50 53 50 4D 36 36 38 32 30" and then patch the section after that with hard coded patches that AND change the param.sfo between lots of different ones (swap the file), its almost copy and paste. .There seems to be big gaps of zeros in between items which is handy too.

 

[ecto]

Hi, thanks for the calming words! That's very interesting what you found out!
I'm not quite sure if it will resolve my problem, but so far it seems good. Since my PSP emulator crashes occur after some time (usually the next day) I can't say for sure. The crashes stop when I change something on my system (like changing von REBUG to Normal mode in the toolbox, reinstalling the PSP launchers, etc.) but after a while they come back.
I had a long discussion with Joonie about that, but he's also at his wits end. That's why I want to try wiping my system clean. Probably something is corrupt (I cannot use the fscheck from the service menu without having to format my hdd due to some unrecoverable errors. Thank god I made a dd image before trying that!)

 

[sandungas]

As well, there is that little "blink" the ps3 does after you install a package, this reloads database and also checks xml queries again as i discovered when creating the CFW Extras category mod. We just need to be able to call that system function without installing a pkg , should not be too hard in theory, it is a separate operation that happens after extracting the pkg.

Yes, thats the most notable difference in what i was mentioning before about doing the (non official) database edits from an app or from a plugin
I said doing it with a plugin was a bit more "invasive" but that word was not a good choice, i just meant this visual "poof" on screen that is needed to do incase of doing it with a plugin

Is not much important, as you say is the same that happens at the end of a PKG installation, and it seems to be made by explore_plugin, scroll back in the thread to one of the posts of aldostools where he posted a link to wiki to a page where mysis posted that function names
There is one to reload categories, and other to reload segments by his <View_ID> (you know, inside the XMBML files)

The difference when doing this database edits inside an app is the user doesnt realizes about what happened (and neither the PS3 firmware). Im going to write the sequence to prepare a "fake" "PSP launcher" installation:
1)enter the app (as example, managunz)
2)copy the "PSP launcher" files (EBOOT.BIN PARAM.SFO, etc...) to dev_hdd0/game/whatever
3)add a new entry for the fake "PSP launcher" installation in dev_hdd0/mms/db/metadata_db_hdd
4) exit app
5)XMB loads the modifyed metadata_db_hdd

Thats what should happen the first time this "hack" runs (we are supposing the user didnt had any "PSP launcher" installed before, so we need to install it for first time)
The next times (lets say you are inside managunz changing PSP settings, or changing PSP game is the same, but with this differences:
2) replace/update EBOOT.BIN PARAM.SFO (if new setings differs from last used settings)
3) make the needed changes in the database to match with the new PARAM.SFO

Yeah, Im sure the changes to param.sfo can be made just be swapping the file, or patching it on hex level.

I was looking at changes in my database when i change category on a PSP Launcher from remaster to mini, that was the only change i made, see differences in attached files.


Mini:

View attachment 16097


Remaster:

View attachment 16098


I have attached the 2 database files if you want to see the other differences.

I think it would be possible to search for the Content ID "50 53 50 4D 36 36 38 32 30" and then patch the section after that with hard coded patches that AND change the param.sfo between lots of different ones (swap the file), its almost copy and paste. .There seems to be big gaps of zeros in between items which is handy too.

Very convenient, it seems are only 2 bytes and the pattern used to find them is straightforward

So it seems we can change the category of the "PSP launcher" in the database easilly (without need to understand the whole database internal structure)

But i have to insist, i think is not enought with modifying the database file, because the database works a bit as a "preview". In the way that his purpose is to allow the XMB to display the contets to the user the faster way posible (without need to scan the whole hdd to know what is inside it)

But when you "click" in the icon to boot the game... at this point the firmware reads the real PARAM.SFO file and his contents, and does a comparison with the info stored in the dtabase for that PARAM.SFO
Incase the info differs... the XMB is going to show you an error message on screen... and inmediatly after returning from that error the database is updated with the info from the real PARAM.SFO
So... at that point you have lost your "custom" changes in the database file... also you could not boot the game, and if you try to boot it again (inmediatly after the error) is going to use the info from the real PARAM.SFO

I was having this kind of errors many times while doing experiments with PARAM.SFO files, im not completly sure if is going to happen when trying what you are suggesting, but im guessing is going to happen
So... this is why i think is really needed to change the PARAM.SFO

Also... i guess there are several EBOOT.BINs used in this kind of "PSP launchers", so im guessing is going to be neeeded/usefull to replace them dinamically too (at the same time the PARAM.SFO is modifyed)
And by now the only/easyer way to see that this can be made is by including the compiled/signed EBOOT.BIN inside the app itself. A backup manager like managunz or irisman could include them because there is no size limit for the app... but in a plugin like webman is not so easy, obviouslly they EBOOT.BINs cant be included inside the .sprx... but the .sprx could "manage" them by copypasting them from different paths... or using symlinks

ManaGunZ isn't installing any pkg, it generate the shortcut pkg then it's up to user to install it. I didn't want to work on the database.
It's a good idea to have the option "Create a shortcut on XMB".

BUT

I was looking at his tool, I think he is just reading every param.sfo from the database, which are 'viewable' without doing anything
Adding and removing an entry, it's something else...

But you did an "autoinstall" in one of the previous managunz versions (and later changed it) or im dreaming ?, i thought you did at some point (as something experimental)
Anyway, at this points never minds, because you mentioned you are not parsing that database file, thats the important thing i wanted to know, thx

Btw, i think that app allowed to save the edited database file... not sure if is mentioned in the release text, but he have lot more info in his blog about how it works, i think he mentioned something somewhere that made me think he was parsing the complete (or a good part) of the database file

Managing that hdd database file could be something pretty cool... at this point probably there are several uses that i could not imagine, but the first that comes to mind for managunz are to perform the autoinstalls of that "create shorcut PKG"
Actually, if at some point what im saying works there is no need to create a PKG with them... you could just copy the files to dev_hdd0/game/whatever, update the database and voila... you have a new app installed without need to use the PKG format

This "autoinstalls" are just an addon anyway
What i was saying initially about managing/updating the "PSP launcher" installation can be made by managunz by preparing the PKGs... and after that the user needs to install them
Not so user friendly, too much open to "user errors", could cause confusion, but well... it should work

 

[DeViL303]

I was having this kind of errors many times while doing experiments with PARAM.SFO files, im not completly sure if is going to happen when trying what you are suggesting, but im guessing is going to happen
So... this is why i think is really needed to change the PARAM.SFO

Well swapping the param.sfo is simple, so not even worth discussing too much, every patch to the datbase just needs a param.sfo to match it.

Actually this could be used to our advantage, what we could do , is make it so the param.sfo , actually gets replaced by one that has category SF, so this way the psp launcher can be hidden when not required. Then when game is mounted it appears, with name of game also patched in. Then when different type of game is mounted it gets put back to SF.

This way the launcher appears when required just like any other disc, it could even have a UMD icon.

 

[sandungas]

Well swapping the param.sfo is simple, so not even worth discussing too much, every patch to the datbase just needs a param.sfo to match it.

Actually this could be used to our advantage, what we could do , is make it so the param.sfo , actually gets replaced by one that has category SF, so this way the psp launcher can be hidden when not required. Then when game is mounted it appears, with name of game also patched in. Then when different type of game is mounted it gets put back to SF.

This way the launcher appears when required just like any other disc, it could even have a UMD icon.

Hmmm, following that way of thinking... we could have several different "PSP launcher" installations, everyone of them with different EBOOT.BIN and PARAM.SFO for all the know boot modes found by the scene (at this point i dont know how many there are)

This way there is no need to replace the EBOOT.BIN
And for the PARAM.SFO is needed to change the TITLE for the "active" game, the CATEGORY (for "PSP Minis" or "PSP Remasters"), and the ATTRIBUTE (for remote play and extra stuff)
And hiding all other PARAM.SFOs with category SF

Not sure if is going to be a bit complicated, but initially not bad... something that worths to be considered

 

[DeViL303]

I much prefer the idea of having one and patching it to change it to all that are required, multiple hidden launchers is complicated.

Swapping eboot and param.sfo is the easy bit really.

 

[sandungas]

I much prefer the idea of having one and patching it to change it to all that are required, multiple hidden launchers is complicated.

Yeah, i dont like it much either, it was excesive to have all that hidden installations, the idea came to mind because im thinking is going to be needed to replace the EBOOT.BINs, i think that EBOOT.BINs have some "magic" inside \o/

Btw, if i remember right @habib made one or several of them ?. If you are reading this, do you keep a record at how many different EBOOT.BINs exists ?, or why there was people making different versions of it ?

Incase someone makes a list at how many of the "PSP launchers" exists we can collect the info from the PARAM.SFOs easilly (and by now the only critical info seems to be CATEGORY)
But the EBOOT.BINs i have no idea

 

[Zar]

But you did an "autoinstall" in one of the previous managunz versions (and later changed it) or im dreaming ?, i thought you did at some point (as something experimental)

It was for updating ManaGunZ, I was extracting the pkg directly in /dev_hdd/game .
I removed it for 2 reason :
- my old server didn't allow hosting files...
- the database wasn't updated, the new param.sfo wasn't cached, so in the XMB the old version was displayed, an I had some feedback 'the update isn't working'. Most of the time when they restart the new version was displayed....
So, when i'll work on it, I want to update the cached param.sfo too...

 

[DeViL303]

habib made one or several of them ?. If you are reading this, do you keep a record at how many different EBOOT.BINs exists ?, or why there was people making different versions of it ?

Incase someone makes a list at how many of the "PSP launchers" exists we can collect the info from the PARAM.SFOs easilly (and by now the only critical info seems to be CATEGORY)
But the EBOOT.BINs i have no idea

Well, having a set of patches and eboot.bin and param.sfo that go together is easy enough, swapping files is the easy bit.

Changing the database is hard bit, but i actually think it is easy, all it needs to do is scan for the Content ID, than patch in the whole section. Its not just the 2 letters that are different, see my db files for examples, it changes quite a bit in that section, i think they easiest way is to just patch in the whole block. So the whole entry gets replaced every time.

so literally patch in this whole section every time , with name of game added: and of course it will be in different place every console, so scan required.

 

[sandungas]

It was for updating ManaGunZ, I was extracting the pkg directly in /dev_hdd/game .
I removed it for 2 reason :
- my old server didn't allow hosting files...
- the database wasn't updated, the new param.sfo wasn't cached, so in the XMB the old version was displayed, an I had some feedback 'the update isn't working'. Most of the time when they restart the new version was displayed....
So, when i'll work on it, I want to update the cached param.sfo too...

Ahh, that was it, now i remember it
I never understood how you was doing it, but now i got it, and i was not aware about that problem reported by other users, i thought it was working fine

Another thing related with this that always works fine for me is when enabling the PRIORITY in managunz, now im wondering if this is "indexed" in the database or if is always readed from the real PARAM.SFO

Well, having a set of patches and eboot.bin and param.sfo that go together is easy enough, swapping files is the easy bit.

Changing the database is hard bit, but i actually think it is easy, all it needs to do is scan for the Content ID, than patch in the whole section. Its not just the 2 letters that are different, see my db files for examples, it changes quite a bit in that section, i think they easiest way is to just patch in the whole block. So the whole entry gets replaced every time.

so literally patch in this whole section every time , with name of game added: and of course it will be in different place every console, so scan required.


View attachment 16158

I think you had so many differences because the 2 apps you was installing (using the same TITLE_ID in the same install path, and overriting the old installation) are this ones i mentioned in this post:
https://www.psx-place.com/threads/problems-with-psp-launcher-s.23235/page-2#post-169557

The problem is that PARAM.SFOs are very different, there are many things that differs, so the database had lot of changes for this specific SFO

But initially it should not be so complex... basically what we need to change are just a few things:

CATEGORY
By changing it we can boot as "PSP Minis" or "PSP Remasters", this patch is only 2 bytes, the internal structure of the PARAM.SFO doesnt changes so is very easy to patch and is not problematic

TITLE
Initially, we could forget about this to simplify the patching to the max... because this text string is not critical, right ?
Is just is a cool feature, so we should make an effort to make it work
But keep the idea... we dont really need it for first tests

I can tell you if you change the TITLE by a different string with different number of letters... then there is another value located at the top of the PARAM.SFO that needs to be updated
As example... if the TITLE is "devil" that value is going to store a 0x5 (or a 6 incase it counts the NULL byte at the end of all strings, dont remember right now)... and if you change the TITLE to "gamename" then that value is going to store a 0x8 (or 9 with the NULL byte)

So... in few words, to change the TITLE is needed to apply 2 patches in the database, the title itself, and the value that tells the lenght of the title string

ATTRIBUTE
This is really an extra, at this point im not sure what people enabled for this "PSP launchers", but initially i guess is not needed

PRIORITY
Another extra, but just incase you want to allow to enable/disable it in the future is better to prepare the PARAM.SFOs with it since beginning (this way is going to be stored in database i guess, ready to be patched :P)

---------------------
Not sure if this is all... but if this is all needed to be patched in the database i guess it can be made without needed to process all the database contents (in other words without knowing how the database internal structure works)
Processing the database without knowing his internal structure is a bit dirty, and could be unestable (we are not completly sure at this point), but incase it works it could be handy

 

[DeViL303]

I think you had so many differences because the 2 apps you was installing (using the same TITLE_ID in the same install path, and overriting the old installation) are this ones i mentioned in this post:
https://www.psx-place.com/threads/problems-with-psp-launcher-s.23235/page-2#post-169557

The problem is that PARAM.SFOs are very different, there are many things that differs, so the database had lot of changes for this specific SFO



CATEGORY
By changing it we can boot as "PSP Minis" or "PSP Remasters", this patch is only 2 bytes, the internal structure of the PARAM.SFO doesnt changes so is very easy to patch and is not problematic

The thing, is, i used exact same param.sfo, and only changed the category in param.sfo editor and then updated it, so there should have been no other differences except the 2 letters if your theory was correct, but there are lots of differences. So i think whole block patch will be needed, maybe i am wrong and 2 byres will do, but it does not look that way.

 

[sandungas]

The thing, is, i used exact same param.sfo, and only changed the category in param.sfo editor and then updated it, so there should have been no other differences except the 2 letters if your theory was correct, but there are lots of differences. So i think whole block patch will be needed, maybe i am wrong and 2 byres will do, but it does not look that way.

But most changes happened because the database was updated "in the official way"
Actually, it seems it keeps a record of the last booted app, in your files in HxD when doing a "data comparison" keep pressed F6 key
There are a lot of values related with rebug toolbox that changed, im guessing are related with timestamps ?
It seems you booted rebug toolbox in between
But we dont need to change timestamps

Initially by changing the 2 bytes of the CATEGORY should be enought and should work because is simple, we are not changing the "lenght" of them

With the ATTRIBUTE and PRIORITY is the same stuff, if i remember right the PARAM.SFO doesnt stores the real lenght of them (it stores the max lenght instead, because the firmware considers are fully used, and this "max lenght" never changes)

The tricky one is the TITLE because is going to be needed to change other value somewhere else for his lenght

 

[DeViL303]

Hopefully you are right. I did boot rebug toolbox for FTP

The tricky one is the TITLE because is going to be needed to change other value somewhere else for his lenght

Well, i have an idea for title, so we can have it work all the time regardless of length. What we do is put in long title like XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX in param.sfo , then we patch in the title of game, and put spaces after it to keep length. If its longer than XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX then it gets cut off.

 

[Zar]

Another thing related with this that always works fine for me is when enabling the PRIORITY in managunz, now im wondering if this is "indexed" in the database or if is always readed from the real PARAM.SFO

same 'issue', you must refresh the system to update the database. I can modify the value of the param.sfo in the database but maybe there is an hash, i'm afraid to corrupt the database. I didn't found any info on it in psdevwik, I have to 'decrypt' it myself :p