PS3 PKGi PS3 development

[kozarovv]

Nice work @bucanero

Aaand small bug report. Custom fonts seems to not have JP characters. Maybe is possible to use system font, or add them to current font? Using

 

[sandungas]

afaik, you only need the PARAM.SFO to be indexed. some games (i.e. little big planet) have tens of thousands of files, so when you read how much space it's taking up, it can take a while. to avoid that from happening, sony used an indexer. I think all of the current gen systems use an indexer. last gen - I don't think the 360 used one, which is why it takes so long to scan your internal hdd on the official system menu. though, I have a lot of indie games, and it no longer takes forever as it did in the past. if you're scanning the size in dashlaunch or something, it takes forever again. anyway, afaik, it's only the PARAM.SFO that's important. I modified the PARAM.SFO to alphabetize everything on my system. however, I did find something odd in how the PARAM.SFO editor lists the game name (possibly other things), because a few things wouldn't change unless I hex edited. the final fantasy xiii sequels were like that.

Yes, usually a lot of the info inside the database has been copyed from the PARAM.SFO but this only applyes to games/apps dlc, gamedata, etc... (the kind of contents that uses PARAM.SFO)
But additionally the videos, music, and photos are indexed too, this ones doesnt have a PARAM.SFO, but some of the info in the database about then is common (like timestamps when the content was created, last time accessed, thumbnail size, bytes size... and things like that)

With the .pdb files could be happening the same... they doesnt have a PARAM.SFO but theoretically the database could keep a record about them, im not sure if it does it though, anyway... the idea came to mind because (for simplification purposes) i use to think that all the "content" dsplayed in XMB is loaded from the database files
In other words... when the PS3 boots it reads the databases only... if your custom content has not been correctly indexed in the database then is not displayed

 

[bucanero]

Have you tried refreshing the XMB without rebooting by going to Remote Play and then exiting? Maybe it just needs that and not a full reboot.
Well, I guess the XMB is refreshed when exiting PKGi anyways, idk. Just brainstorming.

I didn't try using the Remote play, I'll give it a try to see if it makes any difference.
Yes, even after exiting PKGi (or another app or game) the "download manager" (on the XMB network tab) doesn't show up. But it seems that after a reboot, the system "discovers" the .pdb files and adds them properly.

Hmmm, i guess is needed to index the .pdb file into the databases of the PS3
Im saying "databases" (in plural) because the PS3 uses several, as far i know are the files located under path dev_hdd0/mms/
metadata_db_hdd <--- this file stores info about all installed games/apps, game updates, videos, and photos (maybe something more)
There are others, but right now i dont remember the contents of each

I'll check those extra directories and see if something changes between "creating the d0.pdb files" and the next reboot. At least the information could be useful to know.

I had forgotten about them not showing up until after a reboot. I was thinking we had found a way to refresh the xmb and get them working but I'm not finding anything.

oh, too bad I saw that in your code, at some point you had some functions to reboot the PS3, but they were commented out and not in use. Your code on Github wasn't rebooting the console, but I couldn't find anything to "refresh" the XMB either.

 

[bucanero]

Nice work @bucanero

Aaand small bug report. Custom fonts seems to not have JP characters. Maybe is possible to use system font, or add them to current font? Using

oh yes, the font is a standard ASCII charset, no JP stuff. I know that libfont supports loading TTF fonts, but I haven't tried loading a TTF font with japanese characters.
If there's any code example to show Jap text used in any other homebrew, let me know so I can check it out and give it a try.

 

[bucanero]

btw, I'm going to prepare a new version release soon, including some fixes and features I added lately

as soon as it's ready, I'll publish code and binaries at GitHub... stay tuned

 

[sandungas]

I been taking a good look at what i mentioned in my previous post, and i changed my mind a bit... long story short, i dont think the .pdb files are stored in the XMB databases
The reason why im saying this is because there is not any database codename related with them, neither the xCB functions (used to acess the database)

So... doesnt seems posible to scan the database searching for pdb files... and the reason for that most probably is because the databases doesnt allows to store info about the .pdb files... as simple as that
But something worthy to be mentioned is the .pdb files itselfs are working a bit like an "extension" of the database... in some way it looks over 90% of the info stored in the .pdb files have a direct relationship with the database "fileds" (available to store that kind of info)

Actually, what happens is when the download is completed (and the pkg installed and indexed or whatever file you downloaded, could be just a video)... at that point most of the info from the .pdb is copypasted to the database

Something really interesting is the ID's used in the .pdb files should have a direct relationship with the "fields" of the database, i mean the ID's that appears in psdevwiki
https://www.psdevwiki.com/ps3/Project_Database_(PDB)
And the ones mentioned in this thread
https://www.psx-place.com/threads/pdb-research-on-background-downloads.16408/

That IDs have an official codename that can be seen in the database fields, as example, in the wiki page there are a few added by @mysis
Video:Video.reserved30
Video:Common.reserved24
Video:Video.reserved29

This is exactly the kind of paths (and the accurate codenames) that needs to be used to access the database contents
You can see how this kind of paths are used in the:
https://www.psdevwiki.com/ps3/XMBML_Files
https://www.psdevwiki.com/ps3/Registory.xml
I wonder how he found them though, lol... im trying to imagine it because (incase they appears in some database file) is posible to identify all them

All that IDs are actually optional, as pink1 proved, and this opens a few posibilities
The "failproof" way (what most people that was doing) is to use the same IDs than the PSN store when you download that kind of content
There is an alternative... what pink1 suggested... trying to remove the uneeded IDs
But there is another alternative... the fact is the databases reserves data "slots" for a lot of fields so we could add other fields in the .pdb (that are going to be copypasted into the database when the download is completed), this opens a lot of posibilities, keep in mind the pdb files are giving us access to update the database with other contents different than PKG (videos, photos, music, playlists, vmc, game patches, dlc, etc...)

 

[bucanero]

nice findings @sandungas , I think it's really interesting to see which information we can discover from these files.

As a small example, by doing some simple test, I found out that entry ID 0x65 is related to the task activity status.
If it's equal 0, means "task is paused", and = 1 means "task is active and downloading". (I've updated the psdevwiki accordingly)

I still couldn't get the task to load without a reboot, but at least now when you reboot, the task is downloading automatically (without the need to "un-pause"/resume the task)

perhaps we should open a new thread on the forum and put all the information there, so it doesn't get lost

 

[sandungas]

Nice, that ID 0x65 is a good example of an optional feature that is good to use, probably there are more that could be handy is just we dont know them

At this point i think the ID's starting from 0x60 up to 0x80 or so are the most importants, and are not intended to be copypasted to the XMB database

And all the other ID's starting from 0xC0 or so are intended to be copypasted to the database, im not going to bet my life in this but it looks like that, i have doubts because they are a lot (the database allows for hundreds, as example a "Game" content have a max of 139 "slots" to store info about it)
The good thing about this ones is we can identify them by indexing a content in the database when we install a PKG and then taking a look at the database to see the official codenames where has been stored the data of the PKG
This method could be tedious, but im 100% sure it allows to identify a bunch of ID's

 

[bucanero]

first complete version (v1.0.0) has been released:
https://www.psx-place.com/threads/pkgi-ps3-v1-0-0-released.27354/

(download it from https://github.com/bucanero/pkgi-ps3/releases/tag/v1.0.0 )

 

[Zar]

oh yes, the font is a standard ASCII charset, no JP stuff. I know that libfont supports loading TTF fonts, but I haven't tried loading a TTF font with japanese characters.
If there's any code example to show Jap text used in any other homebrew, let me know so I can check it out and give it a try.

https://github.com/Zarh/ManaGunZ/blob/master/MGZ/source/main.c#L2408

 

[bucanero]

thanks @Zar , I'll try that code and see if I can add support to non-ascii characters.

 

[zecoxao]

@bucanero can confirm that the download is running fine now with extra large files

 

[bucanero]

@bucanero can confirm that the download is running fine now with extra large files

that's good news!
btw, did you download the large file with the internal "in-app" download, or using the background task method?

btw, I'd like to hear experiences regarding download speeds too (e.g. in-app is slower than background task or vice-versa)

 

[zecoxao]

that's good news!
btw, did you download the large file with the internal "in-app" download, or using the background task method?

btw, I'd like to hear experiences regarding download speeds too (e.g. in-app is slower than background task or vice-versa)

currently with "regular" download i get speeds of 2MB/s or 1.7MB/s at worst. gonna try with background task now...
edit: with background task i get 2.2MB/s

 

[bucanero]

thanks @zecoxao , it seems that the speed is slightly better with background download, but still quite similar

also, if anyone else is trying the background download, please share your experience. I've got report from one user that the tasks are not showing after reboot, so I'm wondering if there's some issue there

 

[iesus gamer]

Hello very good is to ask a question the normal download is going on my PS3 Súperslim at 7.6Mb and another thing I do not use the download in the background could you explain the steps thanks to me. Greetings to all and especially Bucanero great contribution.

 

[Lion_4K]

Interesting, guess what you just got yourself banned. Congrats
Seems you were inquiring about piracy and i seen the site censor
Changed your piracy app into ****PlayStation Store**** ( I still love that )

I don't have patients with members who been here awhile and know the rules.
well new members either as they are sent them when activating their accounts.

our rules will be respected or you can move along.

Sorry if you understood that way.
I wasn't inquiring about it in piracy way, but more to those who were PSN banned and can't use their rightfully owned packages.

And I saw that PKGi could be an ally in that by making it easy.

But I'm not gonna argue any further, since you can easily ban me again, so yeah, you earned my fear, I mean respect.

 

[bucanero]

Hello very good is to ask a question the normal download is going on my PS3 Súperslim at 7.6Mb and another thing I do not use the download in the background could you explain the steps thanks to me. Greetings to all and especially Bucanero great contribution.

hello @iesus gamer , thanks for the information regarding DL speeds.

No worries, I need to update the Readme on Github to explain the background DL better

So, steps to use the background download:

1. (press Triangle) Open the menu
2. on the options, choose "DL mode:" > "Background"
3. press Triangle to close and save new settings

(just to be sure, go to the menu again and see if "Background" download is selected.)

Now that you have background download activated, just go and select any item in your list to download.
The app will create an empty PKG file, and create some additional index files for the new task.

The only issue right now is that the task doesn't show up immediately in the XMB. After you queue all the items you want for background download, just reboot the PS3. When the console boots up, you'll find all the download tasks in progress in "Network" > "Download Manager".

 

[iesus gamer]

hello @iesus gamer , thanks for the information regarding DL speeds.

No worries, I need to update the Readme on Github to explain the background DL better

So, steps to use the background download:

1. (press Triangle) Open the menu
2. on the options, choose "DL mode:" > "Background"
3. press Triangle to close and save new settings

(just to be sure, go to the menu again and see if "Background" download is selected.)

Now that you have background download activated, just go and select any item in your list to download.
The app will create an empty PKG file, and create some additional index files for the new task.

The only issue right now is that the task doesn't show up immediately in the XMB. After you queue all the items you want for background download, just reboot the PS3. When the console boots up, you'll find all the download tasks in progress in "Network" > "Download Manager".

Thanks in the end I found the form but anyway thank you very much for answering.

 

[arcadekidflo]

With normal mode i get download speed of ~ 1.5MB/s...still slow . After download , the pkg can be installed from Package Manager .
With background mode something strage happens.. it displays dl speed of 7.5MB/s but , after finishing , there's no pkg . After a reboot the pkg shows in the XMB , downloading in the background . After completing , you need to install it . So it's not faster at all .