WebMAN-MOD pkg handling commands

ok I see. thats much easier, Ive been used to doing it the other way for XMBPD as it required the link to end in .pkg/.xml etc, that involved changing the "www" to "dl" and changing "dropbox" to "dropboxusercontent", and removing the "dl=0".

It was a right pain.
 
bitsbubba said:
I'll have to look try out things later this week as it's my extended weekend with my kid (plus kinda caught up in PSTVita buzz) but I should be able to later this week (if work allows)
Click to expand...
You should see the smile on my face... [emoji23]

However your kid comes first of course... It goes without saying... but it's better saying it!

Sent with Tapatalk
 
Last edited:
DeViL303 said:
ok I see. thats much easier, Ive been used to doing it the other way for XMBPD as it required the link to end in .pkg/.xml etc, that involved changing the "www" to "dl" and changing "dropbox" to "dropboxusercontent", and removing the "dl=0".

It was a right pain.
Click to expand...
It's the same gymnastics with github files actually...

Sent with Tapatalk
 
I might have a possible solution to the issue of partial downloads remaining when the download is canceled. What seems to happen is that the file is downloaded to the set location, but it is then moved on completion if it is a pkg, to the vsh/game_pkg folder (if background download is selected), The issue with partial downloads only shows up because we are downloading to the packages folder normally, and we can see that folder from inside Package Manager, if the file was to download to a temp folder that the package manager couldn't see this would solve this issue. (I know that when using the XMBPD method the system uses dev_hdd0/tmp/np_pkg for this purpose and then the package gets deleted when its auto installed but that probably doesn't help in this case)

But for this to work then background downloading would have to be forced somehow or a move operation performed after download, so the file then always gets moved to a location where it can be seen and installed from. If forcing background download for every pkg/file is possible this would also solve the issue of pkgs going to different locations so it could be a good all round solution, Rename the "playstation network content" section to "downloaded packages" or something like that and both problems are kind of solved..

Also Im not sure if this is possible but as you were saying we have no way of knowing when a download has finished, if background download was forced somehow, then maybe the message saying "go to [Game] to install" could be used as a trigger for the follow up operation, move/install etc..I suppose it still all comes back to the problem of no call back or options available to force the download in background.
 
Last edited:
FYI

In code, if you don't specify any path when you call the download plugin function from wMM or whatever brew, by default the system uses /dev_hdd0/tmp/downloader as a download path.

The reason why it moves the pkg files automatically with background downloading is that the plugin assumes the files are games/dlc.
Everything else would normally be downloaded to usb storage & background downloading wouldn't be allowed.
Because of this, s#ny didn't bother implementing any other checks that should alter the behaviour. As a result the pop up message is displayed as well whether it's appropriate or not.

Also the download function does not check whether the given url points to a valid file. A wrong URL can sometimes lead to a default error page & not just an error code for instance, in which case the plugin will download whatever the server sends to it & it will create a garbage file in the folder...
The only way around that would be to implement some type of validation but after discussing this with Aldo, we came to the conclusion that it was the user's responsibility to give proper urls!


Sent with Tapatalk
 
Last edited:
have you had any luck getting it to download an xml to hdd from dropbox or any filehost, can you share a working link?

Seems to work ok for pkgs so far.
 
DeViL303 said:
I might have a possible solution to the issue of partial downloads remaining when the download is canceled. What seems to happen is that the file is downloaded to the set location, but it is then moved on completion if it is a pkg, to the vsh/game_pkg folder (if background download is selected), The issue with partial downloads only shows up because we are downloading to the packages folder normally, and we can see that folder from inside Package Manager, if the file was to download to a temp folder that the package manager couldn't see this would solve this issue.

But for this to work then background downloading would have to be forced somehow or a move operation performed after download, so the file then always gets moved to a location where it can be seen and installed from. If forcin g background download for every pkg/file is possible this would also solve the issue of pkgs going to different locations so it could be a good all round solution, Rename the "playstation network content" section to "downloaded packages" or something like that and both problems are kind of solved..

Also Im not sure if this is possible but as you were saying we have no way of knowing when a download has finished, if background download was forced somehow, then maybe the message saying "go to [Game] to install" could be used as a trigger for the follow up operation, move/install etc..I suppose it still all comes back to the problem of no call back or options available to force the download in background.
Click to expand...

Actually the packages being downloaded are shown in the Package Manager due they have a .pkg file extension.

A possible workaround to this issue is to add a special file extension (e.g. .~dl) to the pkgs downloaded with /download.ps3 and make wMM scan these files in the /dev_hdd0/packages every 6 seconds (only if the user is on XMB). This can be done

The best way to know if a pkg have downloaded completely is comparing the current size of the file with the size in the header at offset 0x18. If the size of the file matches with the size in the header, then the file extension can be removed.

I implemented this algorithm in this test build, but I haven't tested it yet.
http://aldostools.org/temp/test/webftp_server_full.sprx

@bguerville the current code of wMM doesn't download to /dev_hdd0/tmp/downloader
The default download folder (if ?to=<path> is not provided) is /dev_hdd0/packages/

Maybe your original code used /dev_hdd0/tmp/downloader but I preferred to simplify the algorithm to use a single download path for everything.
 
aldostools said:
@bguerville the current code of wMM doesn't download to /dev_hdd0/tmp/downloader
The default download folder (if ?to=<path> is not provided) is /dev_hdd0/packages/

Maybe your original code used /dev_hdd0/tmp/downloader but I preferred to simplify the algorithm to use a single download path for everything.
Click to expand...

Sorry Aldo but... No. You didn't understand what I meant... Or maybe my explanation is confusing...
Am talking about the download plugin function that wMM calls, not the wMM implementation...
That function defaults to /dev_hdd0/tmp/downloader if no valid path is passed to it. Nothing to do with my original wMM code whatsoever in which the default path was already /dev_hdd0/packages...
You never changed that part... [emoji6]
Test it yourself if you don't believe me...


As to using the clock instead of a callback & scanning folders for monitoring purposes as a workaround, I have said it already, unless there is absolutely no other choice I really don't like these implementations, at all, but it's up to you..

Sent with Tapatalk
 
Last edited:
bguerville said:
Am not at home just now so I can't try but this should work...

http://www.dropbox.com/s/wba7482vne8wtfb/Package_Manager.xml?dl=1

Sent with Tapatalk
Click to expand...

Can you tell me if this looks right? its not working, getting "an error has occurred during the download operation. (80710A06)"
Code: 
http://127.0.0.1/download.ps3?to=/dev_hdd0/game/XMBMANPLS/USRDIR/FEATURES&url=http://www.dropbox.com/s/wba7482vne8wtfb/Package_Manager.xml?dl=1

I think it doesn't like any dropbox links...even zip/pkg files from dropbox, using http, and making sure to add the dl=1, still don't work.

aldostools said:
Actually the packages being downloaded are shown in the Package Manager due they have a .pkg file extension.

A possible workaround to this issue is to add a special file extension (e.g. .~dl) to the pkgs downloaded with /download.ps3 and make wMM scan these files in the /dev_hdd0/packages every 6 seconds (only if the user is on XMB). This can be done

The best way to know if a pkg have downloaded completely is comparing the current size of the file with the size in the header at offset 0x18. If the size of the file matches with the size in the header, then the file extension can be removed.

I implemented this algorithm in this test build, but I haven't tested it yet.
http://aldostools.org/temp/test/webftp_server_full.sprx
Click to expand...

Wow, that sounds like a great solution, and implemented already, great, I will give it some testing later.
 
Last edited:
@DeViL303
DeViL303 said:
I think it doesn't like any dropbox links...even zip/pkg files from dropbox, using http, and making sure to add the dl=1, still don't work..
Click to expand...
The url does look right at first glance...
Have you tried a different file with the same download path? Does a link to a pkg instead of xml file work?
I used Dropbox among other download sites when I tested the commands during development & I can't remember ever having that error code....
I will have a look in a couple of hours when I get home.

DeViL303 said:
Wow, that sounds like a great solution, and implemented already, great, I will give it some testing later.
Click to expand...
The check to ensure a downloaded file is complete is clever of course but it still requires constant & regular scanning of the folder(s) + associated calculations according to the clock! Unless I am missing something...



Sent with Tapatalk
 
Last edited:
Yeah, tried quite a lot of combinations now, different file same path, different path same file, even no path (so hdd0/packages) with a .pkg on dropbox doesn't seem to be working.

Im fairly sure its dropbox as I havnt got any file to come down off there yet. Or its something else on my end..
 
DeViL303 said:
Yeah, tried quite a lot of combinations now, different file same path, different path same file, even no path (so hdd0/packages) with a .pkg on dropbox doesn't seem to be working.

Im fairly sure its dropbox as I havnt got any file to come down off there yet.
Click to expand...
I have a pkg on Dropbox that I use for testing.
I downloaded it again yesterday several times when you asked about the location of a background download....
It could well be a url syntax issue.
I will send you my pkg url as soon as I can..

Sent with Tapatalk
 
ok thanks. Please give me the full link inc WM code incase I am making some silly mistake there, but its only 1 line so im not sure how, maybe its because I am calling it from an xml, and not entering it into the browser directly. The xmbmods.co pkg links work fine called from an xml so I doubt its that..

but regardless of your working link, these links should be working for you too hopefully, if you can test one of them on your ps3 if you get chance.

https://www.dropbox.com/s/t3uqgep832n1lo9/Mega_Drive_Genesis.pkg?dl=1
https://dl.dropboxusercontent.com/s/t3uqgep832n1lo9/Mega_Drive_Genesis.pkg
http://www.dropbox.com/s/t3uqgep832n1lo9/Mega_Drive_Genesis.pkg?dl=1
http://dl.dropboxusercontent.com/s/t3uqgep832n1lo9/Mega_Drive_Genesis.pkg
 
bguerville said:
@DeViL303
The check to ensure a downloaded file is complete is clever of course but it still requires constant & regular scanning of the folder(s) + associated calculations according to the clock! Unless I am missing something...
Click to expand...

Yes it requires a constant check that is done every 7 seconds ONLY under the following conditions:
If the user started the download of a pkg with /download.ps3 *and* the system is on XMB

Before start the download, .~dl is appended to the download path and a counter increments in 1.

When the pkg has downloaded, the file is renamed back to .pkg and the counter decrease in 1.

This is the complete code (untested):
Code: 
#ifdef PKG_HANDLER
		// Poll downloaded pkg files
		if((pkg_dcount > 0) && (sec & 1) && (gTick.tick == rTick.tick))
		{
			CellFsDirent entry; u64 read_e; int fd; u16 pkg_count = 0;

			if(cellFsOpendir(TEMP_DOWNLOAD_PATH, &fd) == CELL_FS_SUCCEEDED)
			{
				while((cellFsReaddir(fd, &entry, &read_e) == CELL_FS_SUCCEEDED) && (read_e > 0))
				{
					if(!extcmp(entry.d_name, ".pkg", 4))
					{
						int fdl = 0; char *dlfile = msg; _pkg_header pkg_header;
						sprintf(dlfile, "%s%s", TEMP_DOWNLOAD_PATH, entry.d_name); pkg_count++;
						cellFsChmod(dlfile, MODE);

						if(cellFsOpen(dlfile, CELL_FS_O_RDONLY, &fdl, NULL, 0) == CELL_FS_SUCCEEDED)
						{
							if(cellFsRead(fdl, (void *)&pkg_header, sizeof(pkg_header), NULL) == CELL_FS_SUCCEEDED)
							{
								cellFsClose(fdl);

								struct CellFsStat s;
								if(cellFsStat(dlfile, &s) == CELL_FS_SUCCEEDED && pkg_header.pkg_size == s.st_size)
								{
									char pkgfile[MAX_PATH_LEN];
									sprintf(pkgfile, "%s%s", DEFAULT_PKG_PATH, dlfile + strlen(TEMP_DOWNLOAD_PATH));
									cellFsRename(dlfile, pkgfile);
									pkg_dcount--;

									if(pkg_auto_install) installPKG(pkgfile, msg);
								}
							}
							else
								cellFsClose(fdl);
						}
					}
				}
				cellFsClosedir(fd);

				if(pkg_count == 0) pkg_auto_install = pkg_dcount = 0; // disable polling if no pkg files were found (e.g. changed to background download)
			}
		}
#endif

Note: (sec & 1) is true only every 7 seconds and (gTick.tick == rTick.tick) is true only on XMB :

EDIT:
I forgot to tell that this code is performed in the thread that polls the combos. It's executed after checking the pads for 3 seconds (+ .5 second reading the temperature). The check can be done 2-3 seconds later if some combos are used.
 
Last edited:
@aldostools
I was wondering if you were creating a thread just for that purpose or reusing an existing one like combo or temp....Thanks for clearing that up.. [emoji6]
The combo thread is appropriate of course.

Sure, your implementation should work & it's quite smart too. It's basically based on the same workaround concept I was speaking about with @DeViL303 & even though am no big fan of it, this workaround could be used for all post download processing including launching an installation...

Sent with Tapatalk
 
Last edited:
@aldostools

What about using the same workaround to send a circle button pressed event to the system to get rid of the page to close manually at the end of a pkg download paving the way for an automatic installation?
We would need to add a 3rd parameter in the url though to choose between automatic install or simple download....
In case of auto install, the pkg could be either kept or deleted...?

There's one problem I can think of with that though....
What if the user initiates background download... Files get moved automatically & there's no page to close...
We would also need to check if there is any change with the loaded sprx system plugins when the system switches to background downloading. If there is a change then we can detect it & act consequently...

Even with your current code actually, what happens then...?

Sent with Tapatalk
 
Last edited:
If that's possible, what about sending a down and X button pressed event first so that background download is always initiated, or to have a option for that would be cool at least.

If a background download could be initiated every time it would solve the problem of pkgs going to different locations too.
 
DeViL303 said:
If that's possible, what about sending a down and X button pressed event first so that background download is always initiated, or to have a option for that would be cool at least.
Click to expand...
Sure.
It requires a 4th parameter.. [emoji5]️

Edit:
In fact we could give the 3rd parameter (auto install) 3 or more possible values rather just a true/false & 3 parameters may suffice. It depends though because the 3rd parameter I imagined was only for pkg files whereas it could be good to use automatic background downloading for other big files like iso...

The only obstacle to background downloading that I have noticed so far is when the server doesn't provide a file size to the download plugin. In that case background downloading cannot be initiated...
Unfortunately this happens with a number of well known servers...
There might also be a minimum file size to initiate background downloading, I am not 100% sure...

Sent with Tapatalk
 
Last edited:
You must log in or register to reply here.

Similar threads

Back
Top