WebMAN-MOD pkg handling commands

bguerville

bguerville

Moderator
Developer
Important:

Users who want to use these new Web commands can now do so simply by installing the new webMAN-MOD release 1.43.31 from @aldostools.

http://www.github.com/aldostools/webMAN-MOD/releases/tag/1.43.31

Any other mirror still in use in this thread will contain experimental builds & are aimed at testers only.

Also the syntax has changed with time so make sure you refer to the latest syntax when you use the commands.

=============================================

Hi everyone.
I am currently working on new wMM commands to download & to install pkg files.
This is an ongoing project which hopefully could lead to full pkg handling automation via wMM.

To be clear, I am taking little credit for this. The core of the features comes from the research done by mysis & co on the PS3 system plugins so if these features are found reliable enough to be included in wMM, thank them because without their research, I would not have been able to implement these cool commands..

I would be grateful if some of you guys would try the commands using the sprx linked below & report whatever problems you may have experienced when using the commands .
The error handling for these features is to be improved but should be sufficient for testing.

Here is the syntax to be used:

1. Download

ps3-ip-address/pkg.ps3$downloadfrom=

Replace by a valid url, use a full url with http or https
The downloaded files can be found by default in /dev_hdd0/tmp/downloader/ for the moment. Choosing the path to store the downloads to will come soon...
Background downloading works also.

There is no check regarding the downloaded file type so the command can be used to download files other than pkg. I am leaving it like this for now so the feature can easily be tested with other files but ultimately a separate command for non pkg file downloading will most likely be created.

2. Install

ps3-ip-address/pkg.ps3$install=

Replace by the path of the pkg you want to install.

The pkg commands can be used like any other command in wMM including with a custom combo using
GET /pkg.ps3$......

Download: (removed)
There are 2 editions for testing just now (you can ask me for another if you want).

The webftp_server.sprx Full Edition with pkg handling & most other features enabled.

The other one, webftp_server_cobraonly.sprx, is a Cobra/no PS3MAPI (non Rebug) lighter version of wMM with pkg handling but fewer other features.

Edit:
This download urls remain the same but the sprx files get updated as I go along so keep checking for updated versions. This past hour I fixed the couple of errors I had found yesterday & started to work on other improvements.

Many thanks in advance to all testers.

Sent from my GT-I9305 using Tapatalk
 
Last edited:
No0bZiLLa said:
no ps3 sadly. looks neat though.
Click to expand...
Well I hope it can be real in the end.

Aldo warned me of problems with this thing and I absolutely trust his judgement so it might not be such a straight road.
But so far it all seems to work fine for me on my Rebug 4.78.2. I tested each feature 2 dozen times today & I froze the xmb once. I wasn't able to reproduce the freeze either.

As I said the error handling needs improved for sure. I have noticed a couple of problems to fix already. So honestly it's not finished
.
Btw the pop-up messages are for debug info & are not there to stay, or at least not in that state.

I wish I had more time to take care of most of the testing myself but it's not the case. I am sure some guys here will be happy to give me a hand in trying to get this thing working properly.

Sent from my GT-I9305 using Tapatalk
 
Last edited:
Here is an update to this project of mine. Progress is slow due to lack of time...as always.
However this update takes the shape of a complete webMAN-MOD updater pkg this time.
I updated my project to the 1.43.29 to include 4.80 CFW support.

Mirror 1

As far as I know the pkg download command is ok now. It seems to work well even though I don't really like the way I currently use to make it so...but still it works..

In order to choose the path in which to store the downloaded file, I have created a new optional parameter: "downloadto".
If the parameter is omitted or the path is wrong, wMM will try successively and in that order to store the file in
/dev_hdd0/packages
/dev_hdd0/tmp/downloader
I wanted to include /dev_hdd0 in that list but the system download plugin defaults to
/dev_hdd0/tmp/downloader automatically if given the internall HDD root path...

Now the pkg download command syntax is:
hxxp://<ip-console>/pkg.ps3$downloadfrom=<pkg-file-url>&downloadto=<storage-path>
(Don't include the <> characters!)

The pkg install command I am unsure of, I experienced one single unexplained XMB freeze during my latest tests but I could not reproduce it even by following the same steps again...So I dunno...
Its syntax remains:
hxxp://<ip-console>/pkg.ps3$install=<pkg-path>
(Don't include the <> characters!)

Testing was limited. Done only on one slim console and one CFW (Rebug 4.80.1 CEX mode).

I expect that some of you will have issues I have not experienced yet...
Your feedback would be most welcome.:tranquillity:


Edit:
Download link deleted.
The file & the information in this post are both outdated.
Please read on for the new release & change of syntax.
 
Last edited:
Any chance of seeing the new .30 used as a base? aldos new naming has really helped me with all ym kids LEGO games and what not but regardless this is a really neat feature!! I will guive it a go after I get my system back in shape after I re-installed a new MFW last night.
 
atreyu187 said:
Any chance of seeing the new .30 used as a base? aldos new naming has really helped me with all ym kids LEGO games and what not but regardless this is a really neat feature!! I will guive it a go after I get my system back in shape after I re-installed a new MFW last night.
Click to expand...
Well I was going to merge with the .30 source but aldostools is about to release .31 so I thought I would wait for it... [emoji6]

Sent with Tapatalk
 
bguerville said:
Well I was going to merge with the .30 source but aldostools is about to release .31 so I thought I would wait for it... [emoji6]

Sent with Tapatalk
Click to expand...

The latest test build .31 was canceled, due the bug was caused by ps3netsrv. So the fix in webMAN was not needed and would make the scanning of content stored on ps3netsvr less efficient.

If you want to publish your source code as it is, I can merge it with the project (with credits to you & mysis of course).

I tested the pkg installer, and it seems to be working properly.
However, the I get "Web Command Syntax error" with the download command. I tested your full edition.

When I install a pkg, there is cosmetic bug: a "</" after the return button.

I'd prefer the following syntax:
/install.ps3<path-of-pkg> (e.g. 10.0.0.8/install.ps3/dev_hdd0/packages/movian.pkg)
/install.ps3?url=<url-of-pkg> (this command would download and install the pkg)

/download.ps3?url=<url-of-file>
/download.ps3?to=<path>&url=<url-of-file>

The reason of having "?to=" first is to reduce issues with URIs containing parameters.
If "?to=" is omitted, I think it's better to create the folder, instead of try to use multiple possible paths.
 
aldostools said:
The latest test build .31 was canceled, due the bug was caused by ps3netsrv. So the fix in webMAN was not needed and would make the scanning of content stored on ps3netsvr less efficient.

If you want to publish your source code as it is, I can merge it with the project (with credits to you & mysis of course).

I tested the pkg installer, and it seems to be working properly.
However, the I get "Web Command Syntax error" with the download command. I tested your full edition.

When I install a pkg, there is cosmetic bug: a "</" after the return button.

I'd prefer the following syntax:
/install.ps3<path-of-pkg> (e.g. 10.0.0.8/install.ps3/dev_hdd0/packages/movian.pkg)
/install.ps3?url=<url-of-pkg> (this command would download and install the pkg)

/download.ps3?url=<url-of-file>
/download.ps3?to=<path>&url=<url-of-file>

The reason of having "?to=" first is to reduce issues with URIs containing parameters.
If "?to=" is omitted, I think it's better to create the folder, instead of try to use multiple possible paths.
Click to expand...

I forgot to say that at the moment, I have restricted the possible file type to be downloaded to only .pkg. However the command is capable to download other files.. Eventually I could authorise all file types to be downloaded. Therefore a download.ps3 command makes sense...
No problem about the syntax, I will make the modifications you are asking for. [emoji1]

Also the command does try to create the folder first if the path is wrong or omitted & only tries to use the next path in line if folder creation fails!


Strange about this error with the download command, I will check this as soon as I get home.....

Also I will pm you the source code after making the changes & ensuring the download command works.
At the same time you can give me your thoughts about the workaround used to avoid the XMB freezes with file download to see if you can think of alternatives...



Sent with Tapatalk
 
Last edited:
aldostools said:
The latest test build .31 was canceled, due the bug was caused by ps3netsrv. So the fix in webMAN was not needed and would make the scanning of content stored on ps3netsvr less efficient.

If you want to publish your source code as it is, I can merge it with the project (with credits to you & mysis of course).

I tested the pkg installer, and it seems to be working properly.
However, the I get "Web Command Syntax error" with the download command. I tested your full edition.

When I install a pkg, there is cosmetic bug: a "</" after the return button.

I'd prefer the following syntax:
/install.ps3<path-of-pkg> (e.g. 10.0.0.8/install.ps3/dev_hdd0/packages/movian.pkg)
/install.ps3?url=<url-of-pkg> (this command would download and install the pkg)

/download.ps3?url=<url-of-file>
/download.ps3?to=<path>&url=<url-of-file>

The reason of having "?to=" first is to reduce issues with URIs containing parameters.
If "?to=" is omitted, I think it's better to create the folder, instead of try to use multiple possible paths.
Click to expand...

Sorry I made a typo error with the delimiter syntax I posted earlier. I edited the post.

The delimiter to use with the download command is '$' NOT '?' .

For example:
hxxp://<ip-address-console>/pkg.ps3$downloadfrom=<pkg-url>&downloadto=<storage-path>

Sent with Tapatalk
 
@aldostools

I would like to check whether the passed url exists before launching the download itself.
What do you think of this type of check?

CellHttpClientId client = 0; CellHttpTransId trans = 0; CellHttpUri uri;
uri.scheme="<httpmode>"; uri.hostname="<hostname>"; uri.username="";
uri.password="";
uri.path="<urlpath>";
uri.port=<port>;
int code = 0;
int ret;
ret = cellHttpCreateClient(&client);
if(ret<0)
{
sprintf(msg_durl,"An error occurred when checking the Url. Could not create client. \nReturn value: %d\n",ret);
show_msg(msg_durl);
goto end_download_process;
}
ret = cellHttpCreateTransaction(&trans, client, CELL_HTTP_METHOD_GET, &uri);
if(ret<0)
{
sprintf(msg_durl,"An error occurred when checking the Url. Could not create transaction. \nReturn value: %d\n",ret);
show_msg(msg_durl);
goto end_download_process;
}
ret = cellHttpSendRequest(trans, NULL, 0, NULL);
if(ret<0)
{
sprintf(msg_durl,"An error occurred when checking the Url. Could not send request to client. \nReturn value: %d\n",ret);
show_msg(msg_durl);
goto end_download_process;
}
ret = cellHttpResponseGetStatusCode(trans, &code);
if(ret<0)
{
sprintf(msg_durl,"URL is invalid.\nReturn value: %d\n Error code: %d\n",ret,code);
show_msg(msg_durl);
goto end_download_process;
}

Can you think of a better idea?

Sent with Tapatalk
 
Last edited:
bguerville said:
I would like to check whether the passed url exists before launching the download itself.
What do you think of this type of check?

CellHttpClientId client = 0; CellHttpTransId trans = 0; CellHttpUri uri;
uri.scheme="<httpmode>"; uri.hostname="<hostname>"; uri.username="";
uri.password="";
uri.path="<urlpath>";
uri.port=<port>;
int code = 0;
int ret;
ret = cellHttpCreateClient(&client);
if(ret<0)
{
sprintf(msg_durl,"An error occurred when checking the Url. Could not create client. \nReturn value: %d\n",ret);
show_msg(msg_durl);
goto end_download_process;
}
ret = cellHttpCreateTransaction(&trans, client, CELL_HTTP_METHOD_GET, &uri);
if(ret<0)
{
sprintf(msg_durl,"An error occurred when checking the Url. Could not create transaction. \nReturn value: %d\n",ret);
show_msg(msg_durl);
goto end_download_process;
}
ret = cellHttpSendRequest(trans, NULL, 0, NULL);
if(ret<0)
{
sprintf(msg_durl,"An error occurred when checking the Url. Could not send request to client. \nReturn value: %d\n",ret);
show_msg(msg_durl);
goto end_download_process;
}
ret = cellHttpResponseGetStatusCode(trans, &code);
if(ret<0)
{
sprintf(msg_durl,"URL is invalid.\nReturn value: %d\n Error code: %d\n",ret,code);
show_msg(msg_durl);
goto end_download_process;
}

Can you think of a better idea?

Sent with Tapatalk
Click to expand...

I haven't tested your code, but it looks good. I would check also that the response code is not 4xx or 5xx (client or server error).

IRISMAN uses a similar code in updates.c, but it only checks for errors 404 and 403 (not found & forbidden).
I suggest you that view the functions download_file or download_update (which also supports SSL) to get some ideas.

For a complete list of http status codes check this link

BTW the error messages can be smaller ;)
 
Last edited:
aldostools said:
I haven't tested your code, but it looks good. I would check also that the response code is not 4xx or 5xx (client or server error).

IRISMAN uses a similar code in updates.c, but it only checks for errors 404 and 403 (not found & forbidden).
I suggest you that view the functions download_file or download_update (which also supports SSL) to get some ideas.

For a complete list of http status codes check this link

BTW the error messages can be smaller ;)
Click to expand...

Great. SSL support would be nicer.

Yes, the error messages should be changed. They were written quickly & aimed at myself rather than users in general anyway. [emoji1]

Update:
I just updated the pkg file 5mn ago as I found a small typo error in the source.

Everyone please download the pkg again to make sure you have the latest version....

Mirror 1

Edit:
Deleted link. Outdated.
 
Last edited:
I updated the mirror with a fixed pkg.
I realise only now that I uploaded the wrong test build last night.., yeah I know it was careless & sloppy but for my defence it was 4am! :tranquillity:
As it turns out the download command was buggy in that build.

Sorry guys..
Feel free to download this fixed version.

For information, it's not possible to use /dev_hdd0 as a download path, if you do the system download plugin defaults to /dev_hdd0/tmp/downloader.
Except the root of the internal hdd, other paths I tried on /dev_hdd0 can be used without problems.

The URL check code (with SSL support) is also ready. It should be included in the next build.

There is one more tiny change for me to make for the syntax to be identical to what Aldo advised but for now it is as follows:

Download Command
hxxp://<ps3-ip-address>/download.ps3?url=<file-url>&to=<storage-path>
the parameter "&to" is optional, if unused the default directory /dev_hdd0/packages will be chosen and in case of a problem it falls back to /dev_hdd0/tmp/downloader as a last resort.

Install Command
hxxp://<ps3-ip-address>/install.ps3<pkg-file-path>


Download Link: Mirror 1
 
Last edited:
Here is another update.
You can use the link from post #15.

This time the syntax overhaul is complete.
More bug fixes as well.
Next release will include URL check before download.


Final syntax of /download.ps3:
hxxp://<ps3-ip-address>/download.ps3?to=<storage-path>&url=<file-url> using the optional "to" parameter.
hxxp://<ps3-ip-address>/download.ps3?url=<file-url> which defaults to /dev_hdd0/packages for storage path.
The /install.ps3 command is unchanged:
hxxp://<ps3-ip-address>/install.ps3<pkg-file-path>

Enjoy...

@aldostools
The source is most likely coming with the next release! :cheerful:
 
bguerville said:
Here is another update.
Click to expand...

Bug report...
wMM_pkg_bug.jpg


BTW it would be nice if you add the changes in games_xml.h from 1.43.30 (the other changes are space formatting and update of version).

Also you could add this code to file_manager.h (add it before the 2nd #ifdef COPY_PS3)
Code: 
#ifdef INSTALL_PKG
else if( !strcmp(ext, ".pkg") || !strcmp(ext, ".PKG") )
    sprintf(fsize, "<a href=\"/install.ps3%s\">%'llu %s</a>", templn, sz, sf);
#endif
This code would let install any pkg remotely just clicking on the size column of the pkg listed on the web version of the file manager.

A shorter form of the message "Installation of /dev_hdd0/packages/movian.pkg is under way" could be: "Installing /dev_hdd0/packages/movian.pkg"
 
Last edited:
aldostools said:
Bug report...
wMM_pkg_bug.jpg


BTW it would be nice if you add the changes in games_xml.h from 1.43.30 (the other changes are space formatting and update of version).

Also you could add this code to file_manager.h (add it before the 2nd #ifdef COPY_PS3)
Code: 
#ifdef INSTALL_PKG
else if( !strcmp(ext, ".pkg") || !strcmp(ext, ".PKG") )
    sprintf(fsize, "<a href=\"/install.ps3%s\">%'llu %s</a>", templn, sz, sf);
#endif
This code would let install any pkg remotely just clicking on the size column of the pkg listed on the web version of the file manager.

A shorter form of the message "Installation of /dev_hdd0/packages/movian.pkg is under way" could be: "Installing /dev_hdd0/packages/movian.pkg"
Click to expand...
No problem Aldo. Thanks for the bug report & cool suggestions.
I will look into it straight away.

I was wondering whether you would have something to suggest regarding a feature I haven't been able to implement so far.
Obviously I tried to find a way for a pkg to install directly after download however I have no idea how to catch the download plugin unload.
Am pretty sure one of the download_plugin.sprx functions allows to do this but the documentation mysis left us for this plugin is largely incomplete & I have not been able to identify which one. I tried to select best candidates according to passed parameters but without luck.
Any suggestions?

FYI I am currently uploading the source to my github account, it will make things easier for us overall, especially if I keep working on other new commands.



Sent with Tapatalk
 
Update


I updated the project to 1.43.30.
Fixed the the pkg extension check issue in the install command, it was a simple typo using strchr instead of strrchr therefore returning the first occurence of the '.' character instead of the last one.....
The rest seems.ok.

You can download the pkg from this mirror.

Mirror

@aldostools
The change in file_manager.h works great but I had to add to modify the 2nd COPY_PS3 if statement with
#ifndef PKG_HANDLER
|| !strcmp(ext, ".pkg")
#endif
otherwise the copy link took precedence.
I reused strcmp as that was already in the if statement but this extension check could actually use strcasecmp instead, wouldn't it be more appropriate?

Also the cosmetic bug "</" at the bottom of the webpage occurs in http_response but I don't really understand why. I didn't change this function at all. When I check the different values used by the function, I see no discrepancies.
So I dumped the header variable to a log file just before it is sent with the ssend command to see what is being sent.
I get this:

HTTP/1.1 200 OK
X-PS3-Info: [/install.ps3/dev_hdd0/packages/webMAN_MOD_1.43.26_Updater.pkg]
Content-Type: text/html;charset=UTF-8
Cache-Control: no-cache
Content-Length: 242

<body bgcolor="#101010" text="#c0c0c0"><font face="Courier New">webMAN MOD 1.43.30 MOD PKG Edition<hr><h2>Installing /dev_hdd0/packages/webMAN_MOD_1.43.26_Updater.pkg<hr> <input type="button" value=" &#9664; " onclick="location.href='/';"></font></body></html>

Now if I open this log with a browser there is no bug....

So I tried other web commands like http://192.168.0.15/remap.ps3/dev_usb000&to=/dev_hdd0/packages & I get the same cosmetic bug, the header variable being sent to the browser is:

<body bgcolor="#101010" text="#c0c0c0"><font face="Courier New">webMAN MOD 1.43.30 MOD PKG Edition<hr><h2>Remap: <a href="/dev_usb000">/dev_usb000</a><br>To: <a href="/dev_hdd0/packages">/dev_hdd0/packages</a><p>Unmap: <a href="/unmap.ps3/dev_usb000">/dev_usb000</a><hr> <input type="button" value=" &#9664; " onclick="location.href='/';"></font></body></html>

If I open this log with a browser, again, there is no bug...

WTH...???


It's late, time to put my head down.
I will look at it again tomorrow.
 
Last edited:
@aldostools
I just realised that you posted a link to 1.43.31.
I looked quickly at what you did, it looks perfect to me!
No worries about changing the code, of course, on the contrary I was going to ask you to change it as you saw fit anyway.
Nice one....:adoration:

Gotta sleep for a couple of hours before work.
Will look at your release in more details later...
 
You must log in or register to reply here.

Similar threads

Back
Top