PS3 Bypassing the 16KB limitation on XMB XMLs' module-action

Discussion in 'Homebrew Development' started by DADi590, Sep 18, 2019.

  1. 170
    105
    97
    DADi590

    DADi590 Formely known as fkd

    Joined:
    Jun 9, 2018
    Messages:
    170
    Likes Received:
    105
    Trophy Points:
    97
    Occupation:
    ERROR
    Location:
    ERROR
    Home Page:
    Hi everyone. I don't know if this will still be interesting to know after HEN's release, but as I found this to be EXTREMELY useful with HAN Toolbox, I thought it would be good to post here even if outdated. Maybe it's useful for some function of HEN, like HEN Enabler, for example. Or any other project not related to HEN.

    I've been messing with HAN Toolbox, and I came across lmn7's offline scripts. As they are only about 3 or 4, and I wanted put to more there, I started trying to understand how to do it too (and try that it wouldn't have to be me filtering the whole code, which was out of question, as I don't have that much time, only a few days left). After some time of trying by myself, I remembered DeViL303 said that the module-action part takes at most 16KB of JavaScript code, and the PS3 won't accept more than that. So I searched for hours and hours for compressors of JavaScript and only 2 were decent, from what I found, at least. Even after compressing, it wouldn't work, so I asked xps3riments about why it wouldn't work, and after his explanations and some research of mine, I understood how to do it properly (thanks to both, btw).

    So now, I'm putting everything offline in HAN Toolbox, including webpages! I've just put the Cold Boot Installer page offline with only 6.903 characters!! Others though, have almost 16.000, but it's almost, so no problem. By now, nothing passed the 16.000 with this method. With methods before (I made a program to help me decide what to keep and what to delete to filter the code - still, this new way is SO MUCH better even though that way was kinda working), there were about 17.000 characters and it still worked, but that must be the really maximum limit.

    1st way - I didn't test this on the PS3, only in my PC. But with the exploit code (it doesn't seem to be available in the current framework), opening a file (which would contain just one line of code), and passing the returned text to eval(). No more 16KB limitation, I think (I didn't test this, as I said). I tested it on my PC with both loading from an Internet script and from a local script. Worked in both ways (the first one is not that useful in this case haha). Or, if webMAN is installed (CFW/HEN only, of course), as xps3riments proposed, load the file from webMAN's FTP server. On HAN the last option is out of question, while the first one might work, but the framework doesn't have a read_file function (at least from what I understood of it - I don't know how to work with it, yet), so I didn't do any of this. I did the other way around.

    2nd way - What I used is actually EXTREMELY simple haha. After the hours of search for good tools to compress, I came across one of Google ("Google is your friend" hahaha). It's called Closure Compiler - https://closure-compiler.appspot.com/home. It has a Simple mode and an Advanced mode (that's explained there). Using the Advanced mode............. The Cold Boot Installer page code was about 90.000 characters long. It went down to 6.903 in a matter of seconds... It searches for dead code, aside from renaming variables to a smaller size (and other things). Aside from this, putting the code in just one line seems to put it even faster. So just use any tool that removes line breaks and paragraphs breaks. I used this one - https://www.textfixer.com/tools/remove-line-breaks.php. After this, don't forget to escape 2 XML special characters as octal or hexadecimal characters (for some reason, not the 5 of them are necessary to escape): < and > (put \047 and not \47, as if the idea is to unescape \12 and there is \120453294 in a text, for example, it will unescape \120 and not \12 - max 4 characters in octal and hexadecimal characters, including the \). And \ too, in case there are unicode things there, like \uXXX or any other thing that needs to have a \ there. After this, put what's left inside --> eval('here'). And it will work perfectly. After doing this, at least with lmn7's Offline Soft Rebooter, the one that was made after this, was INCREDIBLY faster. The original was more than a minute doing its thing (at least in my PS3), and the new one takes about 5 seconds... Also, if for some reason someone needs to compress even more the code, this tool could be used, for example (which was what was used in the Offline File Copier code to make it smaller): http://dean.edwards.name/packer/. The Base62 encoding REALLY compresses the text. But Google's tool is better. And probably this puts it even slower by being encoding, since Google's tool doesn't encode anything (I'm not an expert, I'm a begginer, so this could be untrue).

    Hope someone finds this useful! So big... Damn. Sorry hahaha.
     
    Last edited: Sep 18, 2019
    sb00, aldostools, sandungas and 3 others like this.
  2. 189
    74
    82
    Louis Garry

    Louis Garry Member

    Joined:
    Jul 11, 2018
    Messages:
    189
    Likes Received:
    74
    Trophy Points:
    82
    Gender:
    Male
    It seems very helpful. Offline HEN Enabler (2.3.1) from 4.84 can be ported to 4.85 by changing the script 4.84 to 4.85. I want 1 script HEN Enabler that can be used for 4.84 and 4.85, can you help me? Thanks.
     
    DADi590 likes this.
  3. 170
    105
    97
    DADi590

    DADi590 Formely known as fkd

    Joined:
    Jun 9, 2018
    Messages:
    170
    Likes Received:
    105
    Trophy Points:
    97
    Occupation:
    ERROR
    Location:
    ERROR
    Home Page:
    HEN Enabler can be from 4.82 to 4.85. On 4.85, if it uses the same memory addresses as HAN does, it's just change 4.84 in the scripts to 4.85 and it will work. Though, that will remove compatibility with 4.84. Something like this should be put there:
     

    Attached Files:

  4. 189
    74
    82
    Louis Garry

    Louis Garry Member

    Joined:
    Jul 11, 2018
    Messages:
    189
    Likes Received:
    74
    Trophy Points:
    82
    Gender:
    Male
    I know, that I want to support only 4.84 and 4.85.
     
  5. 604
    1,009
    222
    lmn7

    lmn7 Developer

    Joined:
    Oct 31, 2017
    Messages:
    604
    Likes Received:
    1,009
    Trophy Points:
    222
    The speed of the offline scripts doesn't really depend on their size, well I say that but it's kind of more complicated. Basically, smaller != faster. The webkit exploit used is just extremely unreliable. Try changing a few bytes in each of the tools and then run them again, you will notice that at least one of them will be much slower.

    However, all of my original scripts were tested on my PS3 and work fine, taking about 5 seconds, even the soft rebooter. Some I've been able to get to 2-3 seconds. The problem is that there are so many variables that can affect initialization time. For example, I've confirmed things like the system language and font settings do affect it, but it's likely there are many more factors.

    Even though there is a 16kb limit, I've been able to convert every script I want for offline use, so this isn't really as much of an issue as it may seem. There is not a huge difference between the scripts btw, it only takes changing a few lines to turn it into a completely different tool. Once you have the base, you can pretty much convert any tool with minimal effort. As for larger scripts that can't be converted, I've had some success loading the stackframe into memory from a file on the HDD (like HEN), I imagine it'd be possible to load JS this way but I haven't tried.

    That being said, it's cool to see someone else looking into this, and unexpected too. Nice work.
    You can just remove the firmware version check from the JS, since the offsets for both 4.84 and 4.85 are the same it will work on both.
     
    Algol, DADi590 and sandungas like this.
  6. 189
    74
    82
    Louis Garry

    Louis Garry Member

    Joined:
    Jul 11, 2018
    Messages:
    189
    Likes Received:
    74
    Trophy Points:
    82
    Gender:
    Male
    Is it only removing if(fwv =="4.84") ?
     
  7. 604
    1,009
    222
    lmn7

    lmn7 Developer

    Joined:
    Oct 31, 2017
    Messages:
    604
    Likes Received:
    1,009
    Trophy Points:
    222
    yeah, you need to remove the brackets too.
     
    Louis Garry likes this.
  8. 170
    105
    97
    DADi590

    DADi590 Formely known as fkd

    Joined:
    Jun 9, 2018
    Messages:
    170
    Likes Received:
    105
    Trophy Points:
    97
    Occupation:
    ERROR
    Location:
    ERROR
    Home Page:
    Hi! Thanks for the reply with all the information! I'd have talked with you already, but I can't message anyone I haven't already messaged, because after I asked STL to change my username to the one it was supposed to be, seems like the forum thinks I'm new here, and I don't have som permissions (I have to wait to post the stuff too, nor can I edit the posts once I publish them). So I talked with xps3riments, which I thought might new some answers, and he actually did, so it was great.
    Oh, I thought the smaller it would be, the better for the PS3. Interesting. The line and paragraphs breaks, when removed, seemed to have put the Soft Rebooter faster, at least. Takes less time, for some reason (but in this, it would have to be in one line anyways - I didn't try code in multiple lines in the XML, but it's faster this way - supposedly...). I tried too not to have to encode any of them (the file copier code, after putting it back to normal - I can't find the original script, I don't know where it is -, I put it in Closure Compiler and it got smaller and not encoded. Though, I'm not sure what the problem was with what you said: "The file copier script is very large and had to be obfuscated then Base64 encoded, for this reason the success rate is very random. I would imagine it would work better with this method". The success rate is affected when the code is encoded?

    Weird. DIdn't know it had so weird variables. xps3riments told me about the position of the characters, but I didn't know about those 2. So weird thing. I thought it would be more stable. At least testing in my PS3 only, the new script of the Soft Rebooter went really better than yours, but I had (have?) no idea why. Now seems like yours was already fast! So now I'm confused which ones to keep... I'll have to run some tests in my 2 PS3s and see what I get. Must be in less than some days though, or I won't release it for months.

    I noticed that some files are named _31, others _301, and others _302. So that means they're different, and some are outdated and others updated. As I don't know if all the scripts have the same things (for exemple, there are scritps with ip_something_adrr and others don't have those), I decided to copy the code of all of them, just in case something would be different, and just edit the part where it checks for the memory addresses, since I have a smaller way of checking it (the auto tools way).

    No one's updating the toolbox, so I thought I could give it a try. After that, I know the site will be down someday. For the toolbox to still work 100%, either people have a copy of the site (which I'll try to put in a proxy app with old features included), or the pages are already in the toolbox and everything else. So I tried it (as long as it wasn't like you did - manually haha).
     
    Last edited: Sep 19, 2019
    Algol likes this.
  9. 189
    74
    82
    Louis Garry

    Louis Garry Member

    Joined:
    Jul 11, 2018
    Messages:
    189
    Likes Received:
    74
    Trophy Points:
    82
    Gender:
    Male
    Are there any ready ones? I do not exactly understand.
    Thanks.
     
  10. 1,727
    1,852
    297
    pink1

    pink1 Moderator Developer

    Joined:
    Feb 25, 2015
    Messages:
    1,727
    Likes Received:
    1,852
    Trophy Points:
    297
    Gender:
    Male
    Nice work bud! I always get excited when I see someone picking up on a project and trying to take it to the next level. This is how a good scene is people working together doing what they can and giving back to the community.

    Keep up the great work everyone and I cannot wait to see who comes up with what next. :tickled pink:
     
  11. 170
    105
    97
    DADi590

    DADi590 Formely known as fkd

    Joined:
    Jun 9, 2018
    Messages:
    170
    Likes Received:
    105
    Trophy Points:
    97
    Occupation:
    ERROR
    Location:
    ERROR
    Home Page:
    Make it simpler then. To make it compatible with ONLY 4.84 and 4.85, replace if(fwv=="4.84) EVERYWHERE (all files) with this:

    if (fwv== "4.84" || fwv== "4.85")

    Just that.
     
    Algol and Louis Garry like this.
  12. 604
    1,009
    222
    lmn7

    lmn7 Developer

    Joined:
    Oct 31, 2017
    Messages:
    604
    Likes Received:
    1,009
    Trophy Points:
    222
    All of the offline scripts I've shared don't have any newlines, I don't even think it's possible due to how the XML file is parsed.

    That's what I thought at the time, now that I understand it a little better I think I probably just got unlucky. There's every chance that if I had adjusted some lines of code, removed some stuff, maybe even changed some of the exploit search parameters then it would have worked just fine. It's all random, in my experience even changing one byte can drastically slow the init speed.

    The exploit searches a small section of memory for variables. If a tool works fine with the system language set to English, it probably wouldn't work the same way if it was set to something like Spanish because those texts are loaded into memory and it changes the position of everything else. Same applies to fonts or changing the script itself, anything can displace the search area really.

    Yeah, I don't use those though. I only take stuff out of them if it's required for certain tools. I've been using the same base script since 4.82 OFW.
     
    Algol, DADi590 and aldostools like this.
  13. 170
    105
    97
    DADi590

    DADi590 Formely known as fkd

    Joined:
    Jun 9, 2018
    Messages:
    170
    Likes Received:
    105
    Trophy Points:
    97
    Occupation:
    ERROR
    Location:
    ERROR
    Home Page:
    No, I meant that putting in just one line puts it faster, but yes, all of yours are in one line only. Just thought I'd say they seem faster in one line than in various lines (I'm testing them in an HTML in my server and entering in them on the PS3 and first I test in multiple lines, and then in one to check if it's still all working).

    I'll have to test this in various languages and the 3 fonts then... Some mixes only though. I don't have time nor I'd want to test all the combinations haha. But I'll try some and see what happens. My System Language is now set to Portuguese with the original font. I'll see some more combinations with more popular languages on this thing. Not many because I don't have much time, not even to know if the conversion to offline scripts is 100% working. Won't brick anything, since I didn't mess with the exploit code, but the external might have bugs or something like that. I'll leave online options on this anyways (if I get to understand how to add a folder on this...).

    Thanks for all the informations! Really useful!
     
    Louis Garry, DeViL303 and lmn7 like this.
  14. 3,260
    5,352
    522
    aldostools

    aldostools Developer Developer

    Joined:
    Oct 30, 2014
    Messages:
    3,260
    Likes Received:
    5,352
    Trophy Points:
    522
    @DADi590 nice post :)

    Just a thought.... only HEN offline enabler is limited to 16KB. Once HEN is enabled, if you have a web server plugin running (sMAN, webMAN or webMAN MOD) the other XMB scripts can be hosted locally as HTML/CSS/JS on /dev_hdd0 without the 16KB limitation and without need to compress the script or encode the characters (in a human readable format).
     
  15. 170
    105
    97
    DADi590

    DADi590 Formely known as fkd

    Joined:
    Jun 9, 2018
    Messages:
    170
    Likes Received:
    105
    Trophy Points:
    97
    Occupation:
    ERROR
    Location:
    ERROR
    Home Page:
    @lmn7 Btw, would you know why on the site, it's all normal? The things always work there. Or at least out of the XMB. The code I put on the XML is taking infinity to get to 100% of explotation and then says it failed. But if I copy the exact same code to the HTML file and run it on the PS3, it works instantly. It doesn't even say Exploit Initialization, it just goes to its thing. Would you know why? This shouldn't be like this... They should be both equal if it's the same code! (I think?)
     
  16. 604
    1,009
    222
    lmn7

    lmn7 Developer

    Joined:
    Oct 31, 2017
    Messages:
    604
    Likes Received:
    1,009
    Trophy Points:
    222
    Online/offline scripts initialize differently. Offline is most reliable for testing because afaik the timing is always the same. Using online scripts introduces random timing/delay issues, even when using a local web server. Also the fact you need to minify the script obviously changes it, and any kind of change can completely screw up the init time.
     
    Louis Garry and DADi590 like this.
  17. 170
    105
    97
    DADi590

    DADi590 Formely known as fkd

    Joined:
    Jun 9, 2018
    Messages:
    170
    Likes Received:
    105
    Trophy Points:
    97
    Occupation:
    ERROR
    Location:
    ERROR
    Home Page:
    What if the script is loaded from a local file, and loaded always in the same way, and the code doesn't need to be minified, just in one line? Do you know if it would work well if the loading code was correctly made? (maybe the problem would be the code to read the file, but after that maybe it would work 100% no?)
     
    Last edited: Sep 19, 2019
  18. 170
    105
    97
    DADi590

    DADi590 Formely known as fkd

    Joined:
    Jun 9, 2018
    Messages:
    170
    Likes Received:
    105
    Trophy Points:
    97
    Occupation:
    ERROR
    Location:
    ERROR
    Home Page:
    Not wanting to unsay what you said, because you know more (much more...) than me about this, but I've been trying to put the Flash XML Replacer working, because that one wouldn't initialize at all on offline.
    As you said changing things on it might put it better or worse, and I didn't know what I could do, I started deleting things from the script (mainly strings, as I've done with the Flash RCO Replacer in which I had to remove almost all text for it to work). After some deletions, still nothing happened and it would go all the way to 100% without working. One of the deletions, put it working at about 95%, so sometimes it wouldn't work. As I saw that happened, I thought I could try deleting more things on it to see what would happen. After another deletion, it worked at about 75%. Another deletion (this time not HTML strings, but the code to print the Environment Info) put it working always at 35%!
    These things I said would only be like that if I restarted the browser (and I'm not sure about clearing the cache and cookies, as I've just only restarted the browser and it went to 35% and worked). These last 3 percentages were done in a row. After 3 consecutive deletions in the code. But to note that this is with encoded code to Base62. I don't know what would happen with not encoded code (to Base62, at least), like the auto functions are. And even by this, it could be just one case. I've been trying with this one to see if I could put it working, because it wouldn't work at all. Ah btw, I didn't change neither the font nor the language. The testing had the same environment (except internal system things, but even after restarting, this kept on 35% and before it would go to 100% and not work). I have no idea if this means anything, but at least with the Flash XML Replacer worked. The encoding was to Base62 with Dean's packer, and after having been minified by Google's Closure Compiler. I'll try this with other scripts to see if the same happens in these same conditions. Again, not to unsay what you said, I just thought I'd let you know of at least this particular case.

    UPDATE: I've just reinstalled one of the other versions of the replacer to see if it would go to a higher percentage, and it is. 95% haha (first try, I didn't restart the browser nor cleared anything). It's the 1 successful try (I didn't know which one I was installing back). And after this, I installed now the newest version back, and it's back on the 35% (second try after only restarting the browser, the first one went to 100% and didn't work - no idea why it wasn't on the first one, maybe cached things which were deleted when the cache was full?). I'd love if this meant stability (speed to?) in smaller code (at least encoded to Base62 and by having used his packer, which even with Base62 not selected - and I have never short variables enabled because Google already did it -, packs the code a bit, but I'm not sure that will do anything or not). But I've not run as many tests as you did, so I'm not sure what this means. Thought I'd let you know about this. Maybe the size counts with encoded code? Hope the report is complete enough haha. Tried to make it complete to be analyzed.
     
    Louis Garry likes this.
  19. 170
    105
    97
    DADi590

    DADi590 Formely known as fkd

    Joined:
    Jun 9, 2018
    Messages:
    170
    Likes Received:
    105
    Trophy Points:
    97
    Occupation:
    ERROR
    Location:
    ERROR
    Home Page:
    EDIT: DEX version of that works at 40% always (the offline versions of the sites have CEX and DEX versions, and I've testing the CEX one). I didn't test that one more times, I have the PS3 in CEX and being restarting it all the time is boring.
     
  20. 189
    74
    82
    Louis Garry

    Louis Garry Member

    Joined:
    Jul 11, 2018
    Messages:
    189
    Likes Received:
    74
    Trophy Points:
    82
    Gender:
    Male
    It works, but the time for Enable HEN is very long
     
Tags:

Share This Page