PyPS3tools : a suite of python tools for PS3 flash memory dump files

Discussion in 'Downgrading' started by littlebalup, Jun 14, 2015.

  1. 469
    99
    38
    littlebalup

    littlebalup Developer

    Joined:
    Oct 16, 2014
    Messages:
    469
    Likes Received:
    99
    Trophy Points:
    38
    Location:
    43°36'16.0"N 1°26'36.1"E
    I continue my crusade for unix based systems users. So I decided to create new dump tools that can be run natively on a maximum of systems (including windows).
    As the wonderfull Judges's NORway & NANDway scripts are written in python 2, I naturally chosen the same langage to complete them by creating a suite of tools including a "patcher", a "checker" and a "rebuilder" that can be run into the same environment.
    With all of those python scripts, we are able to dump, check, patch, rebuild and flash on all python 2 compatible systems. And a batch/bash script can easily be written to automate all of this (for NOR at least. An "interleaver" still missing for NAND).

    Python scripts, detailed readme and changelog are available to my git : https://github.com/littlebalup/PyPS3tools

    "PyPS3checker" overview :

    [​IMG]


    "PyPS3patcher" overview :

    [​IMG]


    "PyPS3rebuilder" overview :

    [​IMG]


    Enjoy your flash ;)
     
    Last edited: Nov 3, 2015
    T.A.U likes this.
  2. 469
    99
    38
    littlebalup

    littlebalup Developer

    Joined:
    Oct 16, 2014
    Messages:
    469
    Likes Received:
    99
    Trophy Points:
    38
    Location:
    43°36'16.0"N 1°26'36.1"E
    A "short" demonstration made under windows :
    [video=youtube;3gZPbvNKwTg]https://www.youtube.com/watch?v=3gZPbvNKwTg&feature=youtu.be[/video]

    It's a raw video. No time for video editing :embarrassed:
     
  3. 469
    99
    38
    littlebalup

    littlebalup Developer

    Joined:
    Oct 16, 2014
    Messages:
    469
    Likes Received:
    99
    Trophy Points:
    38
    Location:
    43°36'16.0"N 1°26'36.1"E
    Work in progress :

    [​IMG]
     
  4. 114
    58
    38
    playerkp420

    playerkp420 Developer

    Joined:
    Feb 24, 2015
    Messages:
    114
    Likes Received:
    58
    Trophy Points:
    38
    Very nice! I love it.

    I could of used this a few times. When fixing some PS3s, that users were able to read n write with the clip not seated correctly. Frankenstein'ing dumps together by copy n paste write in hex editor is time consuming.



    Sent from my SAMSUNG-SM-G870A using Tapatalk
     
  5. 469
    99
    38
    littlebalup

    littlebalup Developer

    Joined:
    Oct 16, 2014
    Messages:
    469
    Likes Received:
    99
    Trophy Points:
    38
    Location:
    43°36'16.0"N 1°26'36.1"E
    A beta version of the rebuilder is available for test purpose only !

    [​IMG]

    here : https://www.dropbox.com/s/j7ts4xvi147ym46/rebuilder.py?dl=1

    Note:
    Supports NOR and NAND dumps.

    For NOR you can restore a byte reversed dump with a non byte reversed dump as donor. Or vice versa. The script take care to swap bytes if necessary. The generated file will be the same type as the original file.
     
  6. 469
    99
    38
    littlebalup

    littlebalup Developer

    Joined:
    Oct 16, 2014
    Messages:
    469
    Likes Received:
    99
    Trophy Points:
    38
    Location:
    43°36'16.0"N 1°26'36.1"E
  7. 536
    49
    38
    Sdw100

    Sdw100 Moderator

    Joined:
    Nov 17, 2014
    Messages:
    536
    Likes Received:
    49
    Trophy Points:
    38
    Location:
    ENGLAND
    Do u need python to use cant you make it basic windown exe file, none the leas very good work [MENTION=48]littlebalup[/MENTION]
     
  8. 469
    99
    38
    littlebalup

    littlebalup Developer

    Joined:
    Oct 16, 2014
    Messages:
    469
    Likes Received:
    99
    Trophy Points:
    38
    Location:
    43°36'16.0"N 1°26'36.1"E
    Yes, you need python 2.
    And no, I'll not make an exe. Goal is to have multi-OS simple scripts.
    But it is not a big deal to install python on Windows. If you use a Teensy, you should have it.
     
  9. 536
    49
    38
    Sdw100

    Sdw100 Moderator

    Joined:
    Nov 17, 2014
    Messages:
    536
    Likes Received:
    49
    Trophy Points:
    38
    Location:
    ENGLAND
    I have it allredy but was thinking if u could doit, no big deal
     
  10. 666
    84
    38
    No0bZiLLa

    No0bZiLLa Moderator Developer

    Joined:
    Oct 1, 2014
    Messages:
    666
    Likes Received:
    84
    Trophy Points:
    38
    i got a question, well really more of a statement than question, ok its not a question at all lol. anyways...

    if a user dumps their flash package, and it has bad data in it, why would anyone want to patch it to make it seem "good" or "valid" on a pc, in the end the console is what validates it anyway, or your console wont boot.

    if you dont have good clip seating or bad solder connections, you still will be messing up. because it takes more power to actually flash than it does to read, so if your not reading it properly, then your def not writing it properly 100%. thats just my two cents. instead of making a tool to interleave two dumps, maybe they should just get better clip seating/solder connections.

    this could pose to be troubles for mismatch coreos, and in the end user will need to solder better or adjust clip seating anyway. i wouldnt give the option to interleave good and bad dumps, i would just tell user to make better connections.

    if you want me to explain further please ask, but i am sure you know what i am talking about. if not, i do not mind sharing my views on it, just ask.
     
    Last edited: Jul 11, 2015
  11. 469
    99
    38
    littlebalup

    littlebalup Developer

    Joined:
    Oct 16, 2014
    Messages:
    469
    Likes Received:
    99
    Trophy Points:
    38
    Location:
    43°36'16.0"N 1°26'36.1"E
    I agree with the fact that, first, all must be done to have a good dump. Of course. Maybe I didn't warn it clearly, you are right.
    The rebuilder is a "last chance" tool for those who writen their flash without to take care they have a good dump. If by some miracles the perconsole datas are not corrupted, they can transfer them to a know valid dump.
    It is not "to make it seem "good" or "valid" on a pc", it is to have well formed file tables, headers, vtrm, etc...
     
  12. 666
    84
    38
    No0bZiLLa

    No0bZiLLa Moderator Developer

    Joined:
    Oct 1, 2014
    Messages:
    666
    Likes Received:
    84
    Trophy Points:
    38
    the point i am trying to make is, say for instance, i dump my flash, it is on 4.70. each file in coreos has built in hash check (which is why i dunno why people use md5 validation for this section anyway) so if aim_spu_module.self has a certain hash, then unless you are patching the same version firmware, then you will be corrupting aim_spu_module.self because the hash of the file wont match because it has parts in it from different firmware, this goes for any other file in coreos or anywhere else that has hash calculations.

    another way for me to specifically say is, if you dump unsuccessfully, then your not going to be able to overwrite the areas that you could not dump. therefore, if you overwrite it with a different firmware revision, then each file in the dump will be corrupted because the whole file will not have been overwritten. only parts of each file. its hard to explain lol.

    it would work if the firmware revision was exactly the same tho, maybe a check could be implemented to make sure the coreos of donor dump is same as original bad dump. but either way, in order to write back a patched dump, you will need good connectivity thru the clip or soldering or else each file in the dump will be diluted with a different firmwares encrypted data. and you are rightabout bad per console datas = paper weight
     
    Last edited: Jul 11, 2015
  13. 114
    58
    38
    playerkp420

    playerkp420 Developer

    Joined:
    Feb 24, 2015
    Messages:
    114
    Likes Received:
    58
    Trophy Points:
    38
    Of course it is useless if the user soldered wrong, and read and wrote. But if clip was not seated correctly, it can be fixed. I know from experience, because have done it a few times.

    First seat clip correctly to get the missing data. Most times seating the clip correctly alone will not solve the problem. The new dump will look more damaged than before. You will have to search offsets and sizes of per console data for your model. If lucky enough to find the important data intact from both bad dumps, and put them where they should go in one dump (cvtrm is not important, cause will only cause rsod).

    After that, u will not be able to get all the generic info. This is where an automated program would come in handy. Find a donor dump from same sku and revision (coreos version from donor doesn't matter, because we will patch for FSM Method downgrade). Without automated program, u would have to use hex editor and wiki Flash page as reference for copy n paste write.

    Patch new dump u Frankenstein'd with 3.55 patches. FSM install will format all regions and encrypt all files in dump with your keys. Of course will only work, if your important per console data was put back correctly.

    Now the PS3 is fixed and only needs exit FSM, setup settings, and dehash.

    It works.

    Sent from my SAMSUNG-SM-G870A using Tapatalk
     
  14. 666
    84
    38
    No0bZiLLa

    No0bZiLLa Moderator Developer

    Joined:
    Oct 1, 2014
    Messages:
    666
    Likes Received:
    84
    Trophy Points:
    38
    yeah i have done it a few times as well. but the main point i am trying to make is, the user will have to reseat the clip and/or resolder. and they should dump again instead of flashing something their not even sure if is valid. just because validation tools say it is valid doesnt mean it is brick proof. also i posted a youtube video of fixing a dump a year or two ago, i know it is possible so dont get me wrong. but the first thing i have the users do is reclip/resolder and redump a few times. then compare. alot of times everytime they reclipped and redumped the data was different.

    im just saying, it is better to reclip/resolder and redump to see if that fixes it. because if you cannot dump the flash correctly then you damn sure cant write it correctly
     
  15. 476
    54
    38
    psykosis

    psykosis Developer Developer

    Joined:
    Dec 7, 2014
    Messages:
    476
    Likes Received:
    54
    Trophy Points:
    38
    I think noobs point is like this scenario:

    You shouldn't EVER eat preparation H, but in case you do, there's poison control......

    It's kind of one of those scenarios that shouldn't ever happen, but there's a tool in case someone really went past the limit of common sense and basic intelligence :)
     
  16. 666
    84
    38
    No0bZiLLa

    No0bZiLLa Moderator Developer

    Joined:
    Oct 1, 2014
    Messages:
    666
    Likes Received:
    84
    Trophy Points:
    38
    well not really, like i said, if you cant read correctly, then you cannot write correctly. so patching a dump that is not valid to be valid wont lead to a valid write without reclipping/resoldering anyway. like i said i have recovered consoles like this as well, but i had individuals reclip and get the best dumps they could. say you have 4.20 filled ros, it will have files in flash for 4.20, then you patch for 4.75... well if you cannot write the flash 100%, then you are writing jumbled data into the flash, which could bring you back to exactly where you were to begin with.

    however, usually if ros is corrupt, then it will request you to reinstall firmware. i wouldnt repair a bad dump to be a good dump. i would try to get a good dump or better data by reclipping/resoldering. and if your pc data is bad then this repair tool wouldnt help anyway.

    i also do not care what anyone elses point of view is on this either. common sense will tell you that if you cant get a good read, then you will never get a good write without reseating clip or resoldering.

    so if you were not able to dump the file tables/headers/vtrm correctly, you expect to write file tables/headers/vtrm back 100% without reclipping/resoldering? that would be amazing because i dont see it happening if you didnt get a good read/dump in the first place. but if you reseat/resolder, you could just redump better data right?

    anyways i am done with this discussion. yall have a good one.
     
    Last edited: Jul 19, 2015
  17. 4,623
    1,380
    123
    pinky

    pinky Bitsiboo's Other Half Developer

    Joined:
    Mar 8, 2015
    Messages:
    4,623
    Likes Received:
    1,380
    Trophy Points:
    123
    Gender:
    Male
    Location:
    The Great Gig in the Sky
    I admire ur dedication to the ps3. I've moved on to the 3ds/vita/ps4/xbone mostly, but all of my info will never be posted. I'm looking/testing/somewhat researching the 3ds, mostly on save files. anyway, many r aware of how much I helped other sceners especially those on ps3repack. I just wanted to say I'll never post any info on new systems ever due to Fig! and Nightdex. I think people should be on their own now which was a tough lesson for me to learn, but u shouldn't feel entitled but rather u should earn ur place. :-/
     
  18. 666
    84
    38
    No0bZiLLa

    No0bZiLLa Moderator Developer

    Joined:
    Oct 1, 2014
    Messages:
    666
    Likes Received:
    84
    Trophy Points:
    38
    im glad you admire my dedication to the ps3. it makes me feel entitled and that ive earned my place. ;)

    EDIT: also, i dont need to earn anything from you or anyone else. i wont be flashing data back to my ps3 or anyone elses ps3 without getting a legible dump first. and i also wont use a tool that makes my bad dump magically good on pc, especially when im not even able to write back to the flash 100% successfully in the first place.

    unless you can convince me that i am wrong, please dont tell me i need to earn my place, please school me and tell me where im wrong. otherwise not only am i entitled to tell my opinion, but i WILL tell my opinion about it. if you can best me in my opinion i will contract my arguments and will gladly admit that i am wrong.

    like i said, i have fixed ps3s exactly like this in the past. but it takes making sure you have a good connection first. in every post i made so far i have pretty much said MAKE SURE YOU HAVE A GOOD CONNECTION FIRST. if you think that is wrong, then you are seriously demented and your ps3 is hell bound.

    EDIT 2: why would anyone write something back to their ps3 without even being able to get a good dump first? :hmmm:
     
    Last edited: Jul 20, 2015
  19. 469
    99
    38
    littlebalup

    littlebalup Developer

    Joined:
    Oct 16, 2014
    Messages:
    469
    Likes Received:
    99
    Trophy Points:
    38
    Location:
    43°36'16.0"N 1°26'36.1"E
    You are completely right, but:

    Because

    - it may happens the NOR chip is damaged (internally). So, no way to have a valid dump even if connections are OK. But it's really rare. And even more rare to fix it.

    - some (stupid) noobs are too lazzy to read correctly tutorials, are eager to unwrap their E3 Flasher and jailbreak their console just "checking" their dumps (sorry : their only dump) using hex editor or e3 nor dump checker. Of course they will brick because bad connexions. They will retry and retry and retry, moving their clip each time untill they'll ask some help.
    It's heartbreaking but it is a fact. Still exist (and may be more).

    So, this tool is certainly not for noobs. It is for advanced users/technicians they know what they are doing to try to fix ruined consoles.
    And if this script can help to save at least one console, I'll be happy.
     
  20. 114
    58
    38
    playerkp420

    playerkp420 Developer

    Joined:
    Feb 24, 2015
    Messages:
    114
    Likes Received:
    58
    Trophy Points:
    38
    Yeah, the PS3s I fixed that way, never had a valid dump taken from them. The noobs that originally tried to downgrade, never validated their dumps. The clip was not seated correctly, but was able to read n write. Luckily when they wrote to the NOR, the clip was still not seated correctly.

    When I received the PS3s, I seated my clip correctly. But the dumps looked even worse than the bad dumps they made. I was able to get metldr and eid sections from their dump. Was able to only get bootldr from my dumps. So no i was not able to redump better data. I used all the rest of the info from a donor PS3's dump, including cvtrm from the donor. From using cvtrm from another console, it caused RSOD. But I easily fixed that with rsod.fix, after FSM downgrade.



    I don't think anyone implied you would not have to seat the clip correctly. Obviously to flash a valid dump, you would have to. But having the clip seated correctly, doesn't guarantee that you will get a valid dump. Not after someone has flashed data while it was not correct.

    Anyone can make a dump look valid on a PC. It is another thing to actually make a valid dump, with puzzle pieces. Then fix the console with it. Which is exactly what I did. A lot of those pieces weren't even from the same PS3. If there was a program to help me put those pieces together then, it would of saved me a lot of time copy n paste writing in HxD.



    Sent from my SAMSUNG-SM-G870A using Tapatalk
     

Share This Page