PS2 HDDChecker

Discussion in 'PS2 Homebrew' started by sp193, Jan 6, 2016.

  1. 739
    1,319
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    739
    Likes Received:
    1,319
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    Update 2018/12/08: v0.963 released - http://www.psx-place.com/threads/hddchecker.3855/page-2#post-148041
    Update 2018/06/20: v0.961 re-released - http://www.psx-place.com/threads/hddchecker.3855/page-2#post-127544
    Update 2018/06/19: v0.961 re-released - http://www.psx-place.com/threads/hddchecker.3855/page-2#post-127436
    Update 2018/06/18: v0.961 released - http://www.psx-place.com/threads/hddchecker.3855/page-2#post-127367
    Update 2018/06/12: v0.96 released - http://www.psx-place.com/threads/hddchecker.3855/#post-126693
    Update 2018/02/24: v0.95 released - http://www.psx-place.com/threads/hddchecker.3855/#post-101244
    EDIT 2017/02/28: v0.942 released
    EDIT 2016/12/04: v0.941 released
    EDIT 2016/09/16: v0.94 released
    EDIT 2016/08/09: v0.932 released
    Update 2016/01/06: v0.93 re-released!
    Update 2016/01/04: v0.93 released!
    Update 2014/06/26: v0.92 re-released!
    Update 2014/06/23: v0.92 released!
    Update 2013/09/30: Bug report.
    Update 2013/08/06: Initial public release.

    HDDChecker is a basic disk diagnostic tool meant for testing the health of your PlayStation 2 console's Harddisk Drive unit.

    It was conceived and constructed because I didn't want to see anymore poor SCPH-20400 units being cut open, just to have the disks within them taken out for testing. >_>

    It'll also be useful for those people who need to check the condition of their HDD units, but don't have a working IDE port on their computers to connect the disk to.

    It has the following features:
    1. Detects and lists the model, serial, firmware and S.M.A.R.T. status of HDD unit 0 (Primary Master).
    2. Performs a surface scan of the disk.
    3. Supports 48-bit LBA disks up to 2TB.
    4. Performs zero-filling of the disk with large blocks (65536 sectors for ATA-5-compliant disks, and 256 sectors for ATA-4-compliant disks).
    5. With the surface scan, bad sectors found might be remappable. Based on my research, it appears that writing to a to-be-remapped sector (those hard-to-read sectors, as recorded by the disk) will kick off the actual sector remapping process. However, this has not been widely tested.
    6. Checks for damage to the APA scheme and PFS partitions.
    7. Defragments the HDD to reclaim space.

    Notes/known limitations/bugs:
    1. Do NOT use a disk larger than 2TB with this tool, as there are currently no safety locks to prevent a LBA wraparound from occurring!
    2. Do not use (usually old) disks that are not compliant with the ATA-4 specification. Like with every other PlayStation 2 software out there that supports the ATA interface, the disk is expected to support UDMA mode 4 and S.M.A.R.T.
    3. If the disk's S.M.A.R.T. status is indicated to be no good (NG status), the disk is about to fail and should be replaced.
    4. This tool may not be able to successfully remap sectors on all drives, as there isn't an official specification on remapping sectors within the ATA specification. If writing to a bad sector isn't sufficient to cause the disk to automatically remap it, the manufacturer's tools may have to be used instead.
    5. If a bad sector cannot be remapped, the disk is dying and should be replaced.
    6. As zeros will be written to the bad sector in an attempt to cause the disk to remap it, the data stored within the bad sector will be lost. Do not choose to remap the sector if it contains data that cannot be lost.

    Supported languages
    For more information on supported languages and how support for languages can be completed, click here. A template for translating this software is provided in the downloads section of this page.

    Supported languages and their translation status LanguageStatus
    JapaneseUnassigned
    EnglishCompleted and built-in.
    FrenchUnassigned
    SpanishUnassigned
    GermanUnassigned
    ItalianUnassigned
    DutchUnassigned
    PortugueseUnassigned
    Changelog for v0.93:
    1. Updated to compile with the new PS2SDK.
    2. Renamed to HDDChecker.
    3. Revamped UI.
    4. Rewritten to use libgs, so gsKit is no longer required.
    5. Added hdck (APA checker), fsck (PFS checker), hdsk (APA defragmenter) and fssk (PFS trimmer).
    6. Changed fonts to the Google English and Japanese fonts.
    7. Added support for booting from the HDD unit.

    Downloads/links
    HDDChecker project page (go here for downloads): - PS2 HDDChecker support page -
     
    Last edited by a moderator: Mar 31, 2019
    jolek, Berion and STLcardsWS like this.
  2. 739
    1,319
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    739
    Likes Received:
    1,319
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    Today, I've sat down to try to replace my project icons with the new icons that @Berion made available... but there seems to be some problem with the old code, somewhere. The icons do not seem to be coloured. :|

    I've also found some issue with the simulation code in the HDSK module, but my HDD was just optimized earlier so I can't see if the progress computation problem (when the "optimize" option is selected) was solved.

    I do not know why the icons cannot be drawn correctly, but I may be getting wrapped up in other real-life stuff pretty soon... so I'll leave this binary here for people to test the functionality with. The latest PS2SDK modules (i.e. ATAD, HDD and PFS) were used too, so hopefully the other weird glitches (i.e. hangs when a bad sector is encountered) are taken care of.

    If/when the icon rendering issue is solved and if there are no other issues, I'll make a formal release.

    To use the binary, simply replace the same file from the latest public version with it. Link: https://www.sendspace.com/file/12ex3t
     
    uyjulian, DeViL303, Berion and 3 others like this.
  3. 2,315
    2,247
    372
    Berion

    Berion Developer

    Joined:
    Feb 3, 2015
    Messages:
    2,315
    Likes Received:
    2,247
    Trophy Points:
    372
    Gender:
    Male
    Location:
    Poland
    @sp193: Maybe it's lack of support 4/8bit PNG with indexed CLUT? Every bitmaps which I cut is optimized by optipng (which setting compression to maximum, and if colours can be reduced to 2-256 it also doing it). As I remember, I have made all buttons 4bit PNG (16 and indexed colours to keep transparency). Maybe pnglib for PS2 lack support for them (which would be new for me ;]).
     
  4. 739
    1,319
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    739
    Likes Received:
    1,319
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    I didn't exactly use the files that you provided because of what you wrote - there would be savings in space and code if everything was just nicely laid out in the form of an atlas. Also because I realized later on that the icons for D-pad buttons within the VSH icon set would be a better fit for my UI. So I used GIMP to edit (removed the extra temperature icons, home icon and the light select + start buttons) the VSH icon set to fit my needs.

    When I saved the icons, I used RGB mode. Windows says that it is a 32-bit PNG file, so I am not very sure what is wrong. But somehow this problem seems familiar, almost as if I have seen it before. :|

    Also as a side effect of me using GIMP on my own, I don't have an 8-bit icon set that looks nice anymore. I tried to convert the provided PSD file into indexed mode, but the way GIMP merges the transparency layer makes me cringe. @@
    But that is a different problem.
     
  5. 2,315
    2,247
    372
    Berion

    Berion Developer

    Joined:
    Feb 3, 2015
    Messages:
    2,315
    Likes Received:
    2,247
    Trophy Points:
    372
    Gender:
    Male
    Location:
    Poland
    @sp193: Send me please all of this bitmaps and I will look into them. :)
    Gimp doesn't fully support PSD (it's a real mess format), sometimes also lack of some effects.
     
    kozarovv and DeViL303 like this.
  6. 739
    1,319
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    739
    Likes Received:
    1,319
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    PM sent. Thank you for volunteering to look into it!
     
    kozarovv likes this.
  7. 159
    268
    122
    uyjulian

    uyjulian Developer

    Joined:
    May 27, 2017
    Messages:
    159
    Likes Received:
    268
    Trophy Points:
    122
    Gender:
    Male
    It was probably me using optipng on the assets in Open PS2 Loader, which cause some assets to not load correctly.
     
    kozarovv likes this.
  8. 739
    1,319
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    739
    Likes Received:
    1,319
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    Ah, that sounds familiar indeed. Thanks!

    So there might be a problem with our port of libpng. Or perhaps it is just a problem with the way it is used here.
     
    uyjulian and kozarovv like this.
  9. 2,315
    2,247
    372
    Berion

    Berion Developer

    Joined:
    Feb 3, 2015
    Messages:
    2,315
    Likes Received:
    2,247
    Trophy Points:
    372
    Gender:
    Male
    Location:
    Poland
    Optipng using maximum possible compression and removing all meta data (besides of course resolution, pallete etc.). I have also discovered that CellOS on PS3 doesn't handle properly alpha channel if i.e theme author doesn't turn on "clean transparent" during bitmap exporting in app/plugin (like in default buildin PNG plugin in Gimp or SuperPNG plugin in Photoshop). I have no idea what exactly those options doing but no doubt fixing strange glitches. Another issues which I found are very very light bitmaps which are not supported (?) by PSVita. It's hard to explain but in example if user have background which can be very good compressed below some (critical as it looks like) size, it will not be read/load. But if I paint something on it which making higher size after "optimization" by optipng, it will be read/load properly.

    My point is that PNG is not so simple format and it's high probability that not all libpng on the wild fully support this format.
     
  10. 739
    1,319
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    739
    Likes Received:
    1,319
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    Your new icons also gave the same problem... but I discovered that it occurs regardless of the format (8-bit and 32-bit icons). However, the "glitch" goes away if I changed the levels to 0x80 (128) for each channel of RGB (!).
    The documentation for the GS explains that behaviour, which happens if the texture mode is set to modulate, so that explains everything.
     
    Last edited: Jun 11, 2017
    kozarovv and Berion like this.
  11. 2,315
    2,247
    372
    Berion

    Berion Developer

    Joined:
    Feb 3, 2015
    Messages:
    2,315
    Likes Received:
    2,247
    Trophy Points:
    372
    Gender:
    Male
    Location:
    Poland
    ? ;)

    But I'm happy that You solved the riddle.
     
    kozarovv likes this.
  12. 739
    1,319
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    739
    Likes Received:
    1,319
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    I was planning to make a new release of HDDChecker and write a reply at the same time, but there were various outstanding problems with the PS2SDK, so that had to be put on hold.

    The GS allows us to specify how our textures will be colour-blended with the surface. It was left on MODULATE, which meant that the GS would multiply the two values. The multiplication caused the result to overflow, which naturally resulted in undefined behaviour.

    If I am not wrong, I just changed the channels of each colour of the texture to be 0x80, which is equivalent to a factor of 1.0 for the modulate colour-blending process. I could have probably also changed the colour-blending mode to DECAL, which would have caused the GS to just take the colours of the texture "as-is".

    Thanks. It has been around for way too long.
     
    Berion and kozarovv like this.
  13. 739
    1,319
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    739
    Likes Received:
    1,319
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    Changelog for v0.95:
    • Corrected logic error in HDSK simulation code (estimation for disk optimization part 2).
    • Updated to work with the latest PS2SDK revision: better stability and correctness of operation.
    • Replaced icons with new icons from Berion.

    To be honest, I got sidetracked for so long that I forgot what was changed. I only remember that a lot of important fixes were made, but perhaps they were in the form of updates to the PS2SDK's ATAD, DEV9, PFS, and HDD (APA driver) modules.

    Downloads/links:
    HDDChecker project page: https://sites.google.com/view/ysai187/home/projects/hddchecker
     
    Last edited by a moderator: Apr 19, 2019
    shingen, CountDuckula, svotib and 4 others like this.
  14. 2
    0
    5
    shingen

    shingen Forum Noob

    Joined:
    Feb 16, 2018
    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    5
    Gender:
    Male
    Does your drive have to have SMART for this to work cause I’m using an SD card.
     
  15. 2
    0
    5
    shingen

    shingen Forum Noob

    Joined:
    Feb 16, 2018
    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    5
    Gender:
    Male
    Never mind my adapter seems to have it somehow.
     
  16. 739
    1,319
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    739
    Likes Received:
    1,319
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    For this tool to work, S.M.A.R.T. does not have to be implemented.

    It is, however, required by the MBR boot program, hence any device that does not support S.M.A.R.T. cannot be used for booting.
     
  17. 739
    1,319
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    739
    Likes Received:
    1,319
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    I wanted to make a proper, new release with the new logging feature, but I am really not satisfied by the weird problems that seem to be plaguing USB support recently. It appears that the corruption that I mentioned before, were caused by a mixture of bad code that I added, plus some hardware compatibility problem between Sandisk's device and the PS2. According to Sony anyway, devices that are compliant with the USB specification should support 62-byte frames, but my Sandisk Cruzer Nano had issues with that...
    Hence I added new code to USBHDFSD that would work around the hardware errata, through correcting the unaligned buffer address in software.

    I do not know why, but my log file for fsck is always the only one of the 4 logs that will have lots of 0x00 bytes in it.
    It may be some problem with USBHDFSD, but strangely the other 3 logs have no problems... so I think it might be fine, just caused by some existing software problem that I cannot see at this point.

    I think my HDD in my SCPH-39006 has damage, but it is not a new thing (it's a 13-year old disk, with perhaps many hours of power-on and load/unload cycles). That might be why I always have 2 errors corrected, which never go away.

    The optimization progress display seems to have problems, stemming from the reports from fssk. But because I have been focusing on functional glitches in PFS and fsck, this has always been, and will probably continued to be ignored for now.

    So if anyone is willing to help try it out, here is the new file: https://www.sendspace.com/file/x1vgaf
    For the safety of your data, it may be best to not use a USB disk that contains valuable data, in case USBHDFSD is really still bugged).

    From this modification, there will be a way for people to see what was being done, and to allow us to find out whether there is actually have a problem with HDDChecker's modules repairing anybody's filesystems incorrectly. I have doubts that the logs will help much in either case, but...

    Log files will be generated:
    • Scanning:
      • hdck.log
      • fsck.log
    • Optimization:
      • fssk.log
      • hdsk.log
    There will be no log files generated for surface scanning nor zero-filling, since the goal of this logging feature is to explain the weird corruption problems that still plague PFS to date.

    I have checked through more PFS functions recently, but did not find anything that might cause actual problems. So I think the corruption-related problems that users have experienced, might have been caused by existing software that ran a bugged version of PFS, which might have ruined PFS in ways that could not be fixed. But it is very difficult to check for and to prevent....

    EDIT: I have found a problem with the fsck module. Fixing that, takes care of the log getting corrupted.
     
    Last edited: Jun 12, 2018
    jolek and svotib like this.
  18. 1,511
    1,198
    347
    jolek

    jolek Senior Member

    Joined:
    Dec 29, 2017
    Messages:
    1,511
    Likes Received:
    1,198
    Trophy Points:
    347
    Gender:
    Male
    Since you've found a root of a problem in fsck, there is no need to check it?

    Finally (error) log will be created.

    Thanks, so maybe soon we will have new version 0.96. :D
     
  19. 739
    1,319
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    739
    Likes Received:
    1,319
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    HDDChecker v0.96 released!

    Changelog for v0.96:

    • Added logging feature.
    • Fixed off-by-one error in fsck, which caused undefined behaviour.
    • Fixed incorrect logic if hdsk, which caused the simulated behaviour to not match the actual APA-level optimization operation.
    • Fixed progress computation jumping to random values or causing a freeze (division by 0).
    • Updated to build with the latest PS2SDK for updates to PFS.
    • UI update for nicer font-rendering.
    Logging Feature

    As of HDDChecker v0.96, a new logging feature has been added.
    4 log files will be generated:
    • When scanning is done:
      • hdck.log
      • fsck.log
    • When optimization is done:
      • fssk.log
      • hdsk.log
    No log files will be generated for surface scanning or zero-filling, as the purpose of these log files is allow what was done during the scanning and optimization operations to be explained.

    If you experience problems with HDDChecker's scanning and optimization functionality, please contact the developer and provide the log files.

    Downloads/Links
    HDDChecker project page (downloads at bottom): https://sites.google.com/view/ysai187/home/projects/hddchecker

    Yup. That's a good thing, since it has been a very long time since I have found anything actually wrong with it.
    With some luck, that was the only problem left.
     
    Last edited by a moderator: Apr 19, 2019
  20. 1,511
    1,198
    347
    jolek

    jolek Senior Member

    Joined:
    Dec 29, 2017
    Messages:
    1,511
    Likes Received:
    1,198
    Trophy Points:
    347
    Gender:
    Male
    I've uploaded my log from the newest version 0.96.
    Now 34 problems has been found & fixed.
    For me it's a little bit odd.
    To copy files from USBHDD to PS2HDD I used wLe from 2018/06/12.

    BTW navigating through app, when item is highlighted for the first time,
    is slow as it is with FMCB 1.96. ;)
     

    Attached Files:

    • log.7z
      File size:
      1.2 KB
      Views:
      91

Share This Page