PS2 wLaunchELF Release Thread

Discussion in 'PS2 Homebrew' started by sp193, May 10, 2017.

  1. 738
    1,318
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    738
    Likes Received:
    1,318
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    Hi again,

    Did you wait? I know it can be slow when it comes to deleting files, sometimes taking 1-2 minutes.

    Performance will go up, if you use larger clusters. You can use 32KB clusters on a 8GB disk. The only cost, is wasted space if you keep a lot of small files on the disk (i.e. even if the file is 1 byte in size, it will take up 1x32KB cluster).
     
  2. 107
    66
    82
    Algol

    Algol Member

    Joined:
    Sep 20, 2016
    Messages:
    107
    Likes Received:
    66
    Trophy Points:
    82
    Gender:
    Male
    Bonjour @sp193.

    Oui, j'ai attendu de l'ordre de 30 secondes mais pas plus car que ce soit pour un fichier de 8Mo (une VMC) ou de 64Mo (VMC aussi) c'est pareil, wLE se plante et la LED de la clef USB reste clignotante. C'est trop long !!! uLE-4.42ev est bien plus rapide dans ce genre de suppression quelles que soient les tailles des Clusters (clef de 16, 32, 64 Go). Ces clefs ont été formatées par mon Win-7 mais le problème est identique si c'est le Mac-Mini qui le fait.
    J'ai travaillé dans l'Informatique industrielle dans une grande entreprise Française et j'ai aussi été un bon "Hacker" par le passé.

    Yes, I waited about 30 seconds but no more because for a file of 8MB (VMC) or 64MB (VMC too) it's the same, wLE crashes and the LED of the key USB remains flashing. It is too long !!! uLE-4.42ev is much faster in this kind of removal regardless of the size of Clusters (key 16, 32, 64 GB). These keys have been formatted by my Win-7 but the problem is the same if it is the Mac-Mini that does.
    I worked in Industrial IT in a large French company and I was also a good "hacker" in the past.

    Kind regards.
     
    jolek likes this.
  3. 738
    1,318
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    738
    Likes Received:
    1,318
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    I understand. But if it needs 1-2 minutes to complete updating FAT and you disconnected/reset the PS2 before it completes, it is a user error.
    The PS2 only supports USB 1.1 (12Mbit). The cache system is not very good for speed either, as we try to save RAM. So it will be worse than a Pentium PC from 1997.

    Old versions of LaunchELF may be fast, but they are not fully compliant with the USB Mass Storage specification. Reliability is more important than speed.
     
    Berion likes this.
  4. 1,219
    660
    222
    TnA

    TnA Senior Member

    Joined:
    Jul 1, 2018
    Messages:
    1,219
    Likes Received:
    660
    Trophy Points:
    222
    Gender:
    Male
    Location:
    Germany --> Saxony
    Well, but there seems to be an issue even with a low amount of files...

    Even a. PC from 1997 should be capable of deleting i.e. 8 files in less than half a minute...
     
    Algol likes this.
  5. 738
    1,318
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    738
    Likes Received:
    1,318
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    It's not really just about the number of files, but how the cache works and how many blocks it'll read. This design prevents longer reads to be made, forcing transfers into tiny groups of 4096 bytes. As of 2014, reading & writing was noted to have become significantly slower due to the introduction of more code that will handle the completion of the I/O request.
    With longer transfers, it is still possible to get over 1MB/s, while the throughput with smaller request will be mediocre.
    The cache's design is also responsible for the workaround here, as the cache blocks may not be aligned with the clusters. It's also why there is a need to read a full block before writing when writing a file, even if the whole cluster is to be replaced.

    If you can and are willing to pour in resources to redo the design and implementation of the cache system, please do. For as far as I know, it can delete files properly, just that it takes a while. I will not do this because there are too many other things to do, which are more important than trying to fix an obscure feature, for which there is no easy solution to.

    I did try to see why there was this difference in transfer speeds after this commit in 2014, but didn't really find the difference. The longer code just seemed to cause that. Even now, it just seems that way.
    Maximus32 might have found the reason for it during his experiments, but I forgot whether he found the answer.
     
    TnA likes this.
  6. 107
    66
    82
    Algol

    Algol Member

    Joined:
    Sep 20, 2016
    Messages:
    107
    Likes Received:
    66
    Trophy Points:
    82
    Gender:
    Male
    Là, je suis bien d'accord avec toi @TnA. Le plus mauvais des PC fait bien mieux que wLE ne serais-ce que sur de l'USB1.
    J'ai un vieux Mac qui n'a que de l'USB1 et l'effacement de données "en-tête seul" mais pas un effacement sécurisé (simple ou multi-passe) est aussi rapide que notre vieux "uLE-4.42ev". Cela prends moins de 15 secondes, largement.

    Merci pour ton soutien.
     
  7. 107
    66
    82
    Algol

    Algol Member

    Joined:
    Sep 20, 2016
    Messages:
    107
    Likes Received:
    66
    Trophy Points:
    82
    Gender:
    Male
    Toi (@SP193) ainsi que Maximus32 et tous ceux que j'oublie, je vous remercie pour vos recherches et solutions données à l'amélioration de uLE.
    Si, un jour, vous trouvez la réponse à cette énigme de ralentissement en effacement "simple" d'en-tête (ici on ne parle pas d'effacement sécurisé de données) alors, je serai toujours à vos côtés pour faire certains tests approfondis. J'en ai les moyens et le temps (retraité).

    Algol "le papy".
     
  8. 428
    130
    72
    Zwei

    Zwei Member

    Joined:
    Mar 21, 2015
    Messages:
    428
    Likes Received:
    130
    Trophy Points:
    72
    What's the latest version of LaunchELF?
     
  9. 738
    1,318
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    738
    Likes Received:
    1,318
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    Technically speaking, the last commit was on November 24th: https://github.com/AKuHAK/uLaunchELF/commits/master

    I want to hand over the building of new builds to @akuhak, since he already has a bot for doing so. Right now, there seems to be some technical difficulties, but I no longer wish to get into the way of his efforts; automating the builds of daily builds is surely the way to go.
     
    TnA likes this.
  10. 1,507
    1,196
    347
    jolek

    jolek Senior Member

    Joined:
    Dec 29, 2017
    Messages:
    1,507
    Likes Received:
    1,196
    Trophy Points:
    347
    Gender:
    Male
    I don't know if I did the right thing... if not [​IMG]
    I've taken out wLe from latest FMCB\FHDB v1.965 (08\12\2018) and uploaded into the Resources.

    Automated daily build system for OPL & wLe are broken since 01\11\2018.
    But I understand that maintaining\updating script can be "painful".

    Like I mention before, if I did something wrong [​IMG]
    and I will not do it again.
    I really also do not want to get into the way of someone else efforts.
     
  11. 738
    1,318
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    738
    Likes Received:
    1,318
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    In light of this problem of the bot not working, I think it is okay. :grin:
     
    jolek likes this.
  12. 1,219
    660
    222
    TnA

    TnA Senior Member

    Joined:
    Jul 1, 2018
    Messages:
    1,219
    Likes Received:
    660
    Trophy Points:
    222
    Gender:
    Male
    Location:
    Germany --> Saxony
    Aaahahaaa... That Emoji @jolek!
     
  13. 1,507
    1,196
    347
    jolek

    jolek Senior Member

    Joined:
    Dec 29, 2017
    Messages:
    1,507
    Likes Received:
    1,196
    Trophy Points:
    347
    Gender:
    Male
    Is there is something wrong with commit 0ed4589 from 24\11\2018?

    At github I notice this error:
    [​IMG]
     
  14. 738
    1,318
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    738
    Likes Received:
    1,318
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    Travis CI has not been working properly on that repository for a long time. But because it is not my repository, I don't think I can fix it. Also because I lack the knowledge for working on it.
     
    jolek likes this.
  15. 48
    71
    62
    akuhak

    akuhak Member

    Joined:
    Jul 4, 2017
    Messages:
    48
    Likes Received:
    71
    Trophy Points:
    62
    Its a funny thing cause this was first time when Travis not failed with timeout error. For now only reason why Travis does not work - it is ps2toolchain.
    Script every time tries to compile ps2toolchain from scratch. But ps2toolchain compilation produce enormous logs and Travis freeze after 10 000 lines of output passed (ps2toolchain uses much more than 10 000). If i completely silenced all outputs from ps2toolchain then Travis errored after some time with "No output more then 10 minutes" error. I kept this variant for now.

    Recently sp193 changed ps2toolchain so it can be compiled by using multi-core systems - so suddenly ps2toolchain compiled before time limit passed. Unfortunately, it seems that ps2toolchain does not support multi-core compilation so Travis failed at ps2packer compilation.

    I was planning long time ago to switch to Docker image but it will lead to some potential problem that ps2toolchain should be updated manually and from my repository. I probably will do it cause ps2toolchain can be left outdated.
     
    sp193 and jolek like this.
  16. 738
    1,318
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    738
    Likes Received:
    1,318
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    Is it not possible to let it wait indefinitely for the ps2toolchain script to finish? It will always return a status (as in everything else in Linux, 0 = no error).

    Would it be desirable if I changed the scripts to accept an option for silencing output from Make? I believe this should make things easier for you to maintain too. Or are you already able to do this without changing ps2toolchain?

    I'm curious. If you aren't using Docker now, what are you currently using?
     
    akuhak likes this.
  17. 48
    71
    62
    akuhak

    akuhak Member

    Joined:
    Jul 4, 2017
    Messages:
    48
    Likes Received:
    71
    Trophy Points:
    62
    This would be very nice thing! I was trying to find a fix for it but all solutions failed. Travis has built-in feature to wait more than 10 minutes, but it dont work dunno know why. Im using ugly solution
    Code:
    bash ./toolchain.sh 3 > /tmp/PS2TOOLCHAIN3.log 2>&1
    for silencing all outputs. Stage 3 on single thread takes more than 90 minutes. If it will be possible to make ps2toolchain scripts more silent (e.g. showing only warnings and errors) it will be really nice addition.
    Im using docker in daily autobot, however it has the same limitation with ps2toolchain. In Docker ps2toolchain is compiled from my github forked repository so when some changes are done in main repository, there can be some gap cause I should update it manually.

    But in uLaunchELF repo I was trying to get rid of docker. Not only because ps2toolchain can be outdated, but also because docker has really crappy travis integration. Below simplest example for mounting docker image
    Code:
      - docker run -d akuhak/ps2toolchain bash -c "while true; do sleep 5; done" > container_id
      - docker exec -t $(cat container_id) bash -c "cd /; ls -l"
    This2 lines only for running ls -l inside root directory :) Not that easy as simple make inside travis.
     
  18. 1,219
    660
    222
    TnA

    TnA Senior Member

    Joined:
    Jul 1, 2018
    Messages:
    1,219
    Likes Received:
    660
    Trophy Points:
    222
    Gender:
    Male
    Location:
    Germany --> Saxony
    If the PS2SDK and stuff would support multi-core/Thread-support and build correctly with it, that would be a hughe improvement regarding speed and performance.

    That could especially be interesting for the things which take a lot of time to compile (a big app, or the PS2SDK/toolchain, etc.).
     
    Last edited: Dec 15, 2018
  19. 2
    0
    5
    Sparks

    Sparks Forum Noob

    Joined:
    Jan 1, 2019
    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    5
    Gender:
    Male
    Hi All,

    Just got a HDD for my PS2 just wondering if someone can re-upload the files for the "LaunchELF build 2018/07/23:" all the links ive tried appear to have expired :(
     
  20. 738
    1,318
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    738
    Likes Received:
    1,318
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    Why do you need that file? If possible, delete any new software that was compiled between June 8th and late October, as those will likely contain a faulty USBHDFSD module that could corrupt files, if used to write files to a USB device.
     
    Algol likes this.
Tags:

Share This Page