PS2 wLaunchELF Release Thread

[sp193]

Hi again,

In Engrish by Google :

Good evening.

It was not my intention to talk about writing speed even though at the end of my talk I talked about it.

The main problem is that no erasure is possible at home on this key and also surely on other more or less big. wLE crashes, unable to erase file data by file or multiple files at the same time (multiple selections). A much older ULE does it very well, no problem appears, the erase operation is finished very well. The copy, file transfer goes perfectly.

Where is, where does the problem come from ??? I am surely not the only one who has malfunction on USB !!!

Did you wait? I know it can be slow when it comes to deleting files, sometimes taking 1-2 minutes.

What is the most efficient "cluster" size with an 8GB key?

Performance will go up, if you use larger clusters. You can use 32KB clusters on a 8GB disk. The only cost, is wasted space if you keep a lot of small files on the disk (i.e. even if the file is 1 byte in size, it will take up 1x32KB cluster).

 

[Algol]

Bonjour @sp193.

Oui, j'ai attendu de l'ordre de 30 secondes mais pas plus car que ce soit pour un fichier de 8Mo (une VMC) ou de 64Mo (VMC aussi) c'est pareil, wLE se plante et la LED de la clef USB reste clignotante. C'est trop long !!! uLE-4.42ev est bien plus rapide dans ce genre de suppression quelles que soient les tailles des Clusters (clef de 16, 32, 64 Go). Ces clefs ont été formatées par mon Win-7 mais le problème est identique si c'est le Mac-Mini qui le fait.
J'ai travaillé dans l'Informatique industrielle dans une grande entreprise Française et j'ai aussi été un bon "Hacker" par le passé.

Yes, I waited about 30 seconds but no more because for a file of 8MB (VMC) or 64MB (VMC too) it's the same, wLE crashes and the LED of the key USB remains flashing. It is too long !!! uLE-4.42ev is much faster in this kind of removal regardless of the size of Clusters (key 16, 32, 64 GB). These keys have been formatted by my Win-7 but the problem is the same if it is the Mac-Mini that does.
I worked in Industrial IT in a large French company and I was also a good "hacker" in the past.

Kind regards.

 

[sp193]

I understand. But if it needs 1-2 minutes to complete updating FAT and you disconnected/reset the PS2 before it completes, it is a user error.
The PS2 only supports USB 1.1 (12Mbit). The cache system is not very good for speed either, as we try to save RAM. So it will be worse than a Pentium PC from 1997.

Old versions of LaunchELF may be fast, but they are not fully compliant with the USB Mass Storage specification. Reliability is more important than speed.

 

[TnA]

Well, but there seems to be an issue even with a low amount of files...

Even a. PC from 1997 should be capable of deleting i.e. 8 files in less than half a minute...

 

[sp193]

Well, but there seems to be an issue even with a low amount of files...

Even a. PC from 1997 should be capable of deleting i.e. 8 files in less than half a minute...

It's not really just about the number of files, but how the cache works and how many blocks it'll read. This design prevents longer reads to be made, forcing transfers into tiny groups of 4096 bytes. As of 2014, reading & writing was noted to have become significantly slower due to the introduction of more code that will handle the completion of the I/O request.
With longer transfers, it is still possible to get over 1MB/s, while the throughput with smaller request will be mediocre.
The cache's design is also responsible for the workaround here, as the cache blocks may not be aligned with the clusters. It's also why there is a need to read a full block before writing when writing a file, even if the whole cluster is to be replaced.

If you can and are willing to pour in resources to redo the design and implementation of the cache system, please do. For as far as I know, it can delete files properly, just that it takes a while. I will not do this because there are too many other things to do, which are more important than trying to fix an obscure feature, for which there is no easy solution to.

I did try to see why there was this difference in transfer speeds after this commit in 2014, but didn't really find the difference. The longer code just seemed to cause that. Even now, it just seems that way.
Maximus32 might have found the reason for it during his experiments, but I forgot whether he found the answer.

 

[Algol]

Well, but there seems to be an issue even with a low amount of files...

Even a. PC from 1997 should be capable of deleting i.e. 8 files in less than half a minute...

Là, je suis bien d'accord avec toi @TnA. Le plus mauvais des PC fait bien mieux que wLE ne serais-ce que sur de l'USB1.
J'ai un vieux Mac qui n'a que de l'USB1 et l'effacement de données "en-tête seul" mais pas un effacement sécurisé (simple ou multi-passe) est aussi rapide que notre vieux "uLE-4.42ev". Cela prends moins de 15 secondes, largement.

Merci pour ton soutien.

 

[Algol]

Toi (@SP193) ainsi que Maximus32 et tous ceux que j'oublie, je vous remercie pour vos recherches et solutions données à l'amélioration de uLE.
Si, un jour, vous trouvez la réponse à cette énigme de ralentissement en effacement "simple" d'en-tête (ici on ne parle pas d'effacement sécurisé de données) alors, je serai toujours à vos côtés pour faire certains tests approfondis. J'en ai les moyens et le temps (retraité).

Algol "le papy".

 

[Zwei]

What's the latest version of LaunchELF?

 

[sp193]

What's the latest version of LaunchELF?

Technically speaking, the last commit was on November 24th: https://github.com/AKuHAK/uLaunchELF/commits/master

I want to hand over the building of new builds to @akuhak, since he already has a bot for doing so. Right now, there seems to be some technical difficulties, but I no longer wish to get into the way of his efforts; automating the builds of daily builds is surely the way to go.

 

[jolek]

What's the latest version of LaunchELF?

Technically speaking, the last commit was on November 24th: https://github.com/AKuHAK/uLaunchELF/commits/master

I want to hand over the building of new builds to @akuhak, since he already has a bot for doing so. Right now, there seems to be some technical difficulties, but I no longer wish to get into the way of his efforts; automating the builds of daily builds is surely the way to go.

I don't know if I did the right thing... if not

I've taken out wLe from latest FMCB\FHDB v1.965 (08\12\2018) and uploaded into the Resources.

Automated daily build system for OPL & wLe are broken since 01\11\2018.
But I understand that maintaining\updating script can be "painful".

Like I mention before, if I did something wrong

and I will not do it again.
I really also do not want to get into the way of someone else efforts.

 

[sp193]

I don't know if I did the right thing... if not

I've taken out wLe from latest FMCB\FHDB v1.965 (08\12\2018) and uploaded into the Resources.

Automated daily build system for OPL & wLe are broken since 01\11\2018.
But I understand that maintaining\updating script can be "painful".

Like I mention before, if I did something wrong

and I will not do it again.
I really also do not want to get into the way of someone else efforts.

In light of this problem of the bot not working, I think it is okay.

 

[TnA]

Aaahahaaa... That Emoji @jolek!

 

[jolek]

Is there is something wrong with commit 0ed4589 from 24\11\2018?

At github I notice this error:

 

[sp193]

Travis CI has not been working properly on that repository for a long time. But because it is not my repository, I don't think I can fix it. Also because I lack the knowledge for working on it.

 

[akuhak]

Is there is something wrong with commit 0ed4589 from 24\11\2018?

At github I notice this error:

Its a funny thing cause this was first time when Travis not failed with timeout error. For now only reason why Travis does not work - it is ps2toolchain.
Script every time tries to compile ps2toolchain from scratch. But ps2toolchain compilation produce enormous logs and Travis freeze after 10 000 lines of output passed (ps2toolchain uses much more than 10 000). If i completely silenced all outputs from ps2toolchain then Travis errored after some time with "No output more then 10 minutes" error. I kept this variant for now.

Recently sp193 changed ps2toolchain so it can be compiled by using multi-core systems - so suddenly ps2toolchain compiled before time limit passed. Unfortunately, it seems that ps2toolchain does not support multi-core compilation so Travis failed at ps2packer compilation.

I was planning long time ago to switch to Docker image but it will lead to some potential problem that ps2toolchain should be updated manually and from my repository. I probably will do it cause ps2toolchain can be left outdated.

 

[sp193]

Its a funny thing cause this was first time when Travis not failed with timeout error. For now only reason why Travis does not work - it is ps2toolchain.
Script every time tries to compile ps2toolchain from scratch. But ps2toolchain compilation produce enormous logs and Travis freeze after 10 000 lines of output passed (ps2toolchain uses much more than 10 000). If i completely silenced all outputs from ps2toolchain then Travis errored after some time with "No output more then 10 minutes" error. I kept this variant for now.

Is it not possible to let it wait indefinitely for the ps2toolchain script to finish? It will always return a status (as in everything else in Linux, 0 = no error).

Would it be desirable if I changed the scripts to accept an option for silencing output from Make? I believe this should make things easier for you to maintain too. Or are you already able to do this without changing ps2toolchain?

I was planning long time ago to switch to Docker image but it will lead to some potential problem that ps2toolchain should be updated manually and from my repository. I probably will do it cause ps2toolchain can be left outdated.

I'm curious. If you aren't using Docker now, what are you currently using?

 

[akuhak]

Is it not possible to let it wait indefinitely for the ps2toolchain script to finish? It will always return a status (as in everything else in Linux, 0 = no error).

Would it be desirable if I changed the scripts to accept an option for silencing output from Make? I believe this should make things easier for you to maintain too. Or are you already able to do this without changing ps2toolchain?

This would be very nice thing! I was trying to find a fix for it but all solutions failed. Travis has built-in feature to wait more than 10 minutes, but it dont work dunno know why. Im using ugly solution

bash ./toolchain.sh 3 > /tmp/PS2TOOLCHAIN3.log 2>&1

for silencing all outputs. Stage 3 on single thread takes more than 90 minutes. If it will be possible to make ps2toolchain scripts more silent (e.g. showing only warnings and errors) it will be really nice addition.

I'm curious. If you aren't using Docker now, what are you currently using?

Im using docker in daily autobot, however it has the same limitation with ps2toolchain. In Docker ps2toolchain is compiled from my github forked repository so when some changes are done in main repository, there can be some gap cause I should update it manually.

But in uLaunchELF repo I was trying to get rid of docker. Not only because ps2toolchain can be outdated, but also because docker has really crappy travis integration. Below simplest example for mounting docker image

  - docker run -d akuhak/ps2toolchain bash -c "while true; do sleep 5; done" > container_id
  - docker exec -t $(cat container_id) bash -c "cd /; ls -l"

This2 lines only for running ls -l inside root directory Not that easy as simple make inside travis.

 

[TnA]

If the PS2SDK and stuff would support multi-core/Thread-support and build correctly with it, that would be a hughe improvement regarding speed and performance.

That could especially be interesting for the things which take a lot of time to compile (a big app, or the PS2SDK/toolchain, etc.).

 

[Sparks]

Hi All,

Just got a HDD for my PS2 just wondering if someone can re-upload the files for the "LaunchELF build 2018/07/23:" all the links ive tried appear to have expired

 

[sp193]

Why do you need that file? If possible, delete any new software that was compiled between June 8th and late October, as those will likely contain a faulty USBHDFSD module that could corrupt files, if used to write files to a USB device.