newlib porting challenges

I'm testing on SCPH-50002 booting with latest wLE via USB.

I do have a saved cfg, video mode set to 1080i.
 
Here's a new build of OPL:
https://gitlab.com/ps2max/Open-PS2-Loader/-/jobs/445516521/artifacts/browse

A bug has been fixed here:
https://gitlab.com/ps2max/ps2sdk/-/commit/a96ae8ee982a0d56594f5676df2b3650f17ad81d

Since the "data" section (without the "." prefix) does not exist, a new section named "data" was created automatically AFTER the "_end" symbol. New memory allocations using malloc would then corrupt this "data" section.
Only the newlib file io function pointers are located in the "data" section. So using newlib file io functions after malloc could result in the crash.

The same happens in the current ps2sdk, here:
https://github.com/ps2dev/ps2sdk/blob/master/ee/libc/include/stdio.h#L196
But since these functions are almost never used, the problem would not be as visible.

@Krah @Tupakaveli @Others : could you please test OPL again, and see if there's any more issues?
 
Did some quick tests; Saving cfg in any video mode is fine now, no crashes, themes and Lang also seem fine. A couple things I experienced though.

First 2 times I tried to initiate HDD resulted in a crash. (Then it randomly worked every time after that).
Cover Art is not displaying. (My ART is bmp)
Scrolling through my games list too quickly results in a crash.

All these things could be related though since I'm doing it all from HDD, "may" be an issue with hdd support?

Loading themes from HDD is fine though, BMP background working also so probably not why cover art isn't working.

Only other thing was Notifications don't work which isn't a huge deal, I was never really happy with how it's coded anyway and wouldn't disagree if it needed to be removed.

@jolek Have you got any spare time to test this build? It'd be good to see if USB & SMB have the same issues.
 
Last edited:
Krah said:
@jolek Have you got any spare time to test this build? It'd be good to see if USB & SMB have the same issues.
Click to expand...

Thanks for mentioning me.
Currently during weekend I'm quite busy, but I'll try to do it (test it) in workdays on SLIM (through USB & SMB).
Best regards.
 
  • Like
Reactions: TnA
Krah said:
Did some quick tests; Saving cfg in any video mode is fine now, no crashes, themes and Lang also seem fine. A couple things I experienced though.
Click to expand...
Good news!

Krah said:
First 2 times I tried to initiate HDD resulted in a crash. (Then it randomly worked every time after that).
Cover Art is not displaying. (My ART is bmp)
Scrolling through my games list too quickly results in a crash.

All these things could be related though since I'm doing it all from HDD, "may" be an issue with hdd support?

Loading themes from HDD is fine though, BMP background working also so probably not why cover art isn't working.
Click to expand...
I've tested on a slim using USB and SMB. They both seem very stable, but I must say I'm not a heavy user of OPL. I will test using FAT with HDD.
The HDD code relies heavily on fileXio extensions that cannot be ported to newlib, so it's possible there's an issue there with all the changes made for the port.
The BMP code in gsKit also uses UCAB memory (WHY?!), this can explain the random crashes. Perhaps they are the same random crashes we had before with HIRES, caused by using UCAB memory. I'll look into this.

Krah said:
Only other thing was Notifications don't work which isn't a huge deal, I was never really happy with how it's coded anyway and wouldn't disagree if it needed to be removed.
Click to expand...
I'll look into this. I didn't notice it was missing. Thank you for testing!

EDIT: Notifications are now working again, see change here. New build available here:
https://gitlab.com/ps2max/Open-PS2-Loader/-/jobs/447149970/artifacts/browse
 
Last edited:
I don't know why, but when I set 720p in video settings, save changes, exit to Browser,
I'm getting BSOD.

Notifications seems to work with 1465 (tried few res = AUTO\NTSC\720p).

Settings will be saved\loaded to\from mass:/.

I have all my covers in PNG, so for now everything is fine in my config.
Haven't tried BMP...

Here are some other bugs that are still in this and in Ifcaro repo:
https://www.psx-place.com/threads/gui-overhaul-part-2.26928/page-3#post-224570.

Problem with saving settings into mc1:/
https://www.psx-place.com/threads/gui-overhaul-part-2.26928/page-4#post-225961.
 
Due to issues with current versions of OPL which I 'think' are caused by a recent gsKit update.. I wanted to test the newlib build to see if those issues are solved by newlib..

Currently people are having issues with console locks ups with OPL using the old/current toolchain.. using the newlib build of OPL I was not having any of these issues current OPL has..
However there is still one problem with the newlib build of OPL and its just BMPs. I couldn't get the console to lock up at all by speeding through my games list or anything with HDD anymore which is great!
But BMP COVs definitely don't load (I noticed flashes of garbage when trying to load) PNG COVs load fine.

@Maximus32 I'm hoping this BMP issue can be resolved and then not only will it be great to have a newlib toolchain etc but it will also fix current issues with OPL.

I know the pull requests are currently open; @sp193 in order for OPL to be fixed it seems we need newlib or to revert a gsKit commit until newlib is merged.. I will try bisect tomorrow night if I have time to find out which commit is causing current problems.
 
Krah said:
Due to issues with current versions of OPL which I 'think' are caused by a recent gsKit update..
Click to expand...
What makes you think that? I've read somewhere that not everyone has the same issues. This makes me think not everyone is using the same (latest) version of all sources. Has anyone tried to git clone and compile everything from scratch? ps2toolchain, ps2sdk, ps2sdk-ports, gsKit, OPL?

Krah said:
However there is still one problem with the newlib build of OPL and its just BMPs. I couldn't get the console to lock up at all by speeding through my games list or anything with HDD anymore which is great!
But BMP COVs definitely don't load (I noticed flashes of garbage when trying to load) PNG COVs load fine.

@Maximus32 I'm hoping this BMP issue can be resolved and then not only will it be great to have a newlib toolchain etc but it will also fix current issues with OPL.
Click to expand...
It's good to hear the port is working great. Thank you for bringing the BMP issue to the attention. Would you be able to debug this issue? I've currently got too many things on my plate. Improving gcc9 is what's currently consuming all my free time.

Krah said:
I know the pull requests are currently open; @sp193 in order for OPL to be fixed it seems we need newlib or to revert a gsKit commit until newlib is merged.. I will try bisect tomorrow night if I have time to find out which commit is causing current problems.
Click to expand...
I've recently been granted permissions on ps2dev also. So I can now pull my own PR :). Since the newlib port is a big change I think it's a good thing to leave it open for some time so people get the chance to test and/or use it.
 
I think it'll be great if you could review and merge your own work. You know best, what needs to be done.

Even if I do step in, I don't think I'm actually ever going to add value to any of your work. Things have been different, ever since I started working full-time. Even when I do (finally) get free time, I tend to avoid coding at all. Unfamiliar codebases, or codebases containing code that I don't understand well, bring back dark memories...
 
Maximus32 said:
What makes you think that? I've read somewhere that not everyone has the same issues. This makes me think not everyone is using the same (latest) version of all sources. Has anyone tried to git clone and compile everything from scratch? ps2toolchain, ps2sdk, ps2sdk-ports, gsKit, OPL?
Click to expand...
Just a hunch, there seem to be 2 different issues atm.. if using r1440 compiled by elpatas; people are getting graphics glitches on default theme (likely caused by this version not being compiled with the latest gsKit so it's missing the 4 bit CLUT).. if using r1440 by akuhaks bot; people aren't getting graphics glitches but they are getting console lock ups (likely because this version is built with latest gsKit commits)..

but yea, to be sure I will need to do as you suggested and compile everything from scratch and eliminate commits to find out exactly what's causing issues.. I will try tonight.
 
@Maximus32

Ok I rebuilt everything from scratch and compiled OPL and have freezing immediately at logo with an existing CFG (without it freezes in display settings). I then rolled back a couple commits in gsKit and rebuilt OPL and have it working without issues.. was able to narrow it down to this commit.

For whatever reason its causing problems with the current builds but I remember when you found this issue and that its required for newlib to run OPL in HiRes without problems.. so what I am hoping is that we can (temporarily) revert this commit and perhaps include it in the newlib pull request instead; that way when newlib is merged it will be included (since these lock ups aren't present in the newlib build) but it will also fix current issues people are having with OPL.

Thank you.
 
Krah said:
if using r1440 compiled by elpatas; people are getting graphics glitches on default theme (likely caused by this version not being compiled with the latest gsKit so it's missing the 4 bit CLUT).. if using r1440 by akuhaks bot; people aren't getting graphics glitches but they are getting console lock ups (likely because this version is built with latest gsKit commits)..
Click to expand...
@El_Patas please make sure you keep ps2toolchain, ps2sdk, ps2sdk-ports and gsKit updated.

Krah said:
Ok I rebuilt everything from scratch and compiled OPL and have freezing immediately at logo with an existing CFG (without it freezes in display settings). I then rolled back a couple commits in gsKit and rebuilt OPL and have it working without issues.. was able to narrow it down to this commit.
Click to expand...
Thank you for tracking it down to that commit. It's very strange that commit it causing the issue. It should be more safe now all UCAB allocations flush the data cache before using it. The only 'less safe' thing I see is that FlushCache(0); is replaced with SyncDCache(p, p+size-1);. The latter flushes only what needs to be flushed, the first one flushes everything. Flushing everything should not be needed, unless it accidentally flushes another bug ;-).

Krah said:
For whatever reason its causing problems with the current builds but I remember when you found this issue and that its required for newlib to run OPL in HiRes without problems..so what I am hoping is that we can (temporarily) revert this commit and perhaps include it in the newlib pull request instead; that way when newlib is merged it will be included (since these lock ups aren't present in the newlib build) but it will also fix current issues people are having with OPL.
Click to expand...
I don't like the idea of adding bugs to fix other bugs, so I'll see if I can find the cause of the lockup.
 
Maximus32 said:
@El_Patas please make sure you keep ps2toolchain, ps2sdk, ps2sdk-ports and gsKit updated.
Click to expand...
@Maximum32, as i said on Discord i update all this things always when a developer mark
them or inform in the pull request of a new OPL commit if they are needed to update.
As Tupakaveli said me recently he forggot to mark in the pull request that was needed upate the gskit
for the 4bit clut.

Anyway if the Akuhaks build is updated with all the gskits commits, it have a serious lock ups
and problems which they need to be fixed, and i prefer now to wait for update anything.

And better if developers can provide test builds for all we can test them, for check all the
problems, and before update and merge any new thing.

Best regards.
 
El_Patas said:
@Maximum32, as i said on Discord i update all this things always when a developer mark
them or inform in the pull request of a new OPL commit if they are needed to update.
As Tupakaveli said me recently he forggot to mark in the pull request that was needed upate the gskit
for the 4bit clut.
Click to expand...
I didn't know becouse I'm not on discord, but good.

---

I've debugged the latest OPL with a freshly compiled everything from ps2dev and ifcaro. I can trigger lockups easily with changing the resolution. Using PCSX2 I can see there's no crash or invalid memory access. When breaking OPL in PCSX2 it's always inside the function guiDelay (in OPL) or time (in ps2sdk). The time function in ps2sdk has been recently changed, and the bug is there.

I've fixed it here:
https://github.com/rickgaiser/ps2sdk/commit/0f0b76da2c9ec0910e92e135ee47bc62dbb781ec

@Krah can you check if this fixes all the current freezing issues in OPL? I'll do the PR to ps2sdk when I get feedback it's all ok. You can also check the version you currently think is working (with the reverted gsKit commit). Try changing the resolution a few times, and see if you get the timeout screen for a normal amount of time.
 
Maximus32 said:
The time function in ps2sdk has been recently changed, and the bug is there.
Click to expand...
Thanks for taking the time to find the cause, appreciated.

The inversion must've been rebased into newlib too at some point. Strange that it doesn't cause lockups there too.

Maximus32 said:
can you check if this fixes all the current freezing issues in OPL? I'll do the PR to ps2sdk when I get feedback it's all ok. You can also check the version you currently think is working (with the reverted gsKit commit). Try changing the resolution a few times, and see if you get the timeout screen for a normal amount of time.
Click to expand...
Sure, just finished work so I'll be able to test in a few hours.
 
You must log in or register to reply here.

Similar threads

Back
Top