PS2 Open PS2 Loader v1.1.0

[jolek]

lol sorry for the attachement, it was in memory and i pasted it and i dont know how to remove!

Edit -> More Options... -> Attached Files -> Delete.

 

[Tupakaveli]

Harvest Moon: A Wonderful Life Special Edition

While the game could not be debugged as the glitch never seems to occur once DECI2 communications is established, I could dump memory after the glitch has taken hold by connecting to DECI2 later on (with OPL's built-in copy of RDB).
Since I have a dump of my PS2's kernel and have been disassembling it, I could study the kernel's Thread Control Blocks (TCBs) to know the state of all threads.

This game seems to create alot of threads. When the game gets stuck, all the threads are either suspended or waiting.
What seems to be happening is thread 16 trying to wake up the main thread. However, this might be failing because the game also has additional checks around WakeupThread - to prevent a thread from waking up another thread, if the thread to be woken up is not in a Wait state. So if the main thread has not slept yet, the main thread will enter a state of eternal sleep once it does sleep because thread 16 is prohibited from waking it up.
Thread 16 will not make further attempts to wake up the main thread as it will also sleep, regardless of whether the main thread was successfully woken up or not.

Disabling the extra checks seem to make the glitch go away, but I don't know why they added extra code like that. By design, calling WakeupThread on a thread that is not sleeping will increase the wakeupCount variable of the thread, which will cause it to inhibit a number of calls to SleepThread() by that amount (which I think is fair - if the thread has to be woken up, it should be).

The mistake may be made in various places, but seems to only affect the game's initial loading screen.

   ...
   lw     v1, $0000(s1)     # v1=isMainThreadSleeping
   addiu     a0, zero, $0001
   sd     s0, $0030(sp)
   bne     v1, a0, $0011b6e0   # If main thread is not sleeping, do nothing
   sd     ra, $0040(sp)
   lui     v0, $004a
   daddu     a1, sp, zero
   addiu     s0, v0, $16a8
   jal     $00444060     # jal ReferThreadStatus
   lw     a0, $0000(s0)     # a0=mainThreadId
   lw     v1, $0000(sp)     # Get main thread status
   addiu     v0, zero, $0004     # THS_WAIT
   beq     v1, v0, $0011b6c8   # If main thread is not in THS_WAIT state
   addiu     a0, zero, $000c     # THS_WAITSUSPEND
   bnel     v1, a0, $0011b6e4   # nor in THS_WAITSUSPEND state,
   ld     s0, $0030(sp)     # Don't wake up the main thread.
__0011b6c8:
   lw     a0, $0000(s0)     # Main thread ID
   jal     $00444090     # jal WakeupThread
   nop
   lw     v1, $0000(s0)     # Check whether it was successfully woken up.
   beql     v0, v1, $0011b6e0
   sw     zero, $0000(s1) # Indicate that the main thread was woken up.
   ...

Since I never played Harvest Moon, I don't know whether this change would change the game's functionality.

Originally, I was hoping that I just needed to change the thread priorities. But giving the main thread a higher priority than thread 16 didn't seem to help.

For now, the patch will only apply to the US release (SLUS-21117): https://www.sendspace.com/file/za9i05
There seems to be a lot of Japanese releases. I doubt we'll successfully find owners of all of them to borrow the ELFs from them, so a generic patch might have to be written.

It appears to be working fine now although I did only play for 5 mins.

Thank you for taking a look at it

 

[Peppe90]

I never had problems with this game with every configuration so I can just verify if it still works.

​

It appears to be working fine now although I did only play for 5 mins.

Thank you for taking a look at it

Nothing broken on my config

 

[sp193]

Since there are too many changes, I'm starting to (once again) lean towards aborting any progress on this and phasing out SMSTCPIP. SMSTCPIP might have been based on LWIP v0.7.x, with some fixes backported over the years. LWIP is now at v2.1.2, while the PS2SDK is at v2.0.0.
I know I wrote before that the new stack is larger than SMSTCPIP, so I am hoping to shrink its size by cutting out some functions we don't use. Yes, this will make it differ from the standard LWIP code base, but that's also what SMSTCPIP is anyway - a custom version of LWIP.

I managed to reduce its size to 79KB, which is only approximately 10KB larger than SMSTCPIP. This isn't an accurate way of measurement, as this does not include the memory allocated from the heap and uninitialized data.

Unfortunately, I couldn't seem to get it to work within OPL, as it'll get stuck during the game's IOP reboot. The stack overflow bug check gets triggered, with the affected thread being MODLOAD's thread. Maybe the new LWIP thread also has higher requirements for stack space.

 

[jolek]

So many changes has been done recently.
There is no rush.
Thanks for keeping this project still alive.

 

[El_Patas]

Rev d062a18 fixed Twisted Metal - Black Online...

And what was fixed in this game?
PAL, NTSC-U or NTSC-J version?
HDD, USB or SMB?

Best regards.

 

[Vedita BR]

Mind to check this video? My ISOs, like in OPL 0.9.3 boot normally in PCSX2. I will post a SHA1 of the two ISOs soon as I can!

Twisted Metal - Black Online (USA).iso
445732864 bytes (425 MiB)
SHA1 7FFE161D267D1A19659E0AA8EB354851681CEDDF

Twisted Metal - Black Online (Europe).iso
447035392 bytes (426 MiB)
SHA1 3E1CD6830C356D747A82DB3D4B9E84C4F74BA669

PS: USB MODE!

And what was fixed in this game?
PAL, NTSC-U or NTSC-J version?
HDD, USB or SMB?

Best regards.

It began to work over USB and SMB... Thats all I know. Both PAL and NTSC-U. I am not aware of NTSC-J versions

EDIT: In HDD it was already working

 

[El_Patas]

Aha thank you, is good to know that this game works now in USB and SMB.

Best regards.

 

[Peppe90]

Now it remains:

- Ace Combat Squadron Leader/The unsung war (and probably jap version too): briefings music missing and freezes on some missions (games was perfect on OPL 0.9.2).

- Super Dragon Ball Z (all versions): BSOD at the BOOT. (the game works with OPL 0.9.3 and I think on newer builds too, I don't know precisely on which rev it got broken).

- Transformers [SLES_523.88, SLES_533.09, SLUS_206.68, probably jap version too] This games is broken only playing from SMB since a long time. After playing a while (from 2 minutes to 10/15, it's "random") it starts some audio loops/failings, then BSOD (you can IGR anyway).
The bug is present in OPL 0.9.2 (the game was even worse with it, 'cause differently from new OPL revs, fmv audio issues, couldn't be solved with MODE 1).

 

[Zarper]

So, hearing that people had issues with VMC in GT3 through SMB I decided it would be fun to bisect the issue... it wasn't.
There seem to have been a lot of issues with VMC at the time that this regression was introduced, making it hard to pin down the exact commit that made it happen.

I'll just present my findings so that anyone that wants to can dig a bit deeper and maybe find something.

These revision are in chronological order, even though the names are a bit off (the auto naming code in OPL is a bit wonky from time to time).
Good indicates a revision that is able to load GT3 with VMC and access the data inside it.
Skip = Not able to load GT3 with VMC.
Bad = Able to load but no data on VMC.

OPL r1173-Beta-7d23b38 - Good
OPL r1174-Beta-8dbaaae - Skip
OPL r1175-Beta-f574ea3 - Skip
OPL r1176-Beta-9272dd9 - Skip
OPL r1177-Beta-6dbfc72 - Skip
OPL r1178-Beta-cd88afc - Skip
OPL r1179-Beta-6ccedb7 - Skip
OPL r1180-Beta-0f59471 - Skip
OPL r1181-Beta-40be3fc - Skip
OPL r1182-Beta-5251100 - Skip
OPL r1183-Beta-d25a8ea - Skip
OPL r1184-Beta-1f793f4 - Skip
OPL r1185-Beta-45049f8 - Skip
OPL r1186-Beta-1cd46b9 - Skip
OPL r1187-Beta-7919818 - Skip
OPL r1188-Beta-ad182dc - Skip
OPL r1189-Beta-5205963 - Bad
OPL r1189-Beta-05d06ad - Skip
OPL r1190-Beta-9df6607 - Skip
OPL r1191-Beta-70fb5f5 - Skip
OPL r1193-Beta-a3e2fe0 - Bad
OPL r1194-Beta-35ffe2a - Bad

Here are all the revisions if anyone would like to try themselves.

 

[Peppe90]

So, hearing that people had issues with VMC in GT3 through SMB I decided it would be fun to bisect the issue... it wasn't.
There seem to have been a lot of issues with VMC at the time that this regression was introduced, making it hard to pin down the exact commit that made it happen.

I'll just present my findings so that anyone that wants to can dig a bit deeper and maybe find something.

These revision are in chronicle order, even though the names are a bit off (the auto naming code in OPL is a bit wonky from time to time).
Good indicates a revision that is able to load GT3 with VMC and access the data inside it.
Skip = Not able to load GT3 with VMC.
Bad = Able to load but no data on VMC.

OPL r1173-Beta-7d23b38 - Good
OPL r1174-Beta-8dbaaae - Skip
OPL r1175-Beta-f574ea3 - Skip
OPL r1176-Beta-9272dd9 - Skip
OPL r1177-Beta-6dbfc72 - Skip
OPL r1178-Beta-cd88afc - Skip
OPL r1179-Beta-6ccedb7 - Skip
OPL r1180-Beta-0f59471 - Skip
OPL r1181-Beta-40be3fc - Skip
OPL r1182-Beta-5251100 - Skip
OPL r1183-Beta-d25a8ea - Skip
OPL r1184-Beta-1f793f4 - Skip
OPL r1185-Beta-45049f8 - Skip
OPL r1186-Beta-1cd46b9 - Skip
OPL r1187-Beta-7919818 - Skip
OPL r1188-Beta-ad182dc - Skip
OPL r1189-Beta-5205963 - Bad
OPL r1189-Beta-05d06ad - Skip
OPL r1190-Beta-9df6607 - Skip
OPL r1191-Beta-70fb5f5 - Skip
OPL r1193-Beta-a3e2fe0 - Bad
OPL r1194-Beta-35ffe2a - Bad

Here are all the revisions if anyone would like to try themselves.

There's a VMC problem from SMB with Gran Turismo 4 too. According to the topic on ps2-Home, it got broken on DB_1309 (that correspond to official rev 1176).
It was good with the previous DB_1301 (official r1171).

Therefore r1174 introduced the bug for both games (and maybe for others too?)

 

[ted209]

So, hearing that people had issues with VMC in GT3 through SMB I decided it would be fun to bisect the issue... it wasn't.

Great post! Must have taken you ages, but you've proved that something has been affecting the VMC code recently.

I found a workaround for a VMC issue with SMB for Enthusia (http://www.psx-place.com/threads/vmc-bug.22207/#post-156892) it might be interesting to see if loading the VMC in USB then re-saving it in-game also works for GT3.

 

[Peppe90]

Maybe it's been made an error when doing these modifications?


- rev1306 - SP193 - ATA: Disable the device's write cache for in-game VMC support. - Sun Aug 26 15:20:08 2018 +0800

- rev1305 - SP193 - IGR: Added API to unmount devices (with interrupts enabled) before terminating OPL. - Sun Aug 26 15:20:08 2018 +0800

- rev1304 - SP193 - IGR: Lock accesses to the virtual device, to prevent the physical device from entering some weird state due to interrupting an ongoing transfer. - Sun Aug 26 15:20:08 2018 +0800

- rev1303 - SP193 - IGR: deinitialize if the game calls Exit(). In the current implementation, Exit() will act according to IGR settings. If an exit path is specified, it will boot the specified ELF. Otherwise, it will return to the browser. - Sun Aug 26 15:20:04 2018 +0800

 

[Vedita BR]

So, hearing that people had issues with VMC in GT3 through SMB I decided it would be fun to bisect the issue... it wasn't.
There seem to have been a lot of issues with VMC at the time that this regression was introduced, making it hard to pin down the exact commit that made it happen.

I'll just present my findings so that anyone that wants to can dig a bit deeper and maybe find something.

These revision are in chronicle order, even though the names are a bit off (the auto naming code in OPL is a bit wonky from time to time).
Good indicates a revision that is able to load GT3 with VMC and access the data inside it.
Skip = Not able to load GT3 with VMC.
Bad = Able to load but no data on VMC.

OPL r1173-Beta-7d23b38 - Good
OPL r1174-Beta-8dbaaae - Skip
OPL r1175-Beta-f574ea3 - Skip
OPL r1176-Beta-9272dd9 - Skip
OPL r1177-Beta-6dbfc72 - Skip
OPL r1178-Beta-cd88afc - Skip
OPL r1179-Beta-6ccedb7 - Skip
OPL r1180-Beta-0f59471 - Skip
OPL r1181-Beta-40be3fc - Skip
OPL r1182-Beta-5251100 - Skip
OPL r1183-Beta-d25a8ea - Skip
OPL r1184-Beta-1f793f4 - Skip
OPL r1185-Beta-45049f8 - Skip
OPL r1186-Beta-1cd46b9 - Skip
OPL r1187-Beta-7919818 - Skip
OPL r1188-Beta-ad182dc - Skip
OPL r1189-Beta-5205963 - Bad
OPL r1189-Beta-05d06ad - Skip
OPL r1190-Beta-9df6607 - Skip
OPL r1191-Beta-70fb5f5 - Skip
OPL r1193-Beta-a3e2fe0 - Bad
OPL r1194-Beta-35ffe2a - Bad

Here are all the revisions if anyone would like to try themselves.

Thanks for the link

 

[Zarper]

After a bit of trail and error I believe I've found something. I'm not a programmer, so I don't know why it works and if it's wrong/correct.
But adding back

void DeviceDeinit(void)
{
smb_Disconnect();
}

in "device.smb.c" that was removed in 8dbaaae, VMC starts to work again.

Latest OPL with the line added: https://www.sendspace.com/file/2intcx

 

[sp193]

Rev d062a18 fixed Twisted Metal - Black Online...

That is great...but I never changed anything that could fix this problem. So I would not consider it fixed.

It appears to be working fine now although I did only play for 5 mins.

Thank you for taking a look at it

Thanks.

And what was fixed in this game?
PAL, NTSC-U or NTSC-J version?
HDD, USB or SMB?

Best regards.

I would not consider this fixed, it if was a problem.

After a bit of trail and error I believe I've found something. I'm not a programmer, so I don't know why it works and if it's wrong/correct.
But adding back

void DeviceDeinit(void)
{
smb_Disconnect();
}

in "device.smb.c" that was removed in 8dbaaae, VMC starts to work again.

Latest OPL with the line added: https://www.sendspace.com/file/2intcx

That line was deleted from this commit because I moved it to deviceUnmount(), which I forgot to call. This was corrected later on.

You need to also check whether the VMC file could be opened by OPL. If it displayed the error message about the VMC file, then the VMC cannot be used in that session.

Before that series of commits, it was also not possible to boot the same game again with VMC, due to OPL not releasing the handle during IGR.

 

[Zarper]

That line was deleted from this commit because I moved it to deviceUnmount(), which I forgot to call. This was corrected later on.

You need to also check whether the VMC file could be opened by OPL. If it displayed the error message about the VMC file, then the VMC cannot be used in that session.

Before that series of commits, it was also not possible to boot the same game again with VMC, due to OPL not releasing the handle during IGR.

That was one of the reasons I said it was hard to bisect due to multiple issues with VMC at the same time.

Without "void DeviceDeinit(void){smb_Disconnect();}" GT3 loads fine and there is a MC in slot 1, but it is empty.
With it the game loads fine and the MC has saved data on it.
I provided a build of OPL r1319 with the line re-added to make it easy to test in my previous post.

 

[Zarper]

Just a thought.
If smb_Disconnect() is called inside DeviceDeinit() and works but when it's called inside DeviceUnmount() it doesn't. Could it not be some issue with DeviceUnmount()?

Not that I understand why the shutdown function would affect reading of the VMC. Is that called at any time during setup?

Also, could anyone else report if it fixes their issues with VMC on SMB like GT4 and Enthusia?

 

[sp193]

I have a question: can SMB be currently used to create & format VMCs properly? Does this VMC function even work at all, on USB and/or SMB?

I was adjusting SMBMAN so that reading & writing performance can be higher, by allowing for longer reads & writes to be done. This was attempted sometime in 2015, but I rolled back all changes as I gave up trying to adjust it then.
But somehow, OPL didn't even seem to try to create the VMC before error'ing out, stating that it "Failed to format VMC file".
I didn't even see the SMB Open_AndX command with write access. So it never tried creating a new file.
If I created a new file on my PC, then it would still error out and delete the file I created. But again, still no request with write access was made.

I don't know when this started to become a thing, but I am now also unable to log into my PC with my normal user account. SMB support has always been a can of worms, but I also have left my PS2 to use the GUEST account for as far as I can remember.

I am starting to remember why I hate this particular part of OPL.

Just a thought.
If smb_Disconnect() is called inside DeviceDeinit() and works but when it's called inside DeviceUnmount() it doesn't. Could it not be some issue with DeviceUnmount()?

No. It's only called when IGR is done. DeviceDeinit() is called when the PS2 is going to be powered-off.

Not that I understand why the shutdown function would affect reading of the VMC. Is that called at any time during setup?

Not at all...

 

[Zarper]

I have a question: can SMB be currently used to create & format VMCs properly? Does this VMC function even work at all, on USB and/or SMB?

Using PSgen:1 for testing (hint, hint, nudge, nudge, still have graphical glitches) with r1319 on SMB I can create, format and delete VMCs that work correctly. However, OPL will sometimes freeze the console during the format process, but as long as you wait a sufficient amount of time before turning the console off, the VMC will work correctly.

Same result on my DeviceDeinit() version.