Various PS2 projects

Discussion in 'General PS2 Discussion' started by uyjulian, Oct 11, 2018.

  1. 159
    267
    122
    uyjulian

    uyjulian Developer

    Joined:
    May 27, 2017
    Messages:
    159
    Likes Received:
    267
    Trophy Points:
    122
    Gender:
    Male
    I decided to look a bit at OSDSYS 1.10a/20000727. For those who are interested:

    0x200258 appears to load all sounds.
    0x204E98 is the giant file loading function.
    0x205330 is the function referencing the filenames of the resources.
    0x2290f0 is the function for loading image assets.
    0x2239B8 appears to be another function for loading image assets.
    0x227E00 appears to load all "C" image type.
    0x209C30 appears to load all fonts.
     
    TnA likes this.
  2. 159
    267
    122
    uyjulian

    uyjulian Developer

    Joined:
    May 27, 2017
    Messages:
    159
    Likes Received:
    267
    Trophy Points:
    122
    Gender:
    Male
    Let's talk about PSX emulation on PS2.
    The current PSX emulators on PS2:
    PS2PSXe has low compatibility
    POPStarter relies on official SCE files, and compatibility is still not great

    There is a forked version of PCSX with MIPS recompiler support: https://github.com/dmitrysmagin/pcsx4all
    It would be nice to port it to the PS2.

    Also, let's talk about reviving the PS2 homebrew scene. I notice that Dreamcast has a few games still released by homebrew developers. I think PS2 lasers die faster than Dreamcast lasers.
    Most PS2s can run updates from the memory card, which allows projects like FreeMCBoot to function.
    Nowadays, eMMC chips are cheap (4GB=USD6.77 for one unit.)
    PS2 clone memory cards are also very cheap (128MB=~USD4.99 for one unit.)

    I propose a design: a memory card with an internal processor connected to eMMC that emulates 8MB memory card, but can switch to access the full eMMC. Maybe the Chinese can start producing these instead of the 128MB cards?
     
    TnA likes this.
  3. 53
    12
    37
    Anonamous

    Anonamous Member

    Joined:
    Aug 26, 2018
    Messages:
    53
    Likes Received:
    12
    Trophy Points:
    37
    Occupation:
    Electrical Engineering Student
    It is quite possible to do as long as there is a software implementation in the ps2sdk so that homebrew software and fmcb know that there is in fact more than 8mb and how much is available. It would be like the scum rip-off 10Tb micro SD cards that really only have 4Mb but report 10Tb to the computer... Except the opposite.

    With that being said, what is the benefit compared to a standard USB drive? I'm under the impression that the memory card read/write speeds are about the same if not slower than USB.
     
  4. 1,219
    660
    222
    TnA

    TnA Senior Member

    Joined:
    Jul 1, 2018
    Messages:
    1,219
    Likes Received:
    660
    Trophy Points:
    222
    Gender:
    Male
    Location:
    Germany --> Saxony
    So it is meant to be running on a MIPS?
    Yes this could work, but a custom implementation/emulator specifically meant for the PS2 would be better IMO.

    Regarding a Recompiler... I wouldn't even use that...

    I would rather redirect the Syscalls or Opcodes + use a TLB and execute it natively... (at least the CPU-Emulation)


    LOOOOOOOL... You are not the first to propose this! :D
    I'm even still working on it!

    + you can use it for other things as well (at least that's the intention) like connecting USB-Controllers or BT-Controllers, emulate MultiTaps and so on...

    @pelvicthrustman's code (self made BT-Adapter) is quite interesting, regarding this...!


    @Anonamous: Actually, it is the MC's memory which is slow... It should read ~1.6MB/s from a direct SD-Adapter!
     
    Last edited: Jan 31, 2019
  5. 53
    12
    37
    Anonamous

    Anonamous Member

    Joined:
    Aug 26, 2018
    Messages:
    53
    Likes Received:
    12
    Trophy Points:
    37
    Occupation:
    Electrical Engineering Student
    Do they use a standard protocol. Protocol may not be the right word but is it like a usb or serial connection?
     
  6. 159
    267
    122
    uyjulian

    uyjulian Developer

    Joined:
    May 27, 2017
    Messages:
    159
    Likes Received:
    267
    Trophy Points:
    122
    Gender:
    Male
    SD/eMMC uses SPI or SDIO.
     
  7. 53
    12
    37
    Anonamous

    Anonamous Member

    Joined:
    Aug 26, 2018
    Messages:
    53
    Likes Received:
    12
    Trophy Points:
    37
    Occupation:
    Electrical Engineering Student
    Ah yes interfacing with the chips is no issue, what does the PS2 memory card port use.
     
  8. 159
    267
    122
    uyjulian

    uyjulian Developer

    Joined:
    May 27, 2017
    Messages:
    159
    Likes Received:
    267
    Trophy Points:
    122
    Gender:
    Male
    TnA and Anonamous like this.
  9. 53
    12
    37
    Anonamous

    Anonamous Member

    Joined:
    Aug 26, 2018
    Messages:
    53
    Likes Received:
    12
    Trophy Points:
    37
    Occupation:
    Electrical Engineering Student
    Ok. I can be wrong as ImI still learning but I believe most of the issue faced there can be resolved with I microcontroller. This could.

    Did you have a specific emmc in mind?
     
  10. 159
    267
    122
    uyjulian

    uyjulian Developer

    Joined:
    May 27, 2017
    Messages:
    159
    Likes Received:
    267
    Trophy Points:
    122
    Gender:
    Male
    I don't have a specific model in mind. They should use the same protocol anyway.
     
  11. 1,219
    660
    222
    TnA

    TnA Senior Member

    Joined:
    Jul 1, 2018
    Messages:
    1,219
    Likes Received:
    660
    Trophy Points:
    222
    Gender:
    Male
    Location:
    Germany --> Saxony
    The biggest hurdle for the VMC-Emulation is, that the MCU (if we would use one) needs to be 'ready to answer' within ms...

    Well, atleast if someone wants to boot from it as well... (and then it would require MC-Auth-emulation as well)
     
  12. 736
    1,317
    222
    sp193

    sp193 Developer

    Joined:
    Oct 13, 2014
    Messages:
    736
    Likes Received:
    1,317
    Trophy Points:
    222
    Location:
    Singapore
    Home Page:
    What I remember is that the SIO2 interface can be used ot interface with microSD cards, according to @wisi. However, that will make it incompatible with normal PlayStation 2 memory cards. This is configurable via software.

    We can read the size of the NAND installed. It's how we can support cards larger than 8MB.

    But today, if we have a choice, we can use USB devices instead.
    If you're in favour of a software approach, nobody has taken a look at OPL's VMC functionality since 2010 or something lol

    This requires the instructions to be rewritten on-the-fly. So you will use a binary translator/dynamic recompiler.

    How do you know that?
     
  13. 53
    12
    37
    Anonamous

    Anonamous Member

    Joined:
    Aug 26, 2018
    Messages:
    53
    Likes Received:
    12
    Trophy Points:
    37
    Occupation:
    Electrical Engineering Student
    I was thinking to have a microcontroller provide an interface that would present itself as a normal 8Mb memory card for storing FreeMCBoot and the like and then having the rest of the space available by specific command issued by OPL, ule, freemcFree, etc. Would likely use an avr as they are cheap and easy to use. The main issue I see at the moment is clock speed and instruction time. It would come down to hold long of a delay the PS2 will accept before receiving a response.

    The other problem, or at least uncertainty is as follows. @uyjulian id you mean an SD card or an actual emmc chip? The emmc chips I found are mostly bga and have a capacity of 4Gb or larger. The main issue with this is it makes it difficult for prototyping and may make it more difficult to manufacture.

    In a nutshell it would be a device with two partitions. One of the standard mc format with 8Mb and than another with the rest if the space that can be utilized via the ps2sdk.

    Edit: After doing a bit more research, it may be more logical to use a nand flash chip instead and have a USB interface in the back to make it easier to put files on via computer.
     
    Last edited: Feb 4, 2019
  14. 159
    267
    122
    uyjulian

    uyjulian Developer

    Joined:
    May 27, 2017
    Messages:
    159
    Likes Received:
    267
    Trophy Points:
    122
    Gender:
    Male
    either SD card or eMMC works fine.

    The more components you put, the more the cost increases.
     
    Last edited: Feb 1, 2019
  15. 1,219
    660
    222
    TnA

    TnA Senior Member

    Joined:
    Jul 1, 2018
    Messages:
    1,219
    Likes Received:
    660
    Trophy Points:
    222
    Gender:
    Male
    Location:
    Germany --> Saxony
    Correct!

    Hm... I wonder if it could work on a Multitap, but (sadly) I suppose it can't... It would be great to have a device, where both could still be used.

    The implementation @wisi did and @Maximus32 wrote an SD-Driver for however is a DIRECT connection without any extras! No chips, no MCU, no nothing!

    Also no kinds of emulations (externally within the device instead of on/in the PS2) like for VMC, MagicGate, Multitap and etc. are possible.
    These ARE indeed possible with a bit more sophisticated device! The biggest issue I faced was even predicted by @wisi! How more sophisticated and complex the HW-structure becomes (like using a RasPi, or ESP) and without a MCU or controller-chip programmed for the task you will run into issues 'answering' fast enough BUT ONLY on power on! The devices need too much time to initially boot up, to emulate an MC (even when MG-emulation were supported) to boot FMCB.
    When it already is powered on (external power-supply or even when connected to the PS2's USB), it can work with a RasPi (GPIO) as well! So on a reset, FMCB actually could be loaded from it, like @Anonamous also had in mind.

    I ought to remember that the usefulness and size of those cards is still limited. Well, you probably know that. I am just mentioning it for those who might not know it.
    Here some limits which come to mind.
    1. Filesystem... Was it a (proprietory) variation of 'FAT12' or 'FAT16'?
    2. Some games just don't work with bigger cards (i.e. because of a too small value used for addressing the sectors, much like the 2TB-issue where it used 31Bit instead of 32Bit for addressing sectors. So... Possibly 16Bit sometimes signed sometimes unsigned to work with cards up to 16MB (15Bit) or 32MB (16Bit)?) or...
    3. some games work with certain cards up to the point where they are filled or 'behind'/after the normally addressable sectors!
    4. ...and possibly more...
    True, but here are some reasons why I agree that VMC-Emulation (but NOT limited to it, or it would be quite costly 'for the fun') via an external device is indeed an interesting and useful idea:
    1. No need to care about it being fragmented or not!
    2. The VMC's could be compressed (well maybe not so much of use nowadays given the amount of space available nowadays, but I thought I'd mention it anyway).
    3. No extra-code would run!
    4. The compatibility could be next to or even PERFECT as long as the emulation is done well!
    5. VMCs can then be used everywhere as they would seem to be normal (P)MCs to any program.

    Along these things, it is possible to use the same device to do what @pelvicthrustman created with his custom BT-Dongle for controllers (works for USB and BT as long as the device has such a connection or a connection where it can communicate following that standard).

    For soooo long? o_O
    Well... I think there is still a bit 'room for improvement', but the approach I (and probably @uyjulian and @Anonamous as well) meant is rather an external device which has the capability to emulate VMCs which are present as files on a storage-device/medium (SD-Card, USB-Stick, whatever) connected to that 'emulation-device'.


    That's only true for those which are not natively supported!
    @ffgriever made a 'write-up' (well... The info is rather scattered around in the PS2PSXe-Thread) about it a while ago!
    Apparently, you can execute atleast quite some of these without a recompilation/translation (to the target)!
    Some of these also need to be 'reworked', but not in the total sense of a recompilation, which are those which have an equivalent call (essentially doing the same work), but which uses another name!

    That's one way, but you also can strip down the recompiler to only recompile those where it is needed and 'pass' the others.

    The 'passing' for supported calls is not just limited to the CPU, but partially others HW-Parts as well thus not only limited to the recompiler... AAAAAND it even yields an interesting effect(!). It sometimes looks better on the emulator in certain games, even without additional filters(!) enabled!
    One great example is a kind of hardware-antialiasing-effect on some 3D-Meshes ('bodies') and on T&L (especially light-effects have less jagging, IMO), even tho' there is no code for Anti-aliasing running.
    @ffgriever would be a great resource to explain this!


    This method (recompiler) is thread-safe (as long as the recompiler itself isn't multi-threaded), but introduces quite some latencies, even on those which could be executed natively.
    ...and every interrupt causes the rest of the emulation to 'wait', if it were meant to be synced (quite often this is important for emulators and a reason, why they need sooo much processing-power, how more accurate they should emulate a platform)!


    An alternative to that is a thing I mentioned regarding N64-Emulation, because I came to the conclusion that only HLE (high level emulation) would be efficiently possible on a PS2 using that approach, while 'killing unnecessary structures' which cause latencies and interrupts, MIGHT make LLE of an N64 possible on a PS2.


    I had various ideas regarding this.
    One which is a rather easy implementation is, to split the recompiler into an input-layer which tries to natively execute some stuff directly and the other things are sent to a recompiler...

    The biggest problem here... You still need to catch (which means they are technically buffered on the RAM, checked by the 'mask' and then either send via a task to the instruction-cache (Warning: There you also have to make sure that cache-coherence isn't affected, or the entire console crashes.) or if it is not natively supported through the Recompiler!

    I thought of another approach as well, but I am not sure if the PS2 supports something like 'sigill'! If it does, it is even possible to just send all commands 'blindly' to the CPU (but AFAIK only cached and via a thread/task and not via directly moving it to the instruction-cache) and if an unsupported instruction is detected, this causes the 'mask' to send it through the Recompiler...

    Both methods have their advantages and disadvantages... and I have no clue, how it is possible to combine the best of both...

    The sigill-method would require it to be at least single- or even double-cached. It is killing some latencies and interrupts as well, essentially making it faster (but...)
    The other method could be done via one less caching, thus it has a bit less interrupts there...

    Well... That all might be a bit off topic...


    Either @wisi or @Maximus32 (or did someone else tested the 'SD-Gecko'?) mentioned that in either the SD-SIO2-Thread or in the speed-testing-thread or somewhere else!
    I am quite sure it was on psx-scene.com ~ beginning of 2018 (or so?!).

    Let's hope one of them chimes in and can confirm the speed!

    Yes, that IS possible! However, it either needs a specially/specifically programmed MCU or you would need to start the device somehow before the PS2 starts, because every device (even an ESP32 or so) needs a few seconds to boot up (thus they only work properly on rebooting the PS2 or booting the device beforehand).

    Yes, that's also possible but needs additional support for MC-Auth and so on...

    However... There is yet another issue, which I don't know how to get along with, because the Hardware-ID of the MC is not the same like the MCID/MC-KELF- Signing-ID and we can't read the true HW-ID... Thus, the emulation is currently bound to fail, except if we can re-calculate the Hardware-ID from multiple MC-KELFs bound to the same MC... Btw.: IMO that's also possible, maybe in a similar way how Doormate does it... However... @krHACKen knows why there is this 20MB-file and essentially this would need another one of these and would need a cross-verification-algo which means for every single key checked, the time to calculate it would grow in square (again PER KEY), essentially leading to quite some time to calculate it...

    VMC-Emulation itself? VERY possible!

    MG-Emulation (including decryption and possibly encryption)? Hard!
    Fortunately, it would be possible to change the Hardware-ID of the emulation-device, thus this would be a great tool to decrypt some stuff like MC-KELFs bound to a specific MemoryCard. We could emulate i.e. a Memor32 with it (and Memento's 'protection' wouldn't notice the difference), lol...

    That's a possibility but would require specific software-support for it...
    This is also possible, if you have some kind of 'interaction' available on the device itself (Buttons, or whatever)!
    Having both would be neat (and possible), tho'!

    Yes, that could possibly be used.

    I am using these for my tests:
    • RasPi
    • SHA 2017+ (ESP32-Badge with a small ink-display)
    • An 'El-Cheapo' (not its actual name. . Just a cheap adapter) USB-TT(Y)L/UART-Serial-adapter


    Exactly! + the initial bootup-time of the device!

    Yerp... More info on that on the SD-SIO2-Thread and on the BT-Adapter-Thread!

    http://psx-scene.com/forums/f19/sd-card-sio2-driver-157485/

    http://psx-scene.com/forums/f19/how...3-ps4-controllers-wirelessly-your-ps2-127728/


    To be honest, that's one of the main-reasons why my idea doesn't use a big internal memory, but only a small MCU (to circumventtthe need for pre-booting the device or rebooting the PS2) and a bit memory (for a 'firmware') and the rest is provided via external means, being it an SD-Card or a USB-(Storage)Device!

    I ought to say that IMO it would be cheaper and better to split it into 2 physical storage-devices... One small internal Memory and one for the files which are meant to be stored and/or accessed (i.e. VMC)!
    The smaller memory could also hold the KELF for a fast delivery, once MG-Emulation would work!

    The overall ideas are good IMO, but I have to point out that pretty much everything of this could also be done with external storage-device ls which could possibly be plugged into the device, thus neither limiting the size of the storage, nor would it cost so much in production (it even gets down to the point, where some people could build their own) AND if the adapter supports a kind of pass-through-mode, it could access the SD-Card via wisi's method still, making it perfect to play from it (via OPL)!

    I would rather make the storage removable from the device!!!

    Correct!


    Last but not least: I certainly wrote about this idea on psx-scene.com or so (~3/4 or 1 year ago)! I think it was in the SD SIO2-Thread and maybe even a seperate thread as well and yes I am still working on it once in a while but am just too busy (sadly).

    Where is the living search-engine when you need them? :D @jolek (who curiously finds almost all/everything, lol)


    Edit: Ahaaaaa! Found it!
    http://psx-scene.com/forums/f19/sd-card-ps2-memorycard-adapter-implementation-157491/

    Please read it! It's only 1 page and holds some valuable information, like the following link!

    http://psx-scene.com/forums/f19/ps2-memory-card-infos-spi-bus-commands-format-120052/
     
    Last edited: Feb 3, 2019
    Anonamous likes this.
  16. 53
    12
    37
    Anonamous

    Anonamous Member

    Joined:
    Aug 26, 2018
    Messages:
    53
    Likes Received:
    12
    Trophy Points:
    37
    Occupation:
    Electrical Engineering Student
    So @TnA I don't want to quote that monstrosity but I read a bit of that link. As far as sending the acknowledge bits, depending on what it's actually looking for it can be faked until the mcu is finished setting up. I'd have to look at more details but you would essentially send the acknowledge by hardware and then have the mcu set a gpio pin to disable the hardware completing the acknowledge.

    I have on hand a raspberry pi zero, Arduino uno and duo as well as a handful of atmega 328 chips on hand if you need some testing done. Unfortunately I can't get caught with a soldering iron in my dorm and I'm swamped this semester my time is super limited. This has however landed on my spring break Todo list if you don't mind the assistance.

    There is a chip on the magicgate itself that appears to be an mcu. If no one has any information on it I can take to poking at it.
     
    Last edited: Feb 3, 2019
    TnA likes this.
  17. 159
    267
    122
    uyjulian

    uyjulian Developer

    Joined:
    May 27, 2017
    Messages:
    159
    Likes Received:
    267
    Trophy Points:
    122
    Gender:
    Male
    I might have found a way to import symbols from the STABS section into a certain disassembler.
    There is a way to dump the database of the disassembler to script files. While doing that with a file with DWARF symbols, I found out that functions are called to insert function signatures, stack variables, types, etc. with their associated addresses. objdump -g also dumps the same things but in a different layout. I think it would be easy to convert the output by objdump -g to that scripting format. It may also be possible to just interact with libbfd directly to reformat the output to be compatible with the scripting format.
     
    Last edited: Feb 3, 2019
    TnA and Anonamous like this.
  18. 159
    267
    122
    uyjulian

    uyjulian Developer

    Joined:
    May 27, 2017
    Messages:
    159
    Likes Received:
    267
    Trophy Points:
    122
    Gender:
    Male
    It's been a while since I've started to work on stuff for long periods of time while sitting down. A walking desk might be a good solution, and it might be better than sitting down for long periods of time.

    It looks like the new llvmir2hll converter is merged into RetDec. I should try it out again eventually.

    I have a modified version of irxtool which can generate rename files for RetDec generic function names into function names from ps2sdk. I don't think that there is an easy way to change argument and stack types in RetDec. Also, it now dumps all import/export tables from modules like sio2man properly. I'll push it up eventually.

    I've submitted the mcman and padman header fix upstream finally. I didn't mean for the submission to be delayed for this long, but oh well.

    I touched a bit on the new toolchain and related tools. gsKit works. fileXioInit hangs for some reason on PCSX2. I haven't tried on real hardware yet because I still need to get dsnet reversing done.

    I haven't tried importing STABS symbols into that certain disassembler yet. I still need to write the script to read the addresses and generate the appropriate script.

    I haven't touched pfsfuse for a while. I still need to do more research on the FUSE API. pfsshell works for me for the time being, but I will need to finish writing pfsfuse since editing and pasting commands becomes time consuming.

    I can do a lot of things in a large contiguous amount of time, but not so much when that time is split up into multiple chunks.
     
    TnA likes this.
  19. 159
    267
    122
    uyjulian

    uyjulian Developer

    Joined:
    May 27, 2017
    Messages:
    159
    Likes Received:
    267
    Trophy Points:
    122
    Gender:
    Male
    Let's talk about the PS2 homebrew stack in general.

    We have the following:
    ps2sdk-libc
    patched gdb
    patched gcc 3.2.3
    custom kernel
    partial C++ support
    ERL
    patched newlib 1.10.0
    iomanX
    bare-metal ELF
    GSKit

    It is harder and harder to get programs from newer and more stabilized standards to work on the PS2. No C++17, no pthreads or std::thread, no virtual memory, no RAM disk, no C++ streams, can't run multiple programs at the same time...
    I'd like to port my version of KiriKiri Z to PS2, but since the standardized library is not updated with the latest standards, that is unlikely to happen with the current PS2SDK. There are some RetroArch cores that are difficult to port due to their usage of new standards.
    The devkitPro guys are doing a great job with the Gamecube, Wii, 3DS, and Switch.
    Daedalus has seen performance improvements when it has been using a PSP toolchain ported to GCC 8.

    My ideal setup:
    musl-libc or ulibc
    latest gdb
    latest gcc
    linux kernel
    full C++ support
    Linux ELF shared libraries
    musl-libc or ulibc
    minimal block device drivers on IOP that can be accessed from Linux software
    single-binary IOP software
    Legacy OpenGL for cross platform software
    GSKit port to Linux
    FUSE for accessing pfs/mcfs
    Linux ELF

    The old things can be done with PS2SDK, but the new things can be done with a modern Linux-based stack.

    I can't do much in the next 3 months. In the meantime, I'll attempt to port SDL2 and see how much of my port of KiriKiri Z I can get compiling.
     
    jolek, Anonamous and TnA like this.
  20. 159
    267
    122
    uyjulian

    uyjulian Developer

    Joined:
    May 27, 2017
    Messages:
    159
    Likes Received:
    267
    Trophy Points:
    122
    Gender:
    Male
    TnA and jolek like this.

Share This Page