WIP updated homebrew toolchain for PS2

Discussion in 'General PS2 Discussion' started by uyjulian, Aug 25, 2018.

  1. 244
    439
    122
    uyjulian

    uyjulian Developer

    Joined:
    May 27, 2017
    Messages:
    244
    Likes Received:
    439
    Trophy Points:
    122
    Gender:
    Male
    I updated the OP with instructions on how to use the new toolchain that Maximus32 is working on.
     
    STLcardsWS, jolek, Maximus32 and 3 others like this.
  2. 67
    20
    37
    Anonamous

    Anonamous Member

    Joined:
    Aug 26, 2018
    Messages:
    67
    Likes Received:
    20
    Trophy Points:
    37
    Occupation:
    Electrical Engineering Student
    Is it supposed to take longer to sync everything than it does to build the toolchain?
     
  3. 244
    439
    122
    uyjulian

    uyjulian Developer

    Joined:
    May 27, 2017
    Messages:
    244
    Likes Received:
    439
    Trophy Points:
    122
    Gender:
    Male
    Depends on your internet connection and your hardware. GCC and Binutils repos are large, so it may take a while to pull and checkout everything.
     
  4. 67
    20
    37
    Anonamous

    Anonamous Member

    Joined:
    Aug 26, 2018
    Messages:
    67
    Likes Received:
    20
    Trophy Points:
    37
    Occupation:
    Electrical Engineering Student
    The problem is probably gitlab. They likely limit connection speed as it topped out at around 6 Mbps. I usually get around 50-100Mbps. It literally took at least a half hour.
     
  5. 244
    439
    122
    uyjulian

    uyjulian Developer

    Joined:
    May 27, 2017
    Messages:
    244
    Likes Received:
    439
    Trophy Points:
    122
    Gender:
    Male
    Yeah GitLab SaaS performance is poor compared to self-hosting.
     
  6. 90
    313
    82
    Maximus32

    Maximus32 Developer

    Joined:
    Sep 10, 2019
    Messages:
    90
    Likes Received:
    313
    Trophy Points:
    82
    Gender:
    Male
    Thanks!

    The first time you sync you're actually downloading the entire (git) history of all of ps2dev, gcc (huge!), binutils, etc, etc... so yes, it takes a very long time. If you just want to 'download' you can do the following:
    Code:
    repo init -u https://gitlab.com/ps2max/ps2dev-repo.git -b ee-toolchain-gcc9 --depth=1
    repo sync -j12 -c --no-tags
    You will not get the git history, and no other branches. But it's A LOT faster.

    The next time you sync it will be a matter of seconds. But please note that I rebase a lot onto upstream branches. So you might get merge conflicts even if you haven't changed anything. Usefull repo commands to get an overview of all git repositories:
    Code:
    # create branch in all git repositories
    repo start test-branch --all
    
    # list branches in all git repositories
    repo branch
    
    # list all commits you've added
    repo info -o
    
    # like git status, but for all git repositories
    repo status
     
    Chris_spxl, Anonamous, TnA and 2 others like this.
  7. 67
    20
    37
    Anonamous

    Anonamous Member

    Joined:
    Aug 26, 2018
    Messages:
    67
    Likes Received:
    20
    Trophy Points:
    37
    Occupation:
    Electrical Engineering Student
    This is what the decompiled C work around looks like. Gives a hint to the potential problem.
    Code:
    void VU0MixVec(VU_VECTOR *a,VU_VECTOR *b,float t,VU_VECTOR *res)
    {
      DFtype DVar1;
      FLO_type.conflict FVar2;
      FLO_type.conflict FVar3;
      SFtype *in_a3_lo;
      SFtype SVar4;
      SFtype SVar5;
      SFtype SVar6;
      SFtype in_f14;
     
      DVar1 = __extendsfdf2(in_f14);
      FVar2 = __subdf3(1.00000000,(FLO_type.conflict)DVar1);
      SVar4 = __truncdfsf2((DFtype)FVar2);
      DVar1 = __extendsfdf2(SVar4);
      FVar2 = __subdf3(1.00000000,(FLO_type.conflict)DVar1);
      DVar1 = __extendsfdf2((SFtype)a->x);
      FVar3 = __muldf3((FLO_type.conflict)DVar1,FVar2);
      DVar1 = __extendsfdf2((SFtype)((float)SVar4 * b->x));
      FVar3 = __adddf3(FVar3,(FLO_type.conflict)DVar1);
      SVar5 = __truncdfsf2((DFtype)FVar3);
      SVar6 = (SFtype)a->y;
      *in_a3_lo = SVar5;
      DVar1 = __extendsfdf2(SVar6);
      FVar3 = __muldf3((FLO_type.conflict)DVar1,FVar2);
      DVar1 = __extendsfdf2((SFtype)((float)SVar4 * b->y));
      FVar3 = __adddf3(FVar3,(FLO_type.conflict)DVar1);
      SVar5 = __truncdfsf2((DFtype)FVar3);
      SVar6 = (SFtype)a->z;
      in_a3_lo[1] = SVar5;
      DVar1 = __extendsfdf2(SVar6);
      FVar3 = __muldf3((FLO_type.conflict)DVar1,FVar2);
      DVar1 = __extendsfdf2((SFtype)((float)SVar4 * b->z));
      FVar3 = __adddf3(FVar3,(FLO_type.conflict)DVar1);
      SVar5 = __truncdfsf2((DFtype)FVar3);
      SVar6 = (SFtype)a->w;
      in_a3_lo[2] = SVar5;
      DVar1 = __extendsfdf2(SVar6);
      FVar2 = __muldf3((FLO_type.conflict)DVar1,FVar2);
      DVar1 = __extendsfdf2((SFtype)((float)SVar4 * b->w));
      FVar2 = __adddf3(FVar2,(FLO_type.conflict)DVar1);
      SVar4 = __truncdfsf2((DFtype)FVar2);
      in_a3_lo[3] = SVar4;
      return;
    }
    Here it is on the new toolchain

    Code:
    void VU0MixVec(VU_VECTOR *a,VU_VECTOR *b,float t,VU_VECTOR *res)
    {
      float fVar1;
      float *in_a3_lo;
      float y1;
      float z1;
      float w1;
      float y2;
      float z2;
      float w2;
      float local_10;
     
      y1 = a->y;
      z1 = a->z;
      w1 = a->w;
      y2 = b->y;
      z2 = b->z;
      w2 = b->w;
      fVar1 = 1.00000000 - local_10;
      *in_a3_lo = a->x * local_10 + b->x * fVar1;
      in_a3_lo[1] = y1 * local_10 + y2 * fVar1;
      in_a3_lo[2] = z1 * local_10 + z2 * fVar1;
      in_a3_lo[3] = w1 * local_10 + w2 * fVar1;
      return;
    }
    
    Here lies the problem. This is the disassembly:
    Code:
    VU0MixVec:
        addiu      sp,sp,-0x10
        lqc2       y1,a->x(a0)
        lqc2       y2,b->x(a1)
        lw         v0,0x0(sp)=>local_10
        mtc2.I     v0,vf3
        vaddw.x    vf5,vf0,vf0w
        vsub.x     vf4,vf5,vf3
        vmulax.x   ACC,y1,vf3x
        vmaddx.x   y1,y2,vf4x
        sqc2       y1,0x0(a3)
        jr         ra
        _addiu     sp,sp,0x10
    
    It appears gcc is using a3 for res when it is supposed to be a2
     
    Last edited: May 21, 2020
    jolek, TnA and uyjulian like this.
  8. 67
    20
    37
    Anonamous

    Anonamous Member

    Joined:
    Aug 26, 2018
    Messages:
    67
    Likes Received:
    20
    Trophy Points:
    37
    Occupation:
    Electrical Engineering Student
    So it turns out I was mistaken. The calling convention being used is n32 but I thought it was o32. The parameters are being passed in the correct registers.

    This is the disassembly using the old gcc 3 compiler:
    Code:
    VU0MixVec:
        addiu      sp,sp,-0x10
        swc1       t,0x0(sp)
        lqc2       y1,a->x(a0)
        lqc2       y2,b->x(a1)
        lw         v0,0x0(sp)
        qmtc2.I    v0,vf3
        vaddw.x    vf5,vf0,vf0w
        vsub.x     vf4,vf5,vf3
        vmulax.x   ACC,y1,vf3x
        vmaddx.x   y1,y2,vf4x
        sqc2       y1,res->x(a2)
        jr         ra
        _addiu     sp,sp,0x10
    
    This is the disassembly using the new toolchain:
    Code:
    VU0MixVec:
        addiu      sp,sp,-0x10
        lqc2       vf1,a->x(a0)
        lqc2       vf2,b->x(a1)
        lw         v0,0x0(sp)
        qmtc2.I    v0,vf3
        vaddw.x    vf5,vf0,vf0w
        vsub.x     vf4,vf5,vf3
        vmulax.x   ACC,vf1,vf3x
        vmaddx.x   vf1,vf2,vf4x
        sqc2       vf1,res->x(a3)
        jr         ra
        _addiu     sp,sp,0x10
    
    Note the missing
    Code:
    swc1       t,0x0(sp)
    .
     
    uyjulian and TnA like this.
  9. 5
    3
    7
    AlexTrashDog

    AlexTrashDog Forum Noob

    Joined:
    May 25, 2020
    Messages:
    5
    Likes Received:
    3
    Trophy Points:
    7
    I've been trying to build the GCC 9.x version of the toolchain and keep running into build errors. Specifically:

    Code:
    repo init -u https://gitlab.com/ps2max/ps2dev-repo.git -b ee-toolchain-gcc9 --depth=1 && \
    repo sync -j8 -c --no-tags
    /bin/bash -c "source ./envsetup.sh && ./build-all.sh"
    
    Fails with the following error.

    Code:
     make[2]: Entering directory '/opt/toolchain/build/binutils-iop/opcodes'
    Making all in po
    make[3]: Entering directory '/opt/toolchain/build/binutils-iop/opcodes/po'
    file=../../../../iop/binutils/opcodes/po/`echo ro | sed 's,.*/,,'`.gmo \
      && rm -f $file && PATH=../src:$PATH no -o $file ../../../../iop/binutils/opcodes/po/ro.po
    /bin/sh: 2: no: not found
    make[3]: *** [ro.gmo] Error 127
    Curious if anyone else is seeing this error with commit c2b05c86.

    Here's the complete Dockerfile I'm using for reference.
     
    TnA likes this.
  10. 90
    313
    82
    Maximus32

    Maximus32 Developer

    Joined:
    Sep 10, 2019
    Messages:
    90
    Likes Received:
    313
    Trophy Points:
    82
    Gender:
    Male
    It fails compiling the old IOP binutils (only the EE toolchain is updated).
    So you're probably missing some packages, like gettext, bison, flex, texinfo, etc...
     
    AlexTrashDog and TnA like this.
  11. 5
    3
    7
    AlexTrashDog

    AlexTrashDog Forum Noob

    Joined:
    May 25, 2020
    Messages:
    5
    Likes Received:
    3
    Trophy Points:
    7
    Makes sense. Is there a complete list of dependencies I should be aware of? I'm aiming for a totally reproducible build.
     
  12. 90
    313
    82
    Maximus32

    Maximus32 Developer

    Joined:
    Sep 10, 2019
    Messages:
    90
    Likes Received:
    313
    Trophy Points:
    82
    Gender:
    Male
    My Dockerfile seems to be working now (untested):
    https://gitlab.com/ps2max/ps2dev-scripts/-/blob/ee-toolchain-gcc9/Dockerfile

    The docker image is here:
    registry.gitlab.com/ps2max/ps2dev-scripts:ee-toolchain-gcc9

    Note that it took 85minutes to build (on gitlab CI), and is 428MB in size. That's a big difference from the old toolchain, that takes around 20 minutes to compile, and is around 50MB in size.
     
    AlexTrashDog and TnA like this.
  13. 1,514
    857
    222
    TnA

    TnA Senior Member

    Joined:
    Jul 1, 2018
    Messages:
    1,514
    Likes Received:
    857
    Trophy Points:
    222
    Gender:
    Male
    Location:
    Germany --> Saxony
    Ufff... That's quite "a bit"! o_O

    Btw.: Just making sure it is you, who joined the Discord (?!?).
     
  14. 5
    3
    7
    AlexTrashDog

    AlexTrashDog Forum Noob

    Joined:
    May 25, 2020
    Messages:
    5
    Likes Received:
    3
    Trophy Points:
    7
    If this question is directed at me, yea I'm the same person who joined the Discord.
     
    TnA likes this.
  15. 5
    3
    7
    AlexTrashDog

    AlexTrashDog Forum Noob

    Joined:
    May 25, 2020
    Messages:
    5
    Likes Received:
    3
    Trophy Points:
    7
  16. 5
    3
    7
    AlexTrashDog

    AlexTrashDog Forum Noob

    Joined:
    May 25, 2020
    Messages:
    5
    Likes Received:
    3
    Trophy Points:
    7
    TnA likes this.
  17. 1,514
    857
    222
    TnA

    TnA Senior Member

    Joined:
    Jul 1, 2018
    Messages:
    1,514
    Likes Received:
    857
    Trophy Points:
    222
    Gender:
    Male
    Location:
    Germany --> Saxony
    It was directed at @Maximus32.

    I just made sure it is him, because he wrote that he doesn't have/use Discord. ;)


    @AlexTrashDog: Glad you are interested in the PS2 or Homebrewing in general!
     
  18. 67
    20
    37
    Anonamous

    Anonamous Member

    Joined:
    Aug 26, 2018
    Messages:
    67
    Likes Received:
    20
    Trophy Points:
    37
    Occupation:
    Electrical Engineering Student
    Is there any other code known to cause problems when built with the new toolchain atm?
     
  19. 90
    313
    82
    Maximus32

    Maximus32 Developer

    Joined:
    Sep 10, 2019
    Messages:
    90
    Likes Received:
    313
    Trophy Points:
    82
    Gender:
    Male
    Not exactly like this problem. But there's still lots of problems to fix. To name a few:
    - opl compiles and runs, but games do not run
    - ps2link does not compile
    - thousands of new compile warnings in all ps2 projects, some of them possibly causing the above problems.

    The problems are mostly related to ps2 specific things in the old ps2 codebase, like:
    - assembly code
    - ABI change in assembly code (eabi -> n32)
    - Wrong cache usage (UCAB, flushing, etc)

    Newer projects like RetroArch, with code that's already proven on newer gcc version, have little to no problems with the new toolchain.
     
    uyjulian, TnA and Algol like this.
  20. 67
    20
    37
    Anonamous

    Anonamous Member

    Joined:
    Aug 26, 2018
    Messages:
    67
    Likes Received:
    20
    Trophy Points:
    37
    Occupation:
    Electrical Engineering Student
    I can take a look at all of these except maybe the wrong cache usage. My understanding of caching is still minimal at best but I'm learning.

    The asm code will probably be my first target. The warnings that I've seen are mostly casting/type issues and signed/unsigned usage when the other is expected. They should absolutely be fixed though. I'm a -Werror kind of person though.
     
    TnA and ted209 like this.

Share This Page