Ps3 Memory leaks... (Developing OpenBOR port for PS3)

Discussion in 'Homebrew Development' started by White Dragon, Sep 1, 2018.

  1. 735
    849
    147
    Zar

    Zar Developer

    Joined:
    Oct 15, 2014
    Messages:
    735
    Likes Received:
    849
    Trophy Points:
    147
    I don't know.

    So, you find a way to avoid the crash ?
     
  2. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    The crash seems for exceeded ram usage. So nope...
    I don't know if ps3 has some protection or limit for ram usage or recursion limit.
    Maybe with sce-flag?
     
  3. 735
    849
    147
    Zar

    Zar Developer

    Joined:
    Oct 15, 2014
    Messages:
    735
    Likes Received:
    849
    Trophy Points:
    147
    I did a sample, to test if it crashs if it malloc a too big size of data :

    Code:
    typedef struct {
        uint32_t total;
        uint32_t avail;
    } sysMemoryInfo_t;
    
    s32 sysMemoryGetUserMemorySize(sysMemoryInfo_t * MemInfo)
    {
        lv2syscall1(SYSCALL_MEMORY_GET_USER_MEMORY_SIZE, (uint64_t) MemInfo);
        return_to_user_prog(s32);
    }
    
    int main()
    {
        FILE *fp;
        char str[128];
        u64 size = 0;
        sysMemoryInfo_t MemInfo;
        u8 *test;
       
        fp = fopen("/dev_hdd0/tmp/log.txt", "w");
        if(fp==NULL) return 0;
       
        while(1)
        {
            size += 0x100000;
            memset(&MemInfo, 0, sizeof(sysMemoryInfo_t));
            sysMemoryGetUserMemorySize(&MemInfo);
            sprintf(str, "MEM : %X / %X\n", (unsigned int) MemInfo.avail, (unsigned int) MemInfo.total); fputs(str, fp);
            sprintf(str, "Malloc : %lX", (unsigned long int) size); fputs(str, fp); usleep(100);
            test = (u8 *) malloc(size);
            if(test==NULL) {
                strcpy(str, " FAILED \n"); fputs(str, fp);
                sysMemoryGetUserMemorySize(&MemInfo);
                sprintf(str, "NEW MEM : %X / %X\n", (unsigned int) MemInfo.avail, (unsigned int) MemInfo.total); fputs(str, fp); usleep(100);
                fclose(fp);
                return 0;
            } else {
                strcpy(str, " OK \n"); fputs(str, fp);
                sysMemoryGetUserMemorySize(&MemInfo);
                sprintf(str, "NEW MEM : %X / %X\n", (unsigned int) MemInfo.avail, (unsigned int) MemInfo.total); fputs(str, fp); usleep(100);
            }
           
            free(test);
        }
    }
    and it works fine... here the last loops logs :

    Code:
    MEM : EAF0000 / EC00000
    Malloc : EA00000 OK
    NEW MEM : F0000 / EC00000
    MEM : EAF0000 / EC00000
    Malloc : EB00000 FAILED
    NEW MEM : EAF0000 / EC00000
    I don't get it ><
     
    STLcardsWS likes this.
  4. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    Mmm... Big thanks for your test. I changed the script library to use a char * instead of fixed length arrays.
    Well ram usage is more large : about 180mb against ~130mb. Used a linked list to memorize into stack the potential malloc memory.
    I don't know what is the issue...
    The ps3 emulator works well until 200mb of ram... Now I think the recursion is the problem. Mmm..
     
    Last edited: Sep 7, 2018
    STLcardsWS likes this.
  5. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    Rewritte my dmalloc library and 0 memory leak found!!
    So it works well..
    Just I need if RAM is full for a last attempt...
     
  6. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    I can confirm.
    The ram is ok
    AVAIL / TOTAL
    204152832 / 223346688
     
    Last edited: Sep 13, 2018
  7. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    Hi to all, I'm studying all parser/lexer code.
    I found that used ram is just 28MB when the ps3 freezes.
    In this way I can't debug.
    How can I debug it?
    The strange thing is taht on ps3 emulator it works well, but on a real ps3 it freezes.
    Please help...
    I suspect an illegal memory access.
    into preprocessorlib folder and into pp_lexer.c there is the function pp_lexer_GetNextToken().
    The app crashes before re enter in this func.
    Can you help me?
     
  8. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    I can confirm that into preprocessorlib/pp_parser.c
    there is the function HRESULT pp_parser_lex_token(pp_parser *self, bool skip_whitespace)
    after some loops the instruction step is freezed before
    success = (int)SUCCEEDED(pp_lexer_GetNextToken(&self->lexer, &self->token));
    and never enters into pp_lexer_GetNextToken()

    I searched for a memory violation into self->lexer, self->token, self addresses but it seems all ok...
    why?
    Is it right if I think that there is some recursion limit or... some root limits?
     
  9. 7,474
    5,534
    847
    kozarovv

    kozarovv Super Moderator

    Joined:
    Nov 8, 2014
    Messages:
    7,474
    Likes Received:
    5,534
    Trophy Points:
    847
    Home Page:
  10. 735
    849
    147
    Zar

    Zar Developer

    Joined:
    Oct 15, 2014
    Messages:
    735
    Likes Received:
    849
    Trophy Points:
    147
  11. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    Thanks for your answer.
    I defined my custom stack resize directive

    #define SYS_PROCESS_PARAM_OPENBOR(prio,stacksize) \
    sys_process_param_t __sys_process_param __attribute__((aligned(8), section(".sys_proc_param"), unused)) = { \
    sizeof(sys_process_param_t), \
    SYS_PROCESS_SPAWN_MAGIC, \
    SYS_PROCESS_SPAWN_VERSION_330, \
    SYS_PROCESS_SPAWN_FW_VERSION_330, \
    prio, \
    stacksize, \
    SYS_PROCESS_SPAWN_MALLOC_PAGE_SIZE_1M, \
    SYS_PROCESS_SPAWN_PPC_SEG_DEFAULT\
    };
    SYS_PROCESS_PARAM_OPENBOR(1001, 0x1F000000)


    But no luck..
    I don't know if I wrote it well...
    Thanks! I tested it but I think that for ps3 I need some other func.. That is for Linux.
    Maybe the setrlimt? If I can set it on ps3..


    https://stackoverflow.com/questions...trlimit-and-provoke-a-stack-overflow-segfault
     
  12. 735
    849
    147
    Zar

    Zar Developer

    Joined:
    Oct 15, 2014
    Messages:
    735
    Likes Received:
    849
    Trophy Points:
    147
    the stack size MAX is 0x100000=1MB and the MIN is 0x1000=4KB.

    Too bad it didn't work, it was just a "random" idea, I didn't understand how it is used for.

    Sry, I couldn't help you on this one...

    did you try this to find the recursion limit ?
    https://rosettacode.org/wiki/Find_limit_of_recursion#C
     
    Last edited: Sep 14, 2018
  13. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    thanks, I can test recursivity with rosetta code but I cant' compare it with app.
    I don't know what I can do...
     
  14. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    EUREKAAAAA!!
    I substiture all infinite loops with a for with limited loop count definited into a macro + SYS_PROCESS_PARAM(1001, SYS_PROCESS_SPAWN_MALLOC_PAGE_SIZE_1M)!!
    Not other configs!
    Now it works perfectly!!
    many nested while(1) into recursive functions are not accepted by PS3 architecture (or PPU??).
    Maybe because the processor can't calculate all potential depth...
    What do you think about it?
    Now I'm ready to release it!
    Have you a more elegant method to substitute while(1)?
    Thanks!
     
    Last edited: Sep 15, 2018
    Zazenora, sandungas, DeViL303 and 4 others like this.
  15. 735
    849
    147
    Zar

    Zar Developer

    Joined:
    Oct 15, 2014
    Messages:
    735
    Likes Received:
    849
    Trophy Points:
    147
    I'm glad you found a solution to ur issue. My suggestion ( 'while(1)' ) was a making it worse, :sem blush: sry.

    if it's working, just don't change anything ;)

    I don't think the spawn_malloc_page value is supposed to be used here. its value is 0x100000
     
    DeViL303 likes this.
  16. 1,070
    1,070
    272
    littlebalup

    littlebalup Developer PSX-Place Supporter

    Joined:
    Oct 16, 2014
    Messages:
    1,070
    Likes Received:
    1,070
    Trophy Points:
    272
    Location:
    43°36'16.0"N 1°26'36.1"E
    To the staff: please give him then dev status ;)
     
    Zazenora, sandungas, Zar and 2 others like this.
  17. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    0x100000 is 1Mb so it's right.
    Better write 0x100000 constant directly maybe.
     
  18. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    littlebalup and STLcardsWS like this.
  19. 7,474
    5,534
    847
    kozarovv

    kozarovv Super Moderator

    Joined:
    Nov 8, 2014
    Messages:
    7,474
    Likes Received:
    5,534
    Trophy Points:
    847
    Home Page:
  20. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    Last edited: Sep 16, 2018

Share This Page