Ps3 Memory leaks... (Developing OpenBOR port for PS3)

Discussion in 'Homebrew Development' started by White Dragon, Sep 1, 2018.

  1. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    ----------------------------------------------------------------------
    STAFF EDIT: Project Released: Link (first version)
    ------------------------------------------------------------------------
    Hi to all!
    I'm developing OpenBOR port for PS3.
    Here you can find a working source.

    https://github.com/whitedragon0000/OpenBOR_PLUS

    Some games work well, but other not load (app freeze) for some memory leaks...

    To test:
    in this source file:
    https://github.com/whitedragon0000/OpenBOR_PLUS/blob/master/engine/source/preprocessorlib/pp_lexer.c
    into HRESULT pp_lexer_GetNextToken (pp_lexer *plexer, pp_token *theNextToken) function
    when I put a log with printf(...) before for(;;) it prints but in the next line doesn't print.
    Example:
    Code:
    HRESULT pp_lexer_GetNextToken (pp_lexer *plexer, pp_token *theNextToken)
    {
    printf("I'm Here 1"); //IT PRINTS!
        for(;;)
        {
    printf("I'm Here 2"); //IT DOESEN'T PRINT!
            plexer->theTokenSource[0] = plexer->theTokenLen = 0;
            plexer->theTokenPosition = plexer->theTextPosition;
    plexer->tokOffset = plexer->offset;
    
    It seems a memory leak..

    To test it build the project decompress all project into a dir.
    Decompress ps3-sdk
    run build.bat or create a new bat with:
    Code:
    @setlocal
    @echo off
    set BUILDBATCH=1
    set TOOLS=../tools/bin;../tools/7-Zip;../tools/svn/bin
    set PATH=%TOOLS%
    bash.exe build.sh 11
    @endlocal
    
    @setlocal
    @echo off
    
    set TOOLS=../tools/bin;../tools/7-Zip;../tools/svn/bin
    set PATH=%TOOLS%;%PATH%
    call xbox/make.bat
    bash.exe version.sh 1
    :missing
    @endlocal
    
    pause
    into engine folder.

    Games with many scripts inside don't run...
    Test my game for example:
    https://gamejolt.com/games/teenage-mutant-ninja-turtles-shell-shocked-the-arcade-game/35259

    In windows it works well and... into PS3 emulator it works well!!
    But into a real PS3 I think.. there is a memory leak.. (I thinks...)

    Any help?
    Maybe there is a memory protection on PS3? Can I disable it?
    For info my make file is:
    Code:
    ifdef BUILD_PS3
    PKG_NAME = $(VERSION_NAME).pkg
    INCLUDE = $(foreach dir,$(INCLUDES), -I$(dir)) \
              $(foreach dir,$(LIBDIRS),-I$(dir)/include) \
              $(LIBPSL1GHT_INC) \
              $(foreach dir,$(INCS), -I$(dir))
    include $(PSL1GHT)/ppu_rules
    SFOXML=$(CURDIR)/resources/sfo.xml
    ICON0=$(CURDIR)/resources/ICON0.PNG
    SCETOOL_FLAGS += --self-ctrl-flags 4000000000000000000000000000000000000000000000000000000000000002
    SCETOOL_FLAGS += --self-cap-flags 00000000000000000000000000000000000000000000007B0000000100000000
    all : $(TARGET) $(TARGET_FINAL)
        @echo
    %.o : %.c
        @echo Compiling $(TARGET_PLATFORM) Port: $<...
        @#$(CC) $(CFLAGS) -c $< -o [email protected]
        $(VERB) $(CC) $(DEPSOPT) $(CFLAGS) -c $< -o [email protected] $(ERROR_FILTER)
    $(TARGET) : $(OBJS) $(RES)
        @echo
        @echo Linking $(TARGET_PLATFORM) Port: $(TARGET)...
        @#$(CC) $(CFLAGS) -o $(TARGET) $(OBJS) $(TARGET_RESOURCE) $(LIBS)
        $(VERB) $(PREFIX)gcc $^ $(LDFLAGS) $(LIBPATHS) $(LIBS) -o [email protected]
    %.self : %.elf
        $(VERB) echo CEX self ... $(notdir [email protected])
        $(VERB) mkdir -p "build"
        $(VERB) $(STRIP) $< -o $(BUILDDIR)/$(notdir $<)
        $(VERB) #cp $(VERSION_NAME).elf $(BUILDDIR)/$(VERSION_NAME).elf
        $(VERB) $(SPRX) $(BUILDDIR)/$(notdir $<)
        $(VERB) $(PS3DEV)/bin/$(SELF) $(SCETOOL_FLAGS) --encrypt $(BUILDDIR)/$(notdir $<) [email protected]
        $(VERB) mv $(PS3DEV)/bin/[email protected] $(BUILDDIR)/[email protected]
        $(VERB) #$(FSELF) $(BUILDDIR)/$(notdir $<) $(basename [email protected]).fake.self
    #%.pkg: %.self
        @echo
        $(VERB) echo building pkg ... $(notdir $(PKG_NAME))
        $(VERB) mkdir -p build/pkg/USRDIR
        $(VERB) cp $(ICON0) $(BUILDDIR)/pkg/ICON0.PNG
        $(VERB) $(SELF_NPDRM) $(SCETOOL_FLAGS) --np-content-id=$(CONTENTID) --encrypt $(BUILDDIR)/$(basename $(notdir $<)).elf $(BUILDDIR)/pkg/USRDIR/EBOOT.BIN #$(CONTENTID) >> /dev/null
        $(VERB) $(PS3SDK)/MinGW/Python27/python$(EXTENSION) $(PS3DEV)/bin/$(SFO) --title "$(TITLE)" --appid "$(APPID)" -f $(SFOXML) $(BUILDDIR)/pkg/PARAM.SFO
      
        $(VERB) if [ -n "$(PKGFILES)" -a -d "$(PKGFILES)" ]; then cp -rf $(PKGFILES)/* $(BUILDDIR)/pkg/; fi
        $(VERB) $(PS3SDK)/MinGW/Python27/python$(EXTENSION) $(PS3DEV)/bin/$(PKG) --contentid $(CONTENTID) $(BUILDDIR)/pkg/ $(PKG_NAME) >> /dev/null
        $(VERB) #cp $(PKG_NAME) $(basename $(PKG_NAME)).gnpdrm.pkg
        $(VERB) #$(PACKAGE_FINALIZE) $(basename $(PKG_NAME)).gnpdrm.pkg
    endif
    Thanks..
     
    Last edited by a moderator: Sep 16, 2018
    antonioks, DeViL303, kozarovv and 5 others like this.
  2. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    I know that pS3 has just 256MB of RAM...
    Or it depends about other things about you?
    I written my own dmalloc library to check if I have some unfreed memory.
    But it seems all ok...
     
  3. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    STLcardsWS likes this.
  4. 666
    700
    147
    Zar

    Zar Developer

    Joined:
    Oct 15, 2014
    Messages:
    666
    Likes Received:
    700
    Trophy Points:
    147
    I noticied sometimes some legit line of codes make the homebrew crash... i don't know why. Perhaps, "for( ; ; )" is one of them. Try to remplace it by "while(1)".
     
    kozarovv and STLcardsWS like this.
  5. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    Thanks, I tried, but no luck unfortunately...:frown new:
     
    STLcardsWS likes this.
  6. 666
    700
    147
    Zar

    Zar Developer

    Joined:
    Oct 15, 2014
    Messages:
    666
    Likes Received:
    700
    Trophy Points:
    147
    Can you tell me which game is not working ? I'll try to fix it :p
    nvm, i didn't see tmnt :p
     
  7. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    Thank you so much for your interest.
    For information the games that do not start are those that use more ram (like mine that loads many scripts inside).
    The "for ( ;; )" loop which is now is "while (1)" runs a little and then nothing more (freeze app).
    Is there a possibility to use another type of memory allocation on ps3?
    maybe some ps3 api that replace malloc?

    Ps. if you compile with
    CFLAGS + = -DDMALLOC_MODE
    you can enable debugging on the memory allocation I wrote in this file:
    safealloc.h on github
    no unfreed memory ...
    :concern:
     
  8. 666
    700
    147
    Zar

    Zar Developer

    Joined:
    Oct 15, 2014
    Messages:
    666
    Likes Received:
    700
    Trophy Points:
    147
    I don't know.

    I think when you use malloc, it use the syscall 348&349, maybe if you use it directly with a paging of 64K will be more efficient.

    in psl1ght these syscall are name :
    sysMemoryAllocate(size_t size,u64 flags,sys_mem_addr_t *alloc_addr)
    sysMemoryFree(sys_mem_addr_t start_addr)

    something like that may works :

    Code:
    char *fake_malloc(size_t size)
    {
        if(size = 0) return NULL;
        
        char *mem;
        sys_mem_addr_t addr;
            
        if (sysMemoryAllocate(size, SYS_MEMORY_PAGE_SIZE_64K, &addr) == 0)
        {
            memcpy((void *)addr, (void *) mem, size);
            return mem;
        }   
        
        return NULL;
    }
    
    void fake_free (void* ptr)
    {
        if(ptr==NULL) return;
        sysMemoryFree((sys_mem_addr_t) ptr);
    }
    
    #define free fake_free
    #define malloc fake_malloc
    
    there is also mmap... but i don't rly know how to use it. Here, every syscalls available
    http://www.psdevwiki.com/ps3/LV2_Functions_and_Syscalls

    I looked at your project, and it's very great project, it's like mugen ^^'. I'm glad you are porting it to the ps3.
     
    DeViL303, kozarovv, 3141card and 2 others like this.
  9. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    Big thanks!!
    Well.. I got some compiler errors so I rewrote the code about you advices.
    I write my safealloc.h to simulate strdub, realloc and calloc too.
    Here the file:
    https://www83.zippyshare.com/v/4DqIj69R/file.html
    https://www66.zippyshare.com/v/5Jv3xsGH/file.html
    I uploaded the build bat for PS3.

    But I got a crash...
    If you want to try download the project from github, overwrite safealloc.h
    and use buildPS3.bat into engine folder to build the pkg.
    I don't know what is the issue...
     
    Last edited: Sep 3, 2018
    DeViL303, kozarovv and 3141card like this.
  10. 666
    700
    147
    Zar

    Zar Developer

    Joined:
    Oct 15, 2014
    Messages:
    666
    Likes Received:
    700
    Trophy Points:
    147
    I already compiled your project with my environement (I just had to add SDL2 and VPX lib to my environement).

    I compiled it with this cmd line : "build.sh 11"

    Now, iI'm just like you, i'm trying to fix it but i cannot guaranty any success ;p
     
  11. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    Big thanks!!:tranquillity:
     
  12. 832
    686
    147
    littlebalup

    littlebalup Developer PSX-Place Supporter

    Joined:
    Oct 16, 2014
    Messages:
    832
    Likes Received:
    686
    Trophy Points:
    147
    Location:
    43°36'16.0"N 1°26'36.1"E
    A "team" is born ;)

    Great project.
     
  13. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    Thanks.
    Does it exist a realloc func for ps3 by syscall? Without it we can align the memory. Or just we can implement own alloc system but this can use more memory...
     
  14. 666
    700
    147
    Zar

    Zar Developer

    Joined:
    Oct 15, 2014
    Messages:
    666
    Likes Received:
    700
    Trophy Points:
    147
    I didn't find the realloc.

    Also, I didn't succeed to re-write the malloc function properly with the syscall.
    I think the best one was something like that

    void* ps3_malloc(uint32_t Size) {
    sys_mem_addr_t Addr;
    Size = ((Size + 65536) / 65536) * 65536;
    sysMemoryAllocate(Size, SYS_MEMORY_PAGE_SIZE_64K, &Addr);
    return (void*) (uintptr_t) Addr;
    }
    void ps3_free(void* Addr) {
    sysMemoryFree((sys_mem_addr_t) (uintptr_t) Addr);
    }

    but it's not working.

    I noticied that sometimes when I launch the homebrew (without pak file and with the original safealloc). It doesn't display the menu. I have a blackscreen, it stuck here : https://github.com/whitedragon0000/OpenBOR_PLUS/blob/master/engine/ps3/menu.c#L855 with a blackscreen.

    I don't think anymore that changing the malloc function is going to solve the issue. Perhaps, the issue come from the code or maybe from the lib SDL2 ? I never used it.
    I'm still investigating, this one is hard to find ^^'
     
    3141card likes this.
  15. 434
    260
    97
    haxxxen

    haxxxen Developer

    Joined:
    Oct 16, 2014
    Messages:
    434
    Likes Received:
    260
    Trophy Points:
    97
    it seems you are working with psl1ght. maybe it is better to switch to sdk, since you can use standard c/c++ libraries. there is also a malloc replacement sample

    on the other hand, have you tried the toolchain used for cobra?
     
  16. 126
    167
    72
    3141card

    3141card Developer

    Joined:
    Oct 13, 2014
    Messages:
    126
    Likes Received:
    167
    Trophy Points:
    72
    Location:
    Germany
    Its really long ago I play with this, but I often use SDL if I need a GUI for an app. I remember,
    there was a mem leak with SDL. The old display buffer was not freed, and based on size of
    framebuffer, out of memory after a certain number of frames. Freeze on the last frame.

    Sry I have no time, or PS3, atm for help direct @Zar. :(

    BTW, thx for your port @White Dragon :)
     
    kozarovv, Zar and STLcardsWS like this.
  17. 666
    700
    147
    Zar

    Zar Developer

    Joined:
    Oct 15, 2014
    Messages:
    666
    Likes Received:
    700
    Trophy Points:
    147
    I removed every function of SDL, and then I launched it. It doesn't solve the issue, it can load more files but not all of them.

    Does it load every file from PAK to the memory ? Do you know how much data is loaded in memory ? Perhaps, the ps3 cannot handle too much...
     
  18. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    Hi Zar!!
    Well, the big used memory come from scriptlib: Lexer, Parser etc...
    I tried to make Token source char array into a ptr (theSource *).
    but...
    1) When you call Get_NextToken() it lose the malloc trace... need to write a structure
    2) deframmented malloc memory uses more memory than array and so...
    maybe this is the best result...
    PS3 seems to poor of RAM..
     
    littlebalup and STLcardsWS like this.
  19. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    Soon I can do a release.. :courage:
    BTW Thanks to all!
     
    littlebalup and STLcardsWS like this.
  20. 52
    81
    42
    White Dragon

    White Dragon Developer

    Joined:
    Aug 28, 2018
    Messages:
    52
    Likes Received:
    81
    Trophy Points:
    42
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Lecce (Italy)
    Home Page:
    Oh.. Do you know if have ps3 some recursion limit protection?
     

Share This Page