Ps3 Memory leaks... (Developing OpenBOR port for PS3)

----------------------------------------------------------------------
STAFF EDIT: Project Released: Link (first version)

• http://www.psx-place.com/threads/openbor-plus-engine-ported-to-the-ps3-by-white-dragon.20740/

------------------------------------------------------------------------
Hi to all!
I'm developing OpenBOR port for PS3.
Here you can find a working source.

https://github.com/whitedragon0000/OpenBOR_PLUS

Some games work well, but other not load (app freeze) for some memory leaks...

To test:
in this source file:
https://github.com/whitedragon0000/OpenBOR_PLUS/blob/master/engine/source/preprocessorlib/pp_lexer.c
into HRESULT pp_lexer_GetNextToken (pp_lexer *plexer, pp_token *theNextToken) function
when I put a log with printf(...) before for(; it prints but in the next line doesn't print.
Example:

Code:

HRESULT pp_lexer_GetNextToken (pp_lexer *plexer, pp_token *theNextToken)
{
printf("I'm Here 1"); //IT PRINTS!
    for(;;)
    {
printf("I'm Here 2"); //IT DOESEN'T PRINT!
        plexer->theTokenSource[0] = plexer->theTokenLen = 0;
        plexer->theTokenPosition = plexer->theTextPosition;
plexer->tokOffset = plexer->offset;

It seems a memory leak..

To test it build the project decompress all project into a dir.
Decompress ps3-sdk
run build.bat or create a new bat with:

Code:

@setlocal
@echo off
set BUILDBATCH=1
set TOOLS=../tools/bin;../tools/7-Zip;../tools/svn/bin
set PATH=%TOOLS%
bash.exe build.sh 11
@endlocal

@setlocal
@echo off

set TOOLS=../tools/bin;../tools/7-Zip;../tools/svn/bin
set PATH=%TOOLS%;%PATH%
call xbox/make.bat
bash.exe version.sh 1
:missing
@endlocal

pause

into engine folder.

Games with many scripts inside don't run...
Test my game for example:
https://gamejolt.com/games/teenage-mutant-ninja-turtles-shell-shocked-the-arcade-game/35259

In windows it works well and... into PS3 emulator it works well!!
But into a real PS3 I think.. there is a memory leak.. (I thinks...)

Any help?
Maybe there is a memory protection on PS3? Can I disable it?
For info my make file is:

Code:

ifdef BUILD_PS3
PKG_NAME = $(VERSION_NAME).pkg
INCLUDE = $(foreach dir,$(INCLUDES), -I$(dir)) \
          $(foreach dir,$(LIBDIRS),-I$(dir)/include) \
          $(LIBPSL1GHT_INC) \
          $(foreach dir,$(INCS), -I$(dir))
include $(PSL1GHT)/ppu_rules
SFOXML=$(CURDIR)/resources/sfo.xml
ICON0=$(CURDIR)/resources/ICON0.PNG
SCETOOL_FLAGS += --self-ctrl-flags 4000000000000000000000000000000000000000000000000000000000000002
SCETOOL_FLAGS += --self-cap-flags 00000000000000000000000000000000000000000000007B0000000100000000
all : $(TARGET) $(TARGET_FINAL)
    @echo
%.o : %.c
    @echo Compiling $(TARGET_PLATFORM) Port: $<...
    @#$(CC) $(CFLAGS) -c $< -o [email protected]
    $(VERB) $(CC) $(DEPSOPT) $(CFLAGS) -c $< -o [email protected] $(ERROR_FILTER)
$(TARGET) : $(OBJS) $(RES)
    @echo
    @echo Linking $(TARGET_PLATFORM) Port: $(TARGET)...
    @#$(CC) $(CFLAGS) -o $(TARGET) $(OBJS) $(TARGET_RESOURCE) $(LIBS)
    $(VERB) $(PREFIX)gcc $^ $(LDFLAGS) $(LIBPATHS) $(LIBS) -o [email protected]
%.self : %.elf
    $(VERB) echo CEX self ... $(notdir [email protected])
    $(VERB) mkdir -p "build"
    $(VERB) $(STRIP) $< -o $(BUILDDIR)/$(notdir $<)
    $(VERB) #cp $(VERSION_NAME).elf $(BUILDDIR)/$(VERSION_NAME).elf
    $(VERB) $(SPRX) $(BUILDDIR)/$(notdir $<)
    $(VERB) $(PS3DEV)/bin/$(SELF) $(SCETOOL_FLAGS) --encrypt $(BUILDDIR)/$(notdir $<) [email protected]
    $(VERB) mv $(PS3DEV)/bin/[email protected] $(BUILDDIR)/[email protected]
    $(VERB) #$(FSELF) $(BUILDDIR)/$(notdir $<) $(basename [email protected]).fake.self
#%.pkg: %.self
    @echo
    $(VERB) echo building pkg ... $(notdir $(PKG_NAME))
    $(VERB) mkdir -p build/pkg/USRDIR
    $(VERB) cp $(ICON0) $(BUILDDIR)/pkg/ICON0.PNG
    $(VERB) $(SELF_NPDRM) $(SCETOOL_FLAGS) --np-content-id=$(CONTENTID) --encrypt $(BUILDDIR)/$(basename $(notdir $<)).elf $(BUILDDIR)/pkg/USRDIR/EBOOT.BIN #$(CONTENTID) >> /dev/null
    $(VERB) $(PS3SDK)/MinGW/Python27/python$(EXTENSION) $(PS3DEV)/bin/$(SFO) --title "$(TITLE)" --appid "$(APPID)" -f $(SFOXML) $(BUILDDIR)/pkg/PARAM.SFO
  
    $(VERB) if [ -n "$(PKGFILES)" -a -d "$(PKGFILES)" ]; then cp -rf $(PKGFILES)/* $(BUILDDIR)/pkg/; fi
    $(VERB) $(PS3SDK)/MinGW/Python27/python$(EXTENSION) $(PS3DEV)/bin/$(PKG) --contentid $(CONTENTID) $(BUILDDIR)/pkg/ $(PKG_NAME) >> /dev/null
    $(VERB) #cp $(PKG_NAME) $(basename $(PKG_NAME)).gnpdrm.pkg
    $(VERB) #$(PACKAGE_FINALIZE) $(basename $(PKG_NAME)).gnpdrm.pkg
endif

Thanks..

 

I know that pS3 has just 256MB of RAM...
Or it depends about other things about you?
I written my own dmalloc library to check if I have some unfreed memory.
But it seems all ok...

 

I just compiled a pre-release build:
https://www102.zippyshare.com/v/Ry6WfE6G/file.html
I like, when possible, to know what's up about you

 

I noticied sometimes some legit line of codes make the homebrew crash... i don't know why. Perhaps, "for( ; ; )" is one of them. Try to remplace it by "while(1)".

 

Thanks, I tried, but no luck unfortunately...

 

Can you tell me which game is not working ? I'll try to fix it :p
nvm, i didn't see tmnt :p

 

Thank you so much for your interest.
For information the games that do not start are those that use more ram (like mine that loads many scripts inside).
The "for ( ;; )" loop which is now is "while (1)" runs a little and then nothing more (freeze app).
Is there a possibility to use another type of memory allocation on ps3?
maybe some ps3 api that replace malloc?

Ps. if you compile with
CFLAGS + = -DDMALLOC_MODE
you can enable debugging on the memory allocation I wrote in this file:
safealloc.h on github
no unfreed memory ...

 

I don't know.

I think when you use malloc, it use the syscall 348&349, maybe if you use it directly with a paging of 64K will be more efficient.

in psl1ght these syscall are name :
sysMemoryAllocate(size_t size,u64 flags,sys_mem_addr_t *alloc_addr)
sysMemoryFree(sys_mem_addr_t start_addr)

something like that may works :

Code:

char *fake_malloc(size_t size)
{
    if(size = 0) return NULL;
    
    char *mem;
    sys_mem_addr_t addr;
        
    if (sysMemoryAllocate(size, SYS_MEMORY_PAGE_SIZE_64K, &addr) == 0)
    {
        memcpy((void *)addr, (void *) mem, size);
        return mem;
    }   
    
    return NULL;
}

void fake_free (void* ptr)
{
    if(ptr==NULL) return;
    sysMemoryFree((sys_mem_addr_t) ptr);
}

#define free fake_free
#define malloc fake_malloc

there is also mmap... but i don't rly know how to use it. Here, every syscalls available
http://www.psdevwiki.com/ps3/LV2_Functions_and_Syscalls

I looked at your project, and it's very great project, it's like mugen ^^'. I'm glad you are porting it to the ps3.

 

Big thanks!!
Well.. I got some compiler errors so I rewrote the code about you advices.
I write my safealloc.h to simulate strdub, realloc and calloc too.
Here the file:
https://www83.zippyshare.com/v/4DqIj69R/file.html
https://www66.zippyshare.com/v/5Jv3xsGH/file.html
I uploaded the build bat for PS3.

But I got a crash...
If you want to try download the project from github, overwrite safealloc.h
and use buildPS3.bat into engine folder to build the pkg.
I don't know what is the issue...

 

I already compiled your project with my environement (I just had to add SDL2 and VPX lib to my environement).

I compiled it with this cmd line : "build.sh 11"

Now, iI'm just like you, i'm trying to fix it but i cannot guaranty any success ;p

 

Big thanks!!

 

A "team" is born

Great project.

 

Thanks.
Does it exist a realloc func for ps3 by syscall? Without it we can align the memory. Or just we can implement own alloc system but this can use more memory...

 

I didn't find the realloc.

Also, I didn't succeed to re-write the malloc function properly with the syscall.
I think the best one was something like that

void* ps3_malloc(uint32_t Size) {
sys_mem_addr_t Addr;
Size = ((Size + 65536) / 65536) * 65536;
sysMemoryAllocate(Size, SYS_MEMORY_PAGE_SIZE_64K, &Addr);
return (void*) (uintptr_t) Addr;
}
void ps3_free(void* Addr) {
sysMemoryFree((sys_mem_addr_t) (uintptr_t) Addr);
}

but it's not working.

I noticied that sometimes when I launch the homebrew (without pak file and with the original safealloc). It doesn't display the menu. I have a blackscreen, it stuck here : https://github.com/whitedragon0000/OpenBOR_PLUS/blob/master/engine/ps3/menu.c#L855 with a blackscreen.

I don't think anymore that changing the malloc function is going to solve the issue. Perhaps, the issue come from the code or maybe from the lib SDL2 ? I never used it.
I'm still investigating, this one is hard to find ^^'

 

it seems you are working with psl1ght. maybe it is better to switch to sdk, since you can use standard c/c++ libraries. there is also a malloc replacement sample

on the other hand, have you tried the toolchain used for cobra?

 

Its really long ago I play with this, but I often use SDL if I need a GUI for an app. I remember,
there was a mem leak with SDL. The old display buffer was not freed, and based on size of
framebuffer, out of memory after a certain number of frames. Freeze on the last frame.

Sry I have no time, or PS3, atm for help direct @Zar.

BTW, thx for your port @White Dragon

 

I removed every function of SDL, and then I launched it. It doesn't solve the issue, it can load more files but not all of them.

Does it load every file from PAK to the memory ? Do you know how much data is loaded in memory ? Perhaps, the ps3 cannot handle too much...

 

Hi Zar!!
Well, the big used memory come from scriptlib: Lexer, Parser etc...
I tried to make Token source char array into a ptr (theSource *).
but...
1) When you call Get_NextToken() it lose the malloc trace... need to write a structure
2) deframmented malloc memory uses more memory than array and so...
maybe this is the best result...
PS3 seems to poor of RAM..

 

Soon I can do a release..
BTW Thanks to all!

 

Oh.. Do you know if have ps3 some recursion limit protection?