PS3 [Research] Modifying the Gaia Visualization (custom_render_plugin/earth.qrc)

sandungas said:
I made another small change, now the filenames of the extracted files are more clean, based in my experience using this tool for the experiments we was doing the past weeks i think this is the best filenames we can have by now with this python script
I understand some of the code and i have a clear idea of what needs to be made, but i dont know how to do it, i guess we need to open a forum thread for it incase someone skilled in python wants to take a look at it

But in the meantime, this is CXML decompiler v7 ---> https://pastebin.com/Xj878JHi

This is an example of the .xml file generated by v7 when used with earth.qrc (440-485 firmwares) ---> https://pastebin.com/mSveqnBa
Click to expand...

Wow, I'm busy until late this evening or tomorrow, but that looks like it will help loads and I will check it out then. Thanks for sharing.
 
I thought in some mods, but for some reason most of them are a bit apocalyptical, lol

-Replace the water blue color by orange/red for a "magma oceans mod"
-Replace the clouds textures by a lot of alien spaceships for a "space invaders mod"
-Replace the clouds textures by a network of satellites for a "dyson sphere mod"
-Replace the ground textures by the same versions of them but painted in whites for a "snowball earth mod"
-Replace the ground textures by the death star from star wars for a "death star mod"

And this ones not apocalyptical...
-Replace the ground textures by another planet, best candidates are the moon and mars because probably there are availables big images of his surfaces in good quality from NASA
 
Just an small detail i noticed, the settings in the HDR.mnu files inside earth.qrc are the same than the HDR.mnu files inside lines.qrc

This is earth.qrc/earth/HDR.mnu
Compare it with lines.qrc/HDR.mnu from this link.. same stuff
https://www.psdevwiki.com/ps3/Lines.qrc#HDR.mnu
Code: 
#MNU_1.0
ENABLED:int:1
EXPOSURE:float:0.789
WHITE LEVEL:float:3.40918
GLARE LEVEL:float:5.58999
GLARE THRESH:float:0.552855
GAUSSIAN RAD R:float:2.54
GAUSSIAN RAD G:float:2.67
GAUSSIAN RAD B:float:2.9
GLARE SUM POW:float:1.45048
TEX SIZE:int:8
TEX MAX MIP:int:8
GLARE:int:1
GLARE_ONLY:int:0
TONEBEFORE:int:1
BLUR:int:0
DITHER:float:0.00392157

Also...
All the ground/cloud/specular textures are 512x512 pixels .jpg
There is one in dxt5 format that is an small circle (named earth/flashrom/disk.dds)... i guess is the atmosphere
There is other in dxt1 format at size 1024x1024 filled with random pixels (named earth/flashrom/face_neg_z_ul.dds), i cant imagine what is this

The .ini files looks like this:
Code: 
name "0" eye 1.44094 -0.775129 0.101691 center 0.627144 -0.277511 -0.198493 up -0.556286 -0.816485 0.15459 sun 0.988368 -4.37114e-08 0.152079 
name "1" eye 0.204872 -0.96446 0.494347 center 0.806419 -0.165906 0.473106 up 0.773099 -0.575264 0.26719 sun 0.988368 -4.37114e-08 0.152079
name "2" eye 0.967873 -0.507445 -0.447211 center 0.634513 -0.277953 0.467232 up 0.176141 0.968005 -0.178722 sun 0.988368 -4.37114e-08 0.152079
name "3" eye 1.26552 4.10793 0.571134 center 1.2334 3.12243 0.404542 up 0.571038 -0.154891 0.806179 sun 0.988368 -4.37114e-08 0.152079
name "4" eye 0.888353 -0.59669 0.779265 center 0.422026 0.195101 0.384795 up 0.839021 0.254573 -0.480871 sun 0.988368 -4.37114e-08 0.152079
name "5" eye 1.95592 -0.45782 -1.00303 center 1.22599 -0.275096 -0.344386 up 0.649864 -0.113123 0.751585 sun 0.988368 -4.37114e-08 0.152079
name "6" eye 0.578377 0.734258 -0.668724 center 0.628489 0.522955 0.307411 up 0.975077 0.221859 -0.00203224 sun 0.988368 -4.37114e-08 0.152079
name "7" eye 0.198949 0.705864 1.97533 center 0.148534 0.435372 1.01393 up 0.998562 0.00393033 -0.053469 sun 0.988368 -4.37114e-08 0.152079
name "8" eye -1.02189 0.766253 0.500611 center -0.0315643 0.716997 0.630304 up -0.0491416 0.749687 0.659966 sun 0.988368 -4.37114e-08 0.152079
name "9" eye 2.56622 -0.91969 -3.64661 center 2.03882 -0.725666 -2.81944 up 0.394096 0.918369 0.0358585 sun 0.988368 -4.37114e-08 0.152079

And the camera .path like this, pretty much the same, but it controls timings and the loops
Code: 
time=0 eye 0.422539 2.09046 -1.33119 center 0.622381 1.33315 -0.70946 up 0.305831 0.651039 0.694706 
time=8 eye 1.07567 1.57645 -1.1403 center 0.984072 0.859827 -0.44888 up 0.305385 0.640666 0.704476 
time=16 eye 1.29704 1.14223 -0.841381 center 1.0412 0.484821 -0.132608 up 0.305386 0.640669 0.704473 
time=24 eye 0.828594 1.7248 -0.182823 center 0.86978 0.758412 0.0709437 up 0.319901 0.253367 0.912945 
time=32 eye 0.750056 1.4679 0.197686 center 0.804201 0.475039 0.0914039 up 0.664118 -0.0436771 0.746351 
time=40 eye 0.637955 1.19379 1.13086 center 0.727631 0.404913 0.522887 up 0.842646 -0.265325 0.468563 
time=48 eye 0.655115 0.390202 1.51134 center 0.635987 0.104536 0.553202 up 0.854008 -0.502995 0.132917 
time=56 eye 0.609235 -0.511079 1.48236 center 0.596877 -0.124293 0.560273 up 0.855355 -0.473528 -0.210094 
time=64 eye 0.795642 -1.05035 1.15635 center 0.738402 -0.241922 0.570544 up 0.610354 -0.436005 -0.661337 
time=72 eye 0.485701 -1.23742 0.849529 center 0.70702 -0.355216 0.433917 up 0.363246 -0.470088 -0.804406 
time=80 eye 0.186683 -1.39231 0.717953 center 0.585941 -0.514085 0.454677 up 0.390994 -0.422832 -0.817519 
time=88 eye -0.268674 -1.49701 0.419066 center 0.361473 -0.720654 0.405398 up 0.424042 -0.358822 -0.831526 
time=96 eye -0.661413 -1.39268 -0.197196 center 0.143672 -0.848145 0.0379962 up 0.593111 -0.733978 -0.330903 
time=104 eye -0.20772 -1.15771 -0.316527 center 0.639048 -0.702291 -0.041608 up 0.529305 -0.772887 -0.349973 
time=112 eye -0.439999 -1.02876 -0.378987 center 0.526933 -0.886303 -0.167447 up 0.222594 -0.876247 -0.427369 
time=120 eye -0.505526 -0.993577 -0.390926 center 0.469242 -0.915409 -0.18184 up 0.165201 -0.882558 -0.440226 
time=128 eye -0.545323 -0.968355 -0.400604 center 0.431475 -0.932279 -0.189502 up 0.129489 -0.884612 -0.447989 
time=136 eye -0.566097 -0.947774 -0.420514 center 0.406311 -0.943965 -0.18726 up 0.111905 -0.884937 -0.452067 
time=144 eye -0.584426 -0.913935 -0.467697 center 0.370243 -0.961217 -0.173807 up 0.0964851 -0.884848 -0.455778 
time=152 eye -0.540699 -0.868428 -0.590803 center 0.329894 -0.984728 -0.112742 up 0.124298 -0.888152 -0.442422 
time=160 eye -0.365368 -0.863431 -0.698027 center 0.42286 -0.935706 -0.0869018 up 0.210168 -0.901752 -0.37772 
time=168 eye -0.191048 -0.730348 -1.19933 center 0.325574 -0.813251 -0.347145 up 0.304198 -0.912595 -0.273193 
time=176 eye 0.27874 -0.852207 -1.71783 center 0.38052 -0.80159 -0.724309 up 0.459676 -0.888085 -0.00184606 
time=184 eye 0.453361 -0.785929 -2.08002 center 0.483877 -0.724147 -1.0824 up 0.962315 -0.271643 -0.0126128 
time=192 eye 1.20285 -1.08476 -2.22169 center 0.942719 -0.848435 -1.28548 up 0.884496 0.447229 0.132866 
time=200 eye 2.22196 -1.72544 -2.77621 center 1.76718 -1.38375 -1.95376 up 0.740093 0.658697 0.135574 
closed=0
close_dt=4
loops=1
 
Last edited:
Goodluck @DeViL303 because I don't understand any of that or how to use whatever tools he's using for that, if you get time to do a visual tutorial then maybe I can help but I guess for me it's back to reading the lines.qrc in the ps3 wiki and editing what I can with HEX Editor
 
xf505 said:
Goodluck @DeViL303 because I don't understand any of that or how to use whatever tools he's using for that, if you get time to do a visual tutorial then maybe I can help but I guess for me it's back to reading the lines.qrc in the ps3 wiki and editing what I can with HEX Editor
Click to expand...
Well dont get discouraged, there are some things that are unknown for everyone, all we are noobs in some of them :)
We are using different methods and tricks, is a bit long to explain everything in this post but i will try to explain the basics

What DeViL303 is doing is this procedure:
1) Decrypt custom_render_plugin.sprx into a .prx
2) Replace names in the decrypted .prx
3) Encrypt the .prx into custom_render_plugin.sprx

The names replaced in custom_render_plugin.sprx are the access paths of the files inside lines.qrc, icons.qrc and earth.qrc (and other .qrc files)
By replacing the names in the .sprx we are "cheating" the firmware into loading different files from inside the .qrc files... most of the times you can get an idea if the file that is going to be loaded is going to work just because the names

The CXML decompiler allows to extract the .qrc contents, it also shows a list of the hexadecimal codes used to "index" the files inside the .qrc structure... this values are very useful because we can "cheat" the firmware by modifying them (to force it to load different files)

So as you can see... both methods are pretty much the same... we are cheating the firmware into loading different files from inside the .qrc container :)

-----------------------
What you did to create the green earth btw ?... i guess you are used to the hexeditor, but im not sure if you was modifying the .sprx or the .qrc

Anyway... to modify the .qrc you should get used to this procedure i explined here. Are 6 steps, and all them are required to patch the .qrc files, when you get used to it becomes easy
https://www.psx-place.com/threads/r...der_plugin-sprx-rco.25952/page-19#post-214217

If you take a read at the post in that thread (around that days from my post) you are going to see what we was doing to "remap" files inside the .qrc structure, the procedure right now is something like this:

1) extract .qrc contents with the CXML decompiler python script
2) identify the "source" and "target" files you want to remap from the .xml generated by the tool
3) copy the values of "offset" and "length" together from each file
4) use that values as the SEARCH/REPLACE patterns to patch the .qrc file in a hexeditor


There are some other alternative tricks we did learn in the experiments
-by reducing the "length" to 9 in the .mnu files you can convert them to (empty) dummies
-by reducing the "length" to 0 in the .mnu files you can corrupt them... funny thing is the animation doesnt crashes and sometimes results in special visual effects
-we can replace a file by other of the same or smaller size and adjusting his size by modifying the "length" value
 
Last edited:
Is there any way to fit in larger images I wonder, maybe just use better compression...idk? The ground textures are not great quality tbh.

Be great if we could run extracted qrc files from hdd like the q-games devs could, no limits then. They actually wanted to use higher quality images too.

Interview here about Gaia is kind of interesting.

https://www.gamasutra.com/view/news/107699/Special_QGames_On_PS3s_Gaia_Music_Visualizer.php

Q-Games' chief of technology James McLaren spoke to Gamasutra about how that project got started, explaining: "Originally, we were working on the Gaia project (our name for the earth viewer) as a possible boot sequence for the PS3. The waving cloth background, also created by Q-Games, got the nod when the PS3 launched."
Click to expand...

What you are seeing is a slimmed-down version, due to Flash ROM restrictions, so we are happy to witness a positive reaction on various internet forums
Click to expand...

Is Q-Games helping out Sony with any other updates in the near future? McLaren explains: "Unfortunately, we can't really say much about that. We'd certainly like to expand the current Earth visualizer and allow people a little bit more user control, something we've seen a few requests for on various online forums.
Click to expand...

Perhaps a future update might include the full Blue Marble dataset, but that would need to be a hard disc-resident version..
Click to expand...
 
I don't even know what a CXML compiler is or how to use it so unless there is a visual tutorial I'm out the picture on that one. But if we can add our own custom images I can help there since that's in my realm. I am an old fart with NOOOO computer coding knowledge at all. I'm flying blind on everything I do and absorbing what I can from others. Most of the time when you all are talking I'm sitting with a blank look on my face going WTF are they even talking about, then I have to google it lol
 
DeViL303 said:
Is there any way to fit in larger images I wonder, maybe just use better compression...idk? The ground textures are not great quality tbh.
Click to expand...
Some of the .jpg images where created at 60% compression quality btw, and others at 80%
80% is aceptable (it creates a bit of pixelation), but the 60% is excesive (a lot of pixelation)
So yeah... you are right, the quality of the .jpg images is meh
Are 512x512 pixels, around 46000 bytes size

We are restricted by the bytes size, we cant exceed the size of the original because we are going to "inject" the file in the original .qrc structure

Remember when i was replacing the spline.elf and particles.elf from inside lines.qrc ?... what i was doing is exactly the same
1) locate the original position of the file
2) fill the area used by the file with zeroes (this is optional, not really needed)
3) paste the new file in the area filled with zeroes
4) adjust the "length" (in the index, this is one of the values shown by CXM decompiler) to match with the new file size
 
That is a real shame...Is there no way to change the TOC or whatever and change the file size... it must be possible in theory, these files are not encrypted? We could easily clear 5MB on flash if we needed to for this...

I guess this is not possible as we are inside a file..but it would be cool if we could use a directory traversal attack to get up out of the file and read from dev_flash/vsh/resource/qgl/ ... i mean like use ../ to get up out of the qrc... I know its probably not possible... but as there was this dev_hdd0/game/ABCD12345/ ability maybe there is some hack we do?
 
This is something I've often wondered about, if it was possible to edit or add visualisations for the PS3's music player. Nice work and an interesting read as well :)
 
DeViL303 said:
That is a real shame...Is there no way to change the TOC or whatever and change the file size... it must be possible in theory, these files are not encrypted? We could easily clear 5MB on flash if we needed to for this...

I guess this is not possible as we are inside a file..but it would be cool if we could use a directory traversal attack to get up out of the file and read from dev_flash/vsh/resource/qgl/ ... i mean like use ../ to get up out of the qrc... I know its probably not possible... but as there was this dev_hdd0/game/ABCD12345/ ability maybe there is some hack we do?
Click to expand...
The only good/easy way is by improving the CXML decompiler python script up to the point where it allows to build .qrc files from scratch
Is a bit challenging, but the CXML decompiler code is very well structured and clean, let say... it contains all needed info for rebuilding and is ready for being expanded

There is an alternative way to do it with the tools we have actually, but is a lot of work and only could be used in some files
The idea is to make room for the new (bigger) file by either reducing the size of the other files next to it, or displacing them

This is technically correct because we can adjust the "offset" and "lenght" info of all the files (this values are what we was patching to remap files inside the .qrc structure)
In other words... we can reconstruct the contents of the "file table" by patching the values at the index
 
sandungas said:
The only good/easy way is by improving the CXML decompiler python script up to the point where it allows to build .qrc files from scratch
Is a bit challenging, but the CXML decompiler code is very well structured and clean, let say... it contains all needed info for rebuilding and is ready for being expanded
Click to expand...
That sounds great but way beyond me, hopefully someone does it someday..I guess unlikely at this stage in the life cycle..

sandungas said:
There is an alternative way to do it with the tools we have actually, but is a lot of work and only could be used in some files
The idea is to make room for the new (bigger) file by either reducing the size of the other files next to it, or displacing them
Click to expand...
I was considering that, like remove all the clouds and add quality to the land etc. That sounds more in my skill range :)

sandungas said:
This is technically correct because we can adjust the "offset" and "lenght" info of all the files (this values are what we was patching to remap files inside the .qrc structure)
In other words... we can reconstruct the contents of the "file table" by patching the values at the index
Click to expand...
Can i just copy and paste the jpegs out into individual own files using a hex editor? I have not even attempted to extract a earth.qrc file yet, its next n my list.
 
DeViL303 said:
That sounds great but way beyond me, hopefully someone does it someday..I guess unlikely at this stage in the life cycle..
Click to expand...
Im just throwing the stone, the next thing that can be made is to use the correct file extensions in the extracted files, but this in itself is tricky because the code is very strict
DeViL303 said:
I was considering that, like remove all the clouds and add quality to the land etc. That sounds more in my skill range :)
Click to expand...
Is a tedious task, and there is not much room we could get, but is posible to do it by modifying the same values we was using for remapping them
DeViL303 said:
Can i just copy and paste the jpegs out into individual own files using a hex editor? I have not even attempted to extract a earth.qrc file yet, its next n my list.
Click to expand...
But you should use the CXML python script to extract the files, the good thing of that script is the way how it scans the files is very stable, is going to extract all the files accuratelly (included the linefeeds and padding at the end of the .mnu files incase they uses it)
In other words... is going to extract the files with the same "lenght" that appears in the values we was patching for remapping them

As example, from the xml generated by the CXML decompiler
Code: 
<?xml version="1.0" encoding="utf-8"?>
<qrc>
	<file-table>
		<file src="Offset=0x00255130, Length=0x0000495F.bin" id="earth/flashrom/ground_cm/5/1/000_001.jpg" />
	</file-table>
</qrc>
This is the file Offset=0x00255130, Length=0x0000495F.bin (renamed to .jpg and rotated 180º)
The file is exactly 18783 bytes (the same value we are patching for "length")
qRaclum.jpg

So the correspondency in between the values we are patching in the index and the sizes of the extracted files is perfect
 
Last edited:
Btw, there are several ways to "inject" the files manually in a hexeditor, but this sequence of actions is failproof (unless you made some mistake)

You need to open in the hexeditor 3 files...
-The original .qrc that is going to be patched (containing the file you want to overwrite in next steps)
-The original file that is going to be patched, extracted by the CXM compiler
-The file you want to patch on top of the original (it needs to be equal size or smaller)

step 1)
Select the original file extracted by the CXML decompiler, and "select all" (so you have all the bytes of the file selected entirelly)... then select "copy"

Step 2)
Select the original .qrc and click in "search" so it opens a dialog window where you can enter a search pattern... click in it and "paste" (so you are pasting the other file entirelly)... then "go"
This is going to select the file entirelly inside the .qrc structure

Step 3)
Click in "replace bytes" and use zeroes to cleanup the garbage
At this point the bytes modifyed has canged color... so you know where the file starts

Step 4)
Open the other tab of the hexeditor with your custom file... and "select all" ---> copy

Step 5)
Reurn to the tab with the .qrc
Click in the position where it was located the file, and paste your file there

Step 6)
This is optional, incase your file is smaller (most probably)
You need to take a look at the values that appears for that file in the xml generated by the CXM decompiler, as example with the file i was mentioning before...
Code: 
<?xml version="1.0" encoding="utf-8"?>
<qrc>
	<file-table>
		<file src="Offset=0x00255130, Length=0x0000495F.bin" id="earth/flashrom/ground_cm/5/1/000_001.jpg" />
	</file-table>
</qrc>
Lets say we have replaced that file, his offset is still the same (because we have not moved it), but our new file size (length) is smaller... so we need to "update" that values in the index, this way
The file extracted by the CXML decompiler is named Offset=0x00255130, Length=0x0000495F.bin
Join together both values... so they becomes 002551300000495F and search for them in the hexeditor
We changed his size so we need to replace the 495F
And thats all
 
sandungas said:
You can copypaste the python code and save it in your PC as cxmldecompiler.py ... and then run it in command line
*To run the pyhthon script this way you need to have installed python v2.7 (probably some of you already have it ?)
Click to expand...
I have just got around to checking this out, been away from my PC.

How do I use it? I assume I still need to remove the header and decompress with zlib...then what? I have py 2.7 installed and I have saved the script as a py file. I Just need to know the syntax for command line?
 
Last edited:
DeViL303 said:
I have just got around to checking this out, been away from my PC.

How do I use it? I assume I still need to remove the header and decompress with zlib...then what? I have py 2.7 installed and I have saved the script as a py file. I Just need to know the syntax for command line?
Click to expand...
First you need to open the .qrc in the hexeditor and remove the first 8 bytes (located before the 78 DA), and save the file with a different name
The 78DA are the first 2 bytes of a .zlib file
Then you need to decomrpess that file with a zlib tool, and save it with a different name
After that the file is ready to use the CXML decompiler

To make things easy for first time, create a folder in C root with name "test" and copy the cxmldecompiler.py inside it, and the .qrc (without the 8 bytes and decompressed) too next to it
If you have python installed you will see the .py file with his icon, thats good
Open a command line terminal, write cxmldecompiler.py (without any option) and press enter, this usually shows the intructions about how to use it

The command is like this:
Code: 
c:/>cxmldecompiler.py name.cxml name.xml
In this case the "name.cxml" is the qrc file after you removed the first 8 bytes and decompressed... his name and file extension doesnt matters
The "name.xml" represents the internal structure... is like in rcomage, that file stores all the info required to rebuild it identical to the original

Tips:
-Remember to use the TAB key in the keyboard to autocomplete the paths/filenames instead of typing all characters of the command individually... TAB key does magic !
-The CXML decompiler can extract the contents of QRC files, but also RAF format too (coldboot.raf, or the .raf at the background of dynamic themes), and P3T (playstation 3 themes, static and dynamic)
 
Just a more practical example... you know when i do this kind of things i use to add a new file extension to the file everytime i do some kind of "conversion" to it
This way i have a copy of every modification i made to the file (incase i make some mistake by looking at the previous versions of the files i will realize), also by looking at the filename i know how many modifications i made to it

The original file is named earth.qrc ... after removed the first 8 bytes i create a new file named earth.qrc.cropout ... and after decompression i create a new file named earth.qrc.cropout.zlibout
Yeah... is a bit weird and should not work much well incase of doing lot more modifications, keeping a record of every step could be a complete mess, lol, but with .qrc that are only 3 steps this trick work good

So... at the time im going to extract the file with the CXML decompiler tool... my file is named earth.qrc.cropout.zlibout so the command i use is:
Code: 
c:/>cxmldecompiler.py earth.qrc.cropout.zlibout earth.xml
 
You must log in or register to reply here.

Similar threads

Back
Top