PS2 PSBBN support for larger drives definitely possible

[CosmicScale]

I mentioned that I've been working on the "PSBBN Definitive English Patch" in another thread and now I'm looking into the feasibility of getting it to support drives that containing more than 130GB of data. Here's my findings.

PSBBN is basically PS2 Linux with a flashy interface. When you turn on the console it auto-boots osdboot.elf but the console freezes if the HDD contains more than 130GB of data. The job of osdboot.elf is to launch the Linux kernel and set up the runtime environment. It's basically a modified version of the PS2 Linux Runtime Environment disc that comes with the official Sony Linux kit. PSBBN versions 0.30 and below can been booted up with the actual disc that comes with the official Linux kit.

More interestingly there's the homebrew app called Kernelloader. Kernelloader is a bootloader for PS2 Linux designed to be an open source replacement for Sony's Runtime Environment. Kernelloader can also boot PSBBN 0.30.

The official Runtime Environment and Kernelloader both boot to a (mostly) functional PSBBN and are able to boot games and homebrew from the Game Channel.

The RTE and Kernelloader still boot PSBBN with a drive containing data over that 130GB limit, problem number one solved! When PSBBN has successfully booted you can browse the first 130GB of "PP" partitions in the Game Channel. The fact it only displays the first 130GB is not really an issue because the drive could (in theory) be front loaded with hundreds of PP partitions containing OPL Launcher that would in turn load the PS2 games installed past the 130GB mark. Problem number two arises when you try to launch something, everything now fail to launch. On the plus side, it doesn't corrupt anything on the HDD.

Remember I said that the PSBBN is basically PS2 Linux with a flashy interface? Well, the PSBBN Linux kernel is also open source and could potentially be modified for better APA support.

Ideally the solution to larger drive support would be a hacked osdboot.elf that could boot with larger drives and a Linux kernel with full 48-bit LBA support. Failing that, a modified version of Kernelloader and a modified Linux kernel that isn't bothered by data past the 130GB point and just works with data below that. This could actually be possible with Kernelloader and the kernel being open source.

Kernelloader hasn't been updated in 10 years but I've been able to set up an environment that can compile it. I've also been able to compile the kernel. Making the necessary modifications is unfortunately pushing my limits.

If anybody wants to help with this endeavour, want's more detailed info or want's to give it a shot then that would be great.

If anybody want's to check out my "PSBBN Definitive English Patch" then it can be found on the Internet Archive.

 

[Berion]

The fact it only displays the first 130GB is not really an issue because the drive could (in theory) be front loaded with hundreds of PP partitions containing OPL Launcher that would in turn load the PS2 games installed past the 130GB mark. Problem number two arises when you try to launch something, everything now fail to launch.

Not 130 but 128GiB. It came from LBA28 limitation.

I highly doubt because APA logic is kind of a partitions chain. Everything above supported size will be invisible because parser will not be able to index links above it. So it is not only a problem to launch those partitions but see them at all.

Well, the PSBBN Linux kernel is also open source and could potentially be modified for better APA support.

Where can we download it? Does src are attached to PSBBN discs?

This could actually be possible with Kernelloader

Yes but it must be modified first to support internal HDD with APA. In current state supporting only MC and USB (FAT32 on MBR only, no BDM and no Grimdoomer's BDM support).

 

[CosmicScale]

Not 130 but 128GiB. It came from LBA28 limitation.

I understand that. I said 130GB because it's more representative of how ULE reports used space and easier to understand.

Everything above supported size will be invisible

Those partitions being invisible wouldn't be a problem. A PP partition within the supported area would contain the OPL Launcher kelf, that in turn would launch the OPL elf, also within the supported area. OPL has full 48-bit LBA support so could then load a game from anywhere on the HDD. Although, this would all be moot if full 48-bit LBA support was added to the kernel.

Where can we download it? Does src are attached to PSBBN discs?

The source code was released under GNU General Public License Version 2. It was included on the PlayStation BB Navigator installation discs. It was also made available for download on Sony's website (unfortunately the link is now broken). The source can currently be downloaded from the Internet Archive https://archive.org/details/linux-playstation2-sources

Yes but it must be modified first to support internal HDD with APA. In current state supporting only MC and USB (FAT32 on MBR only, no BDM and no Grimdoomer's BDM support).

Kernelloader would not have to be modified to support APA. I've been using Kernelloader for years to boot Linux on an APA partition. Kernelloader just loads the kernel and sets up the environment. You just need to put the kernel on a PS2 Memory Card or USB storage.

Like I said, Kernelloader boots PSBBN version 0.30 (I couldn't get it to boot versions 0.20 or 0.32)

All you need to do is launch Kernelloader, select the kernel (the kernel MUST be on a Memory Card or USB), edit the kernel parameter as follows:

root=/dev/hda1 crtmode=ntsc graphics CONSOLE=/dev/null

Then it's just a simple matter of selecting "Boot current config". Be patient because it's a bit slow and displays a black screen for awhile but PSBBN does boot.

Of particular interest in the kernel source is the file fs/partitions/ps2.c "support for PlayStation 2 partition(APA)".
I guess I can freely post this here because it's been released under GNU General Public License. Does anybody want a crack at modifying this? I can compile the modified code and test it.

 

[Berion]

Those partitions being invisible wouldn't be a problem. A PP partition within the supported area would contain the OPL Launcher kelf, that in turn would launch the OPL elf, also within the supported area. OPL has full 48-bit LBA support so could then load a game from anywhere on the HDD. Although, this would all be moot if full 48-bit LBA support was added to the kernel.

OPL-Launcher telling OPL to load game from FS area of partition from which was launched. So it is a problem. ;] It doesn't load OPL but OPL with taken args.

Kernelloader would not have to be modified to support APA. (...) You just need to put the kernel on a PS2 Memory Card or USB storage.

Yes but it is waste of time. Not to mention MC is to slow for that task and USB needing for internal HDD stuff is also a annoying.

 

[CosmicScale]

OPL-Launcher telling OPL to load game from FS area of partition from which was launched.

No, OPL Launcher tells OPL to launch a game from a different partition to the one it was launch from. OPL Launcher exists in a PP.GameName partition and instructs OPL to launch the game from the __.GameName partition. As long as the game is in a partition that OPL can read then it should process the argument without an issue.

Yes but it is waste of time. Not to mention MC is to slow for that task and USB needing for internal HDD stuff is also a annoying.

I disagree. It's not slow because the kernel is loaded from Memory Card or USB. The kernel is only around 3MB in size and is loaded into memory at boot. Everything else runs from the HDD as normal. That's just how Linux works on PS2.

 

[Berion]

OPL-Launcher is written inside Attributes area of PP partition, and telling OPL to run game from FS area from exactly this partition. That's for sure because I was one of people who testing it in early days of the idea to replace miniOPL by something flexible.

MC is slow in regards IOPS, not MiBps.
3MiB is to large for majority of people with FMCB and apps.

 

[CosmicScale]

OPL-Launcher is written inside Attributes area of PP partition, and telling OPL to run game from FS area from exactly this partition. That's for sure because I was one of people who testing it in early days of the idea to replace miniOPL by something flexible.

I can see where the confusion comes from. OPL Launcher works differently with PSBBN. This is directly from the OPL Launcher GitHub:

"You can use a combination of 2 partitions: <PP. Partition with PS2 game resources> and <__. Partition with PS2 game>. Partition names should only differ in the first two characters. This will add compatibility with BB Navigator / PSX DESR XMB users. <PP. Partition with PS2 game resources> will contain game resources, <__. Partition with PS2 game> is the game in HDL format."

MC is slow in regards IOPS, not MiBps.
3MiB is to large for majority of people with FMCB and apps.

The official Linux kit runs like that without issues. Besides, you can gzip the kernel then it only takes up around 1MB on the memory card. You don't even have to put it on the same memory card as FreeMcBoot, or as I said it could also go on USB. Hopefully in the future osdboot.elf could be modified to load on drives with 48-bit LBA then this wouldn't even be an issue.

I'm currently looking into adding 48-bit LBA support to the Linux kernel and it looks trivial.

 

[Berion]

So this is new feature I missed then. ^^

 

[GDX]

It is also necessary to be able to create ReiserFS partitions such as:
__linux.2
__linux.1
__linux.1@

Without these partitions, PSBBN does not start.

 

[Berion]

RFS v3.5 to be precise.

__linux.1@?

@CosmicScale Would You like join to PS2-Discord? Maybe also join forces with Vitas which restoring missing BB channels?

 

[CosmicScale]

It is also necessary to be able to create ReiserFS partitions such as:
__linux.2
__linux.1
__linux.1@

Without these partitions, PSBBN does not start.

That's not a problem. I've been using actual PS2 hardware and the official Linux kit to do all the PSBBN modifications. There's a modified version of fdisk called ps2fdisk for PS2 Linux that can create the reiserfs partitions. The other week I actually manually made all the necessary partitions and did a restore to them.

 

[CosmicScale]

@CosmicScale Would You like join to PS2-Discord? Maybe also join forces with Vitas which restoring missing BB channels?

That would be great. I've been working on the kernel modifications and maybe somebody could help with that. It would be cool to look at those BB channels too. Any new channels could be added to my PSBBN patch.

 

[seatran]

Thanks for looking into this one mate. I will silently follow you on this journey. Please keep us up to date with your progress.

 

[GDX]

RFS v3.5 to be precise.

__linux.1@?

@CosmicScale Would You like join to PS2-Discord? Maybe also join forces with Vitas which restoring missing BB channels?

I can't quite remember exactly what the '@' symbol means, but I believe it's used to represent a partition that is subdivided into multiple partitions, something like that.

 

[Berion]

Oh, I get it. It is output from psfshell. It is how current version marked such partitions. In reality it not exist.

 

[El_isra]

i'm not sure on this...

But if our ATAD (and family) drivers are compatible API Wise we could just replace the old drivers

 

[CosmicScale]

With the help of ChatGPT I've been attempting to modify the kernels source fs/partitions/ps2.c and fs/partitions/ps2.h to add 48-bit LBA support. This is what we came up with:

// fs/partitions/ps2.c
#include <linux/genhd.h>
#include <linux/blk.h>
#include "./ps2.h"
#include <linux/slab.h>

static unsigned long long get_ptable_blocksize(kdev_t dev)
{
unsigned long long ret = 1024; // Default block size

// Check if the driver specifies a minimum block size
if (!blksize_size[MAJOR(dev)]) {
return ret;
}

// Determine the block size based on the device's capabilities
switch (blksize_size[MAJOR(dev)][MINOR(dev)]) {
case 2048:
ret = 2048;
break;
case 4096:
ret = 4096;
break;
case 8192:
ret = 8192;
break;
// Handle default and valid block sizes
case 1024:
case 512:
case 256:
case 0: // Zero indicates default block size
break;
default:
panic("Strange block size for partition table\n");
}

// Return the block size (unsigned long long) for 48-bit LBA support
return ret;
}

static int ps2_partition_one(struct gendisk *hd, kdev_t dev, struct ps2_partition *pp, int resv_m, int resv_s)
{
int i, pno;
long long nr_sects; // Use long long for 64-bit LBA support
struct hd_struct *part;
struct hd_seg_struct *seg, *sp;
char *p, *pe;
int resv0;

if (pp->magic != PS2_PARTITION_MAGIC)
return 0;

if ((pp->flag & PS2_PART_FLAG_SUB) != 0)
return 1;

pe = &pp->id[PS2_PART_NID];
if (pp->id[0] != '\0' && pp->id[1] != '\0' &&
strncmp((char *)(&pp->id), PS2_LINUX_ID, strlen(PS2_LINUX_ID)) == 0) {
/* PS2 Linux partition */
resv0 = resv_m;
p = &pp->id[strlen(PS2_LINUX_ID)];
} else
return 1; /* not PS2 Linux partition */

pno = 0;
while (p < pe && *p >= '0' && *p <= '9')
pno = pno * 10 + (*p++ - '0');
if (pno == 0)
pno = 1;
if (pno < 1 || pno > hd->max_p - 1)
return 1;

part = &hd->part[MINOR(dev) + pno];

if (part->hash) {
kfree(part->hash);
part->hash = NULL;
}
if (part->seg) {
kfree(part->seg);
part->seg = NULL;
}
if ((part->hash = (struct hd_seg_struct **)kmalloc(sizeof(struct hd_seg_struct *) * PS2_SEGMENT_HASH_ENTRIES, GFP_KERNEL)) == NULL)
return 0;
seg = (struct hd_seg_struct *)kmalloc(sizeof(struct hd_seg_struct) * (pp->nsub + 1), GFP_KERNEL);
if (seg == NULL) {
kfree(part->hash);
part->hash = NULL;
return 0;
}
part->nr_segs = pp->nsub + 1;
part->seg = seg;
seg->start_sect = part->start_sect = pp->start + resv0;
seg->nr_sects = nr_sects = pp->nsector - resv0;
seg->offset = 0;
seg++;
for (i = 0; i < pp->nsub; i++) {
seg->start_sect = pp->subs[i].start + resv_s;
seg->nr_sects = pp->subs[i].nsector - resv_s;
seg->offset = nr_sects;
nr_sects += seg->nr_sects;
seg++;
}
part->nr_sects = nr_sects;
part->hash_unit = (nr_sects + PS2_SEGMENT_HASH_ENTRIES - 1) / PS2_SEGMENT_HASH_ENTRIES;
sp = part->seg;
nr_sects = 0;
for (i = 0; i < PS2_SEGMENT_HASH_ENTRIES; i++) {
while (nr_sects < part->nr_sects &&
nr_sects > sp->offset + sp->nr_sects)
sp++;
part->hash[i] = sp;
nr_sects += part->hash_unit;
}

return 1;
}

int ps2_partition(struct gendisk *hd, struct block_device *bdev, unsigned long first_sector, int first_part_minor)
{
struct buffer_head *bh;
int dev_bsize, dev_ssize, stob, resv_m, resv_s;
struct ps2_partition *pp;
long sect;
int i;
char buf[8];
kdev_t dev;

dev = to_kdev_t(bdev->bd_dev);

dev_bsize = get_ptable_blocksize(dev);
dev_ssize = 512;
if (hardsect_size[MAJOR(dev)] != 0)
dev_ssize = hardsect_size[MAJOR(dev)][MINOR(dev)];
stob = dev_bsize / dev_ssize;
if (stob == 0)
stob = 1;
resv_m = PS2_PART_RESV_MAIN / dev_ssize;
if (resv_m == 0)
resv_m = 1;
resv_s = PS2_PART_RESV_SUB / dev_ssize;
if (resv_s == 0)
resv_s = 1;

sect = 0;
do {
if ((bh = bread(dev, sect / stob, dev_bsize)) == 0) {
printk("%s: unable to read sector %ld\n", kdevname(dev), sect);
return -1;
}
pp = (struct ps2_partition *)bh->b_data;
if (sect == 0) {
if (memcmp(pp->mbr.magic, PS2_MBR_MAGIC, 32) != 0 ||
pp->mbr.version > PS2_MBR_VERSION) {
brelse(bh);
return 0;
}
}
if (!ps2_partition_one(hd, dev, pp, resv_m, resv_s)) {
brelse(bh);
break;
}
sect = pp->next;
brelse(bh);
} while (sect != 0);

printk(" [APA]");
for (i = 1; i < hd->max_p; i++) {
if (hd->part[MINOR(dev) + i].nr_sects != 0) {
printk(" %s", disk_name(hd, MINOR(dev) + i, buf)); /* new */
}
}
printk("\n");

return 1;
}

// fs/partitions/ps2.h
#ifndef _PS2_H
#define _PS2_H

#include <linux/types.h>

#define PS2_SEGMENT_HASH_ENTRIES 16

#define PS2_PARTITION_MAGIC 0x00415041
#define PS2_MBR_VERSION 0x00000002
#define PS2_MBR_MAGIC "Sony Computer Entertainment Inc."

#define PS2_PART_NID 32
#define PS2_PART_NPASSWD 8
#define PS2_PART_NNAME 128
#define PS2_PART_MAXSUB 64
#define PS2_PART_MAXSUB_SHIFT 6

#define PS2_PART_FLAG_SUB 0x0001

#define PS2_PART_RESV_MAIN (4 * 1024 * 1024)
#define PS2_PART_RESV_SUB (4 * 1024)

struct ps2_partition {
__u32 sum;
__u32 magic;
__u32 next;
__u32 prev;
char id[PS2_PART_NID];
char rpwd[PS2_PART_NPASSWD];
char fpwd[PS2_PART_NPASSWD];
__u64 start; // Updated to __u64 for 48-bit LBA support
__u64 nsector; // Updated to __u64 for 48-bit LBA support
__u16 fstype;
__u16 flag;
__u32 nsub;
char date[8];
__u32 main;
__u32 number;
__u32 modver;
__u32 rsvd[7];
char name[PS2_PART_NNAME];
struct ps2_mbr {
char magic[32];
__u32 version;
__u32 nsector;
char date[8];
__u32 osd_start;
__u32 osd_count;
char rsvd[200];
} mbr;
struct ps2_subpart {
__u64 start; // Updated to __u64 for 48-bit LBA support
__u64 nsector; // Updated to __u64 for 48-bit LBA support
} subs[PS2_PART_MAXSUB];
};

#define PS2_LINUX_ID "__linux."

int ps2_partition(struct gendisk *hd, struct block_device *bdev, unsigned long first_sector, int first_part_minor);

#endif /* _PS2_H */

I'm now getting "block out of bounds" errors on boot. Does anybody know how the block sizes should be defined? I'm not experienced in debugging Linux kernels

generic_make_request: Block 128 out of bounds on dev 03:08
read_super_block: bread failed (dev 03:08, block 64, size 1024)
generic_make_request: Block 16 out of bounds on dev 03:08
read_super_block: bread failed (dev 03:08, block 8, size 1024)
Kernel panic: VFS Unable to mount root fs on 03:08

 

[akuhak]

@CosmicScale
there are bit different things.For example unsigned long isnt that bad. unsigned long (or u32) supports HDD up to 2Tb and actually your code support up to 2Tb initially. Even if somewhere code uses long instead of unsigned long, this will limit HDDs up to 1Tb that is still big enough. I think that replacing long with unsigned long will be enough.

struct hd_struct *part;

struct hd_seg_struct *seg, *sp;

this structure are part of the kernel:

You can see that it operates unsigned long, so PS2Linux doesnt support 64-bit in kernel, so anyway native kernel is limited to 2TB too even if you will compile 2Tb+ ps2 drivers, kernel itself will not be able to operate it.

PS2 driver is limited not to 32-bit (that can handle up to 2Tb), but to 28-bit (137Gb) and this limitatiion lies somewhere else not in 32-bit definitions but inside APA logic.

 

[akuhak]

Also the driver you provided does not handle game partition, its purpose only to mount BBNav partitions (linux ones) and it seems that it can do it even with big disks (as you say you will be able to boot bbnav on 2Tb disk), so the driver is doing its job quite well and you dont need to modify it.

As for game launch from BBNav main executable, I suspect that it is done via some closed source driver. It should be near opt0/bn/bn in some folder and I am afraid it is closed source, so no way to recompile it.

 

[akuhak]

I couldn't get it to boot versions 0.20 or 0.32

It can be due that fact that sony removed a lot from BBNav 0.31:
https://web.archive.org/web/2014102...169/ps2-bb-navigator-linux-properties-111554/
or maybe because they updated the kernel:
2.4.17_ps2.22
to
2.4.17_ps2.26
unfortunately 2.4.17_ps2.22 was lost in time. It was accessible on Sony site, but seems that all mirrors kept only last version (2.26) so it is not possible to know what they did change. Or maybe its because they removed tons of packages (probably faced out of space issue), but none of packages looks related to the hdd.