From 35a1c9a1332a2c70db6af9c5143efa116df21775 Mon Sep 17 00:00:00 2001 From: "Admiral H. Curtiss" Date: Sun, 31 Jan 2016 16:05:47 +0100 Subject: [PATCH] Add Memory Card File System documentation to repository for easy reference. Originally from http://www.csclub.uwaterloo.ca:11068/mymc/ps2mcfs.html. It's public domain, so there's no problem with just putting it in here. --- pcsx2/Reference/PS2-MemoryCardFileSystem.htm | 883 +++++++++++++++++++ 1 file changed, 883 insertions(+) create mode 100644 pcsx2/Reference/PS2-MemoryCardFileSystem.htm diff --git a/pcsx2/Reference/PS2-MemoryCardFileSystem.htm b/pcsx2/Reference/PS2-MemoryCardFileSystem.htm new file mode 100644 index 0000000000..ab70426b36 --- /dev/null +++ b/pcsx2/Reference/PS2-MemoryCardFileSystem.htm @@ -0,0 +1,883 @@ + + + +PlayStation 2 Memory Card File System + + + +

+PlayStation 2 Memory Card File System +

+
+

By + +Ross Ridge + +

+

Public Domain

+
+
+

+This document is a description of the file system layout as used on +PlayStation 2 memory cards. It's based on the research I did while +writing mymc, +a utility for working with PS2 memory card images. This document +tries to be comprehensive an accurate, but some details may be +missing, misleading or just plain wrong. At lot of assumptions had to +be made during my research, and it's hard know to what exactly Sony +intended in every case. All most all of the names for structures, +fields and flags were made up by me. Nothing in this document should +be considered official. +

+

+For brevity, unused fields and flag bits are omitted from the tables. +In most cases unused fields or flags should be assumed to be either +reserved or padding and set to zero when writing. In particular, +you'll need to pad out the structures to the length given at the top +of the table. All values are stored on the card using the +little-endian byte order. +

+
+
+

NAND Flash Memory Basics

+ +
+

Glossary

+ + +

A number of terms are used in this document that you may not +familiar with or are different from other sources describing PS2 +memory cards, so I've created short glossary. +

+ +
+ +
block
+
See "erase block".
+ +
cluster
+
The unit of allocation used in the file system. A cluster +is one or more pages in size.
+ +
ECC
+
Error correcting code. A means of encoding data so that random +bit errors can be detected and corrected.
+ +
erase block
+
The basic erasable unit on a memory card.
+ +
half
+
A two byte unsigned half-word value.
+ +
page
+
The basic addressable unit on a memory card. Corresponds to page +on the flash device used in the memory card, and is analogous +to a sector on hard disk.
+ +
programming
+
The operation of changing erased bits on a flash device from 1 to 0.
+ +
superblock
+
The first page on the memory card where important information +about the structure of the file system is kept.
+ +
word
+
A four byte unsigned word value.
+ +
+
+ +

Since PlayStation 2 (PS2) memory cards use NAND Flash it's helpful to + understand some of the basics about this kind of memory works. + Flash is a non-volatile form of memory that can be electrically + erased and reprogrammed. Since it's non-volatile, meaning it + doesn't need power to retain data, and reprogramable, meaning the + data stored can be changed, it's the ideal form memory to use in + memory card. However, flash memory does have number of limitations + which affect how the PS2 uses its memory cards. +

+

One of the limitations is a relatively slow random access time. + While much faster than hard disk, NAND flash is much slower than + RAM at reading random bytes in memory. Reading the first byte of + data from a flash device takes about 25 microseconds. Fortunately, + reading on sequentially is much faster. After the first + byte is read, each subsequent byte takes only about 50 nanoseconds + to read. For this reason NAND flash memory is organized in to pages, + similar to how a hard drive is organized into sectors. + For example, with the TC58V64AFT, a flash device used in PS2 memory cards, + an entire 528 byte page can be read at a sustained 10 Mb/s + transfer rate. By comparison, the transfer rate for reading random + bytes is much slower, only 40kb/s. Because the serial bus the + PS2 uses to talk to memory cards isn't capable of 10 Mb/s, actual + transfer rates will be slower. +

+

The biggest limitation of flash devices is how they're written. + Instead changing bits of memory to 0 or 1 depending on the data + being written, flash memory can only change a bit from 1 to 0. + Once a bit of memory has been changed to 0, it can't be changed + back to 1 except by erasing it. Erasing is an operation that sets + an entire range of memory, called an erase block, to all 1s. Once + a block is erased, it can be written through an operation called + programming which changes 1 bits to 0 bits. Since an erase block + is made up of a number of pages, this makes writing a more + complicated operation than it would be on a hard disk. In order to + write a single page to a flash device you need to erase to the + block that the page belongs to. However, since that will erase all + the pages that belong to that block, not just the page being + written, you first need to read in all of the other pages that are + going to be erased. After the block is erased you then can program + the erase block with the contents of the page you're trying to + write and along with the content of all the other pages. + Some flash devices work the other way, erasing sets the block + to all 0s while programming changes 0s to 1s. +

+

Another limitation is that flash memory isn't as reliable as RAM + memory. A flash device will typically have a certain number of bad + blocks when it ships from the factory, and it's possible for + defects to appear in the media as it used. Also, a block can + eventually "wear out" after a few hundred thousand program/erase + cycles. For this reason each page is divided into two parts, a + data area a spare area. The data area is used to store ordinary + data, while the much smaller spare area is for software defined + error-correcting codes (ECC), wear leveling, bad block remapping, + and other functions meant to deal with defects in the media. +

+ +

The flash devices used in PS2 memory cards have a 528 byte page + size. This is divided into a 512 byte data area and 16 byte spare + area. The spare area is used to store 12 bytes of ECC data, with + the remaining 4 bytes left unused. Erase blocks are 16 pages long. + The are 16384 pages, for a total combined raw data area capacity + 8,388,608 bytes. +

+ +
+
+

File System Organization

+ +
+

Standard Memory
+Card Layout

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
0x0000Superblock
0x0001Unused
 
0x0010Indirect FAT Table
0x0012FAT Table
 
 
 
0x0052Allocatable Clusters
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
0x3ED2Reserved Clusters
 
 
 
 
 
0x3FE0Backup Block 2
 
0x3FF0Backup Block 1
0x3FFF
+
+ +

The PS2 memory card file system has a fairly simple design, with + some allowances made for the limitations of flash memory. Its + overall structure is similar to the well known MS-DOS FAT file + system. It uses a file allocation table (FAT) to keep track of + allocated space and a hierarchical directory system where all of + a file's metadata is stored in its directory entry. Like the FAT file + system, which groups disk sectors into clusters, the PS2 memory + card file system groups flash memory pages in to clusters. On + standard PS2 memory cards, the cluster size 1024 bytes, or 2 pages + long. +

+
+

The Superblock

+

The key to the PS2 memory card file system is the superblock. + Located in the first page of the memory, this is the only part of + the file system with a fixed location. While some things like + the do end up in fixed locations on standard 8M memory cards, you + shouldn't rely on this. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Superblock (340 bytes)
OffsetNameTypeDefaultDescription
0x00magicbyte[28]-Identifies the card as being formatted. +Set to the ASCII string "Sony PS2 Memory Card Format ". +
0x1Cversionbyte[12]1.X.0.0Version number of the format used. + +
Version 1.2 indicates full support for bad_block_list. +
0x28page_lenhalf512Size in bytes of a memory card page. +
0x2Apages_per_clusterhalf2The number of pages in a cluster. +
0x2Cpages_per_blockhalf16The number of pages in an erase block. +
0x2E-half0xFF00Doesn't seem to be used
0x30clusters_per_cardword8192The total size of the card in clusters. +
0x34alloc_offsetword41Cluster offset of the first allocatable cluster. Cluster values +in the FAT and directory entries are relative to this. + +
This is the cluster immediately after the FAT
0x38alloc_endword8135The cluster after the highest allocatable cluster. Relative to +alloc_offset. + +
Not used.
0x3Crootdir_clusterword0First cluster of the root directory. Relative to +alloc_offset. + +
Must be zero.
0x40backup_block1word1023Erase block used as a backup area during programming. + +
Normally the the last block on the card, it may have a different value +if that block was found to be bad.
0x44backup_block2word1022This block should be erased to all ones. + +
Normally the the second last block on the card.
0x50ifc_listword[32]8List of indirect FAT clusters. + +
On a standard 8M card there's only one indirect FAT cluster.
0xD0bad_block_listword[32]-1List of erase blocks that have errors and shouldn't be used. +
0x150card_typebyte2Memory card type. + +
Must be 2, indicating that this is a PS2 memory card.
0x151card_flagsbyte0x52Physical characteristics of the memory card. +
+ +
+ + + + + + + + + + + + + + + + + + + + +
Card Flags
MaskNameDescription
0x01CF_USE_ECCCard supports ECC. +
0x08CF_BAD_BLOCKCard may have bad blocks. +
0x10CF_ERASE_ZEROESErased blocks have all bits set to zero. +
+
+ +

Most of the fields in the superblock should be self-explanatory. + The fields page_len, pages_per_cluster, + pages_per_block, and clusters_per_card define the + basic geometry of the file system. The FAT can be accessed using + ifc_list and rootdir_cluster gives the first cluster + of the root directory. Cluster offsets in FAT and directory + entries are all relative to alloc_offset +

+ +

File systems ment to be compatible with the PS2's memory card + drivers have a fairly restricted set of geometry options. The + field page_size can be either 512 or 1024. If the page size + is 512 then pages_per_cluster can either be 1 or 2, + otherwise it can only be 1. The limit on pages_per_block is + 16. There doesn't seem to be any upper limit on + clusters_per_card, however because of the size of ifc_list + there can be no more than 8,388,608 allocatable clusters. + While the clusters the make up the FAT can be located anywhere, the + value of rootdir_cluster must be 0, indicating that the first + allocatable cluster is the first cluster of the root directory. +

+ +
+ +
+

File Allocation Table

+ +

The file allocation table is used for keeping track of which + clusters are in use and form a linked-list of the clusters that + belong to each file. Each entry in the table is a 32-bit value. + If the the cluster corresponding to the entry is free then the most + significant bit will be clear. Otherwise, it will be set and the + lower 31-bits of the value are the index of the next cluster in the + file. These indexes are relative to alloc_offset given in + the superblock. A value of 0xFFFFFFFF indicates that this + entry is the last cluster in the file. +

+

Unlike the MS-DOS FAT file system, the table isn't required to to + be in single contiguous range of clusters. Instead a system of + double-indirect indexing is used that allows the clusters that make + up the file allocation table to be scattered across file system. + The ifc_list in the superblock contains a table 32-bit + cluster indexes (relative to the start of the card). The entries + in the ifc_list point to the clusters that make up the + indirect table. The indirect table is also a table 32-bit cluster + indexes and these indexes point to the clusters that make up the + file allocation table. +

+

So assuming a cluster size of 1024, code to access an entry in the + FAT might look something like this: +

+

+fat_offset = fat_index % 256 +
+indirect_index = fat_index / 256 +
+indirect_offset = indirect_index % 256 +
+dbl_indirect_index = indirect_index / 256 +
+indirect_cluster_num = superblock.ifc_table[dbl_indirect_index] +
+indirect_cluster = read_cluster(indirect_cluster_num) +
+fat_cluster_num = indirect_cluster[indirect_offset] +
+fat_cluster = read_cluster(fat_cluster_num) +
+entry = fat_cluster[fat_offset] +

+
+ +
+

Directories

+

Directories are for the most part like regular files, except that they + contain directory entries rather than data. The root directory is + as its name suggests, the root of the card's hierarchical directory + structure. The first cluster of the root directory is given in the + rootdir_cluster field of the superblock, and subsequent clusters + (if any) can be found by following chain of linked clusters in the FAT. +

+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Directory Entry Mode Flags
MaskNameDescription
0x0001DF_READRead permission. +
0x0002DF_WRITEWrite permission. +
0x0004DF_EXECUTEExecute permission. + +
Unused
0x0008DF_PROTECTEDDirectory is copy protected. + +
Meaningful only to the browser.
0x0010DF_FILERegular file. +
0x0020DF_DIRECTORYDirectory. +
0x0040-Used internally to create directories.
0x0080-Copied?
0x0100--
0x0200O_CREATUsed to create files.
0x0400DF_0400Set when files and directories are created, +otherwise ignored.
0x0800DF_POCKETSTNPocketStation application save file. +
0x1000DF_PSXPlayStation save file. +
0x2000DF_HIDDENFile is hidden. +
0x4000--
0x8000DF_EXISTSThis entry is in use. +If this flag is clear, then the file or directory has been deleted. +
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Directory Entry (512 bytes)
OffsetNameTypeDescription
0x00modehalfSee directory mode table. +
0x04lengthwordLength in bytes if a file, or entries if a directory. +
0x08createdbyte[8]Creation time. +
0x10clusterwordFirst cluster of the file, or 0xFFFFFFFF for an empty file. +In "." entries this the first cluster of this directory's parent +directory instead. +Relative to alloc_offset. +
0x14dir_entrywordOnly in "." entries. Entry of this directory +in its parent's directory. +
0x18modifiedbyte[8]Modification time. +
0x20attrwordUser defined attribute +
0x40namebyte[32]Zero terminated name for this directory entry. + +
+ +

The first two directory entries in any directory are always two + dummy entries named "." and "..", in that order. In path names, + these two directory entries represent the current directory and the + parent directory, as they do in Unix, but they serve different + purposes in the file system. The first directory entry, the "." + entry, is used to store a link to the parent directory. The second + entry serves no purpose except as a place holder. The fields these + entries do not reflect state of the directories that they are + supposed to refer to. The length and cluster fields + are always set 0 and the modified time never changes. + +

+ +

The first directory entry in the root directory is special case. + It fills the role of the root directory's own directory entry. + Unlike the "." entry in other directories, the fields in this entry + are used and reflect the state of the root directory. In + particular, the length field contains the number of + directory entries root directory. The exception is the + cluster field which isn't used. +

+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Time of Day (8 bytes)
OffsetNameTypeDescription
0x01secbyteseconds
0x02minbyteminutes
0x03hourbytehours
0x04daybyteday of the month
0x05monthbytemonth (1-12)
0x06yearwordyear
+
+ +

The created and modified fields use the time format + given in the Time of Day table. All time stamps use the Japan + timezone (+9 UTC), regardless of the timezone the PS2 console was + configured to use. A four digit year used. +

+ +

Most of the mode flags don't serve any purpose in the structure of + the file system and only have meaning to higher level software, + like the PS2 browser. The DF_PSX flag indicates that file + was copied from a PSX memory card. If the DF_POCKETSTN flag + is set as well, the file is a PocketStation application file copied + from a PocketStation. +

+ +

Each directory entry is a massive 512 bytes long, so only two + entries fit in each 1024 cluster. The unused bytes at the end of + directory could be used for a longer name, but normally names are + truncated to only 32 bytes. File names are case sensitive, and the + characters "?", "*", and "/" along with all + ASCII control characters are illegal in files. +

+ +
+ +
+
+

Error Management

+ +

A number of strategies are employed in the file system to handle + errors are likely to occur when using memory cards. +

+ +
+

Error Correction Code

+ +

The first line defence is the use of error correcting codes to deal + with any defects in the card's flash media. The 512 byte data area + of each page is divided into 128 byte long chunks and for each + chunk a simple 20-bit Hamming code is calculated. This code allows + a single bit error in a chunk to be both detected and corrected. A + total of three bytes are used to store this 20-bit Hamming code. + The first byte contains the column (or bit-wise) parity bits, with + the even groups in the lower nibble and the odd groups in the upper + nibble. The second and third bytes contain the even and odd groups + respectively for the line (or byte-wise) parity bits. The three + ECC bytes for each of the four 128-byte chunks of a page are stored + in order in the page's spare area. +

+
+ +
+

Backup Blocks

+ +

Two complete erase blocks are reserved to deal with the possibility + of the memory card being removed by the user when data is being + saved. Since writing a single cluster to card requires erasing and + reprogramming the entire erase block the cluster belongs to, a + failure during programming could destroy more then just data being + written. The two backup blocks are used to ensure the an program + operation completes atomically. Either programming completes + successfully and no data is lost, or the erase block being + programmed is left unchanged and only the new data being written is + lost. +

+

Before programming an erase block, both backup_block1 and + backup_block2 are erase. Then backup_block1 + programmed with a backup copy of the new data for block, and the + number of the erase block being programmed is written at the start + of the backup_block2. The erase block being programmed is + then erased and programmed. Finally, backup_block2 is erased. +

+

Recovery from failed program operation caused by removal of the + memory card is implemented whenever a memory card is inserted into + the PS2. First backup_block2 is checked, if it's in a + erased state then the last programming operation completed + successfully and nothing else is done. If it's not erased, then + programming is assumed to have not been completed. The contents of + backup_block1 are then copied to the erase block given in + the first word of backup_block2. Then backup_block2 + is erased. +

+
+ +
+

Bad Sector List

+

The last defence against errors is a list of bad erase blocks kept + in the superblock. If any part of an erase block is found to be + defective it can be added to bad_block_list. No new + clusters in this block should be allocated, however clusters + already allocated in the block can still continue to be used. +

+
+ +
+

Reserved Clusters

+

The standard PS2 memory card drivers artificially reduce the number + allocatable clusters by rounding the number down to the nearest + 1000s clusters. Since clusters in blocks in the + bad_block_list don't count as against the limit, this + effectively creates a range of reserved replacement clusters. As + blocks are marked as bad, clusters in reserved range become + available and so the apparent capacity of the memory card remains + the same. This was probably implemented so that memory cards + shipped with varying numbers of bad blocks would all appear to have + the same amount of free space in the PS2 browser. +

+
+ +
+
+

See Also

+ +
+

NAND Flash Memory

+ +Micron: NAND Flash 101: An Introduction to NAND Flash and How to Design It In + to Your Next Product
+ +Wikipedia: Flash memory
+ +
+ +
+

Error Correction Codes

+ +STMicroelectronics: Error Correction Code in Single Level Cell NAND Flash memories
+ +Micron: Hamming Codes for NAND Flash Memories
+ +Hanimar: Sample code for calculating ECC values in C
+ +
+ + +
+ + + \ No newline at end of file