Commit Graph

968 Commits

Author SHA1 Message Date
JosJuice 6e6864fcbf Revert "DirectoryBlob: Use NonCopyable"
This reverts commit a7a8e467b6.
2017-08-22 16:35:37 +02:00
Leo Lam 3748384008 Merge pull request #5746 from leoetlino/disc-updates
Add support for installing disc updates from the game list
2017-08-16 19:02:42 +08:00
Michael M 1ade08c607 Fix two small lint errors 2017-08-13 19:06:10 -07:00
Leo Lam 54778303ee Merge pull request #5871 from JosJuice/read-swapped-and-shifted
DiscIO: Add a Volume::ReadSwappedAndShifted function
2017-08-11 15:58:42 +08:00
Léo Lam eb1ba4b0e9 DiscIO: Add a blob reader for Volume files 2017-08-10 23:47:18 +08:00
Léo Lam ca0438e26d WiiWad: Allow using WiiWad with more than just files
This adds a WiiWad constructor that takes a BlobReader, so that the
class can be used with more than just files from the host filesystem.

Required for using WiiWad with WADs from update partitions.
2017-08-10 23:47:18 +08:00
Lioncash 01c262fe47 Filesystem: Declare FileInfo iterator's move constructor and assignment operator noexcept
Prevents potential extra copies being created in the context of
std::move_if_noexcept.
2017-08-05 12:29:02 -04:00
JosJuice 5e3c98af1d DiscIO: Add a Volume::ReadSwappedAndShifted function
This is a fairly common operation, so let's make a utility
function for it to cut down on code duplication.
2017-08-02 22:00:51 +02:00
JosJuice 2bcccd893a DirectoryBlob: Remove unused DiscContent member 2017-08-01 22:01:34 +02:00
JosJuice f7032f8deb FileSystemGCWii: Don't add 0-size files to m_offset_file_info_cache
This is done in order to reduce the risk of files not being added
due to them having the same end offset as another file.
2017-08-01 21:58:25 +02:00
JosJuice b155c46aca DirectoryBlob: Fix reading beyond the end of the disc
There were two problems with this:

1. If the starting offset was beyond the end of the disc,
   we would dereference an invalid iterator.
2. The data beyond the end of the disc was non-deterministic.
2017-08-01 21:58:18 +02:00
JosJuice 74ada98e84 DirectoryBlob: Remove redundant assert
DiscContent::Read contains an equivalent assertion.
2017-08-01 21:58:12 +02:00
JosJuice 6b0a60d2ee DirectoryBlob: Don't add DiscContents with size 0
Having DiscContents with size 0 would mean that some DiscContents
might not get added to the std::set because of them comparing
identically to another DiscContent.

This replaces an older piece of code in WriteDirectory that ensures
that no two files have the same starting offset. (We now care about
the ending offset, not the starting offset. The new solution both
ensures that no two files have the same ending offset and that no
two files have the same starting offset.)
2017-08-01 21:58:04 +02:00
JosJuice 78fa98f559 DirectoryBlob: Change the return type of Add/CheckSizeAndAdd
Only return the information that isn't already known to the caller.
2017-08-01 21:57:54 +02:00
JosJuice 1b1a75ee3a DirectoryBlob: Lookup DiscContents by offset + size instead of offset
This simplifies DiscContentContainer::Read.
2017-08-01 21:57:48 +02:00
JosJuice d6260e02ce DirectoryBlob: Replace std::set<DiscContent> with a new class 2017-08-01 21:57:24 +02:00
JosJuice a7a8e467b6 DirectoryBlob: Use NonCopyable 2017-08-01 21:52:30 +02:00
JosJuice b6c6a04f95 DirectoryBlob: Standardize the spelling of disc
DirectoryBlob was using disk in some places and disc in other places.
2017-08-01 11:36:40 +02:00
JosJuice d0c322f6e6 DirectoryBlob: Don't silently set GC region to NTSC-J if bi2.bin is missing
We were already handling the Wii region in region.bin in this way.
2017-08-01 11:36:40 +02:00
JosJuice 3fd0d39a85 DirectoryBlob: Write partition headers properly 2017-08-01 11:36:40 +02:00
JosJuice 5fe3745750 Don't show "extra" files from DirectoryBlobs in game list
For instance, we don't want to show TGC files that might be
inside the /files/ directory of a GameCube DirectoryBlob,
and we don't want to show the /sys/main.dol files for extra
partitions of Wii DirectoryBlobs.
2017-08-01 11:36:40 +02:00
JosJuice 1ea44f425d DirectoryBlob: Change ConvertUTF8NamesToSHIFTJIS's parameter to pointer 2017-08-01 11:36:40 +02:00
JosJuice 34fd08e7eb DirectoryBlob: Support multiple partitions 2017-08-01 11:36:40 +02:00
JosJuice 955ab72b1d DirectoryBlob: Remove m_root_directory from DirectoryBlob 2017-08-01 11:36:40 +02:00
JosJuice e15b957d53 DirectoryBlob: Implement GetDataSize() 2017-08-01 11:36:40 +02:00
JosJuice d84bee3ff4 DirectoryBlob: Prepare more for supporting multiple partitions 2017-08-01 11:36:40 +02:00
JosJuice a3ee61e1a9 DirectoryBlob: Let DirectoryBlob force GC/Wii for partition 2017-08-01 11:36:40 +02:00
JosJuice fb09874f79 DirectoryBlob: Reorder functions in .cpp file
They're now grouped by which class they belong to.
2017-08-01 11:36:40 +02:00
JosJuice ba0ee3f54b DirectoryBlob: Prepare for supporting multiple partitions 2017-08-01 11:36:40 +02:00
JosJuice 953ca9cee1 DirectoryBlob: Make PadToAddress and Write32 static 2017-08-01 11:36:40 +02:00
JosJuice ab4762c4c2 DirectoryBlob: Split out setting BI2 to new function 2017-08-01 11:36:40 +02:00
JosJuice 2ea0a1a086 DirectoryBlob: Make some constants local 2017-08-01 11:36:40 +02:00
JosJuice 0d4ad0e39e DirectoryBlob: Remove m_fst_name_offset 2017-08-01 11:36:40 +02:00
JosJuice ef39b4da46 DirectoryBlob: Improve SetApploader
The code gets simpler by using IOFile instead of ReadFileToString.
Also, we now write -1 in all cases where loading fails.
2017-08-01 11:36:40 +02:00
JosJuice 31d8322c5a DirectoryBlob: Don't use member variables for addresses
Now it's clearer that SetDOL depends on SetApploader
and BuildFST depends on SetDOL.

As a side note, we now load the DOL even if there's
no apploader. (I don't think it matters whether we
do it, but it was easier to implement this way.)
2017-08-01 11:36:40 +02:00
JosJuice b28ec0b7ee DirectoryBlob: Split out setting partition table to new function 2017-08-01 11:36:40 +02:00
JosJuice c4879aa48d DirectoryBlob: Split out setting TMD and ticket to new function 2017-08-01 11:36:40 +02:00
JosJuice 5a00bda490 DirectoryBlob: Move content emplacement out of the constructor 2017-08-01 11:36:40 +02:00
JosJuice 1258d18f21 DirectoryBlob: Restructure SetApploader 2017-08-01 11:36:40 +02:00
JosJuice 8747b1b6ab DirectoryBlob: Support region.bin 2017-08-01 11:36:40 +02:00
JosJuice c73f6b6c26 DirectoryBlob: Add ReadFileToVector function 2017-08-01 11:36:40 +02:00
JosJuice 8afa230787 DirectoryBlob: Rename DISKHEADERINFO_ADDRESS to BI2_ADDRESS
Because DISKHEADERINFO is too easy confuse with DISKHEADER.

Also adding BI2_SIZE and DISKHEADER_SIZE contants.
2017-08-01 11:36:40 +02:00
JosJuice 975b2b40fa DirectoryBlob: Support bi2.bin 2017-08-01 11:36:40 +02:00
JosJuice 251828ccc0 Don't check validity twice when creating DirectoryBlob 2017-08-01 11:36:40 +02:00
JosJuice a6bbf7e21b DirectoryBlob: Don't keep DOL in memory 2017-08-01 11:36:40 +02:00
JosJuice 4cc8d3091c DirectoryBlob: Require header file (boot.bin) 2017-08-01 11:36:40 +02:00
JosJuice 71578b468d DirectoryBlob: Support tickets and TMDs 2017-08-01 11:36:40 +02:00
JosJuice 02610d4b28 DirectoryBlob: Use DISCIO instead of DVDINTERFACE for logging 2017-08-01 11:36:40 +02:00
JosJuice 3f9eec9ae4 DirectoryBlob: Use DiscContent for ReadNonPartition too
ReadPartition and ReadNonPartition are now unified into
ReadInternal, with a parameter controlling whether
the read is in a partition or not.
2017-08-01 11:36:40 +02:00
JosJuice a0fa350ca6 DirectoryBlob: Use DiscContent for everything in ReadPartition 2017-08-01 11:36:40 +02:00
JosJuice 936ef5b9dd DirectoryBlob: Let DiscContent be backed by memory instead of file 2017-08-01 11:36:40 +02:00
JosJuice e1321b131d DirectoryBlob: Create a DiscContent class for use in m_virtual_disc 2017-08-01 11:36:40 +02:00
JosJuice b56214789e DirectoryBlob: Make some functions private 2017-08-01 11:36:40 +02:00
JosJuice b8d8c7370f DirectoryBlob: Move constants to .cpp file 2017-08-01 11:36:40 +02:00
JosJuice 37c09343d8 Turn VolumeDirectory into DirectoryBlob
This lets VolumeDirectory/DirectoryBlob skip implementing
various volume functions like GetGameID, GetBanner, etc.
It also lets us view extracted discs in the game list.

This ends up breaking the boot process for Wii
DirectoryBlobs due to workarounds being removed from the
boot process, but that will be fixed later by adding
proper DirectoryBlob support for things like TMDs.

We now expect the directories to be laid out in a certain
format (based on the format that WIT uses) instead of requiring
the user to set the DVD root and apploader path settings.
2017-08-01 11:36:40 +02:00
JosJuice 5f8935932d Blob: Add interface for reading decrypted Wii data directly
This is useful for blob types that store Wii data unencrypted
(such as WIA and discs extracted to directories) so that
we don't have to waste CPU time encrypting in the blob code
just to decrypt right afterwards in the volume code.
2017-08-01 11:36:40 +02:00
Leo Lam 764c93f932 Merge pull request #5683 from JosJuice/volume-wii-defer
VolumeWii: Defer loading tickets, TMDs and keys until when needed
2017-08-01 17:11:30 +08:00
Leo Lam c207a1072d Merge pull request #5795 from JosJuice/gc-region-detection
Improve GameCube region detection
2017-07-20 14:13:11 +08:00
JosJuice be8e93f472 Ensure that Volume::GetRegion doesn't return invalid enum values 2017-07-16 14:52:17 +02:00
JosJuice db1c534b3e Add warning comments to RegionSwitch and code that uses it 2017-07-16 14:49:28 +02:00
JosJuice c8b4645039 Detect Korean GC releases as Korean when possible
According to http://scanlines16.com/en/blog-3/retro-gaming/game-cube/gamecube-korean-master-list/,
Korean GC releases use the following country codes:
- E or W for games in English
- K for games in Korean
- Unknown value for games in Japanese (my guess is that they might
  have made the discs bit-for-bit identical to Japanese releases
  because the regions of these games are already set to NTSC-J)

As far as I know, the GC has no Taiwanese releases, which is what
the W country code is used for on the Wii. But I could be wrong.

A small note: The country_byte == 'K' check in the code isn't
actually necessary as long as RegionSwitchGC returns NTSC_J
for 'K', but I thought it would be better to not rely on that.
2017-07-16 14:30:41 +02:00
JosJuice ebf0f64a01 Detect GC region based on the actual region value
The county code isn't 100% reliable for detecting the region.
For instance, some games released in Korea have the country
code E even though they're region-locked to NTSC-J consoles.

This commit makes the GC disc region detection match the Wii
disc region detection (apart from the region value being in
a different place on the disc).
2017-07-16 14:29:43 +02:00
JosJuice fe10e8aa6c Fix RegionSwitchGC for the value 'K'
Korean GameCube releases have their region set to NTSC-J.
2017-07-16 13:52:53 +02:00
JosJuice c64d668c3c Use find_first_not_of in FileSystemGCWii::FindFileInfo 2017-07-16 13:19:18 +02:00
Leo Lam b6c3479bb4 Merge pull request #5720 from JosJuice/file-metadata
FileUtil: Redesign Exists/IsDirectory/GetSize
2017-07-11 00:28:34 +02:00
JosJuice edc7a4c895 Fix reading FSTs for multi-disc games
It's a wonder that this managed to work for any discs at all...
2017-07-08 15:17:05 +02:00
JosJuice 78217532e5 VolumeWii: Defer loading tickets, TMDs and keys until when needed
This should make building the game list cache a tiny bit
faster, since we won't have to read anything from
partitions other than the game partition.
2017-07-04 21:31:24 +02:00
JosJuice 93e9740683 Drop unnecessary check from FileSystemGCWii::FindFileInfo
If the current file_info is a file, the ranged for loop will
run 0 times, and we'll reach the last line of the function.
2017-07-04 20:40:32 +02:00
JosJuice 6a31d2598e Support leading slashes in FileSystemGCWii::FindFileInfo 2017-07-04 20:40:08 +02:00
JosJuice 333b6d99d2 VolumeWii: Change the format of partition maps
Having a separate map for each type of data is a bit unnecessary.
2017-07-04 16:20:13 +02:00
JosJuice 5ca3aee00a FileUtil: Add a class for Exists/IsDirectory/GetSize
Some code was calling more than one of these functions in a row
(in particular, FileUtil.cpp itself did it a lot...), which is
a waste since it's possible to call stat a single time and then
read all three values from the stat struct. This commit adds a
File::FileInfo class that calls stat once on construction and
then lets Exists/IsDirectory/GetSize be executed very quickly.

The performance improvement mostly matters for functions that
can be handling a lot of files, such as File::ScanDirectoryTree.

I've also done some cleanup in code that uses these functions.
For instance, some code had checks like !Exists() || !IsDirectory(),
which is functionally equivalent to !IsDirectory(), and some
code was using File::GetSize even though there was an IOFile
object that the code could call GetSize on.
2017-06-29 19:07:29 +02:00
JosJuice a59edfe8cf FilesystemPanel: Put partitions in separate folders when extracting all partitions 2017-06-28 22:25:27 +02:00
JosJuice 478c4fd1f3 FileSystemGCWii: Use DiscExtractor's FST offset/size functions 2017-06-28 22:25:09 +02:00
JosJuice 1b6506f4e8 DiscExtractor: Add support for more things to extract 2017-06-28 22:24:33 +02:00
JosJuice 0b068d84d5 FilesystemPanel: Overhaul the right-click menu 2017-06-28 22:23:01 +02:00
JosJuice 39ff203c1e DiscExtractor: Don't create extra folders when extracting a folder
Before, if you extracted a directory like /map/Final/Release/,
Dolphin would create the nested folders map, Final and Release
in the output directory and put the files in Release instead of
just putting the files directly in the output directory.
2017-06-28 22:22:41 +02:00
JosJuice 6d51992648 Move ExtractDir from FilesystemPanel to DiscExtractor 2017-06-28 22:22:31 +02:00
JosJuice baf3a3b188 DiscExtractor: Don't hardcode names in ExportApploader and ExportDOL 2017-06-28 22:22:13 +02:00
JosJuice 89c901780e DiscScrubber: Deduplicate code for writing to file 2017-06-28 22:22:04 +02:00
JosJuice 1f31390b54 DiscExtractor: Improve error handling 2017-06-28 22:21:55 +02:00
JosJuice a17272c146 DiscExtractor: Minor cleanup 2017-06-28 22:21:48 +02:00
JosJuice d06b532150 DiscIO: Move parts of Filesystem to the new file DiscExtractor 2017-06-28 22:21:23 +02:00
JosJuice 23bb029250 DiscIO: Add convenience methods IsDisc and IsWii for Platform enum 2017-06-28 21:46:43 +02:00
JosJuice 7992c786b9 Fix references to CACHE_REVISION in DiscIO comments
The ISOFile.cpp one has been moved, and the GameFile.cpp one isn't
used for anything because DolphinQt2 doesn't support caching yet.
2017-06-27 21:32:14 +02:00
Léo Lam 6d821de2b9 IOS: Use a std::array for the title key instead of vector
The title key is always 16 bytes, so it doesn't make sense to make it
a std::vector.
2017-06-27 00:01:20 +02:00
JosJuice b440dbd998 FileSystemGCWii: Use case insensitive comparison in FindFileInfo
This was a regression in f49b64c. Some games seem to name the
banner file OPENING.BNR instead of opening.bnr.

Should fix https://bugs.dolphin-emu.org/issues/10354
2017-06-24 14:06:31 +02:00
Starsam80 5399311a13
NANDImporter: Improve certificate extraction 2017-06-19 01:23:58 -06:00
Leo Lam 8f460a1cda Merge pull request #5611 from JosJuice/reorganize-file-namespace
Reorganize File namespace
2017-06-15 23:28:36 +02:00
JosJuice f09ceaa735 Move IOFile to a separate file
Reduces the number of files that need to be recompiled
when making changes to FileUtil.h.
2017-06-15 21:33:50 +02:00
JosJuice 583406d900 Filesystem: Make destructors = default 2017-06-14 15:26:04 +02:00
JosJuice 0b7d2e7c68 Filesystem: Modernize variable names 2017-06-14 15:25:09 +02:00
JosJuice e3a2cd827a Filesystem: Do more validity checking in the constructor
This makes it possible to catch errors earlier so that file systems
simply fail to load instead of behaving oddly. It also makes it possible
to check for errors that weren't checkable before, like the end of a
directory being after the end of the parent directory.
2017-06-14 15:25:05 +02:00
JosJuice ee27be06d7 Filesystem: Add a cache for finding file info by disc offset
FileMonitor calls this every time a read happens, and there's no code that
only calls this a small number of times, so having a cache is worthwhile.
2017-06-14 15:25:02 +02:00
JosJuice 87916fe099 Filesystem: Replace GetFileList()
Instead of expecting callers to know how the size of directory file infos
relates to which files are in which directories, filesystems now offer a
GetRoot() method, and file infos offer a way to get their children. As
a bonus, m_FileInfoVector no longer has to be created and kept around
in RAM. Only the file info objects that actually are used are created.
2017-06-14 15:23:48 +02:00
JosJuice ee2b88ebb6 Filesystem: Store pointer to beginning of FST in file infos
Needed for the next commit.
2017-06-13 22:44:15 +02:00
JosJuice 1262f08ac1 Filesystem: Better validity checking
Now that the FST in read in the constructor, m_Valid
can be set to false when there are errors in the FST.
2017-06-13 22:44:11 +02:00
JosJuice afe2bc60f6 Filesystem: Initialize everything in constructor
Not initializing until the filesystem is used is good when
a filesystem is constructed and then never used, but nobody does that.
This simplifies the code a little and lets all methods be const.
2017-06-13 22:44:07 +02:00
JosJuice d6ee7ec32c Filesystem: Read the entire FST in one go
Instead of using lots of small scattered reads to read the FST,
only one big read is used, which is more efficient.

This also means that the FST only allocates memory once and stores all
strings close to each other - good for the CPU cache. The file info
objects use pointers to this FST memory of containing data themselves.
Keeping around the big m_FileInfoVector containing objects with only
pointers is a bit unnecessary, but that will be fixed soon.
2017-06-13 22:43:41 +02:00
JosJuice f49b64caff Filesystem: Rewrite finding file info by path for performance
Instead of calling GetPathFromFSTOffset for every file info, FindFileInfo
now only looks at names in directories that are included in the path.
For the common case of searching for "opening.bnr", this means that
only root-level files and directories have to be searched through.
2017-06-13 22:41:02 +02:00
JosJuice 7c45afecb2 Filesystem: Use file info in arguments instead of path
Some callers already have the file info, making the relatively slow
FindFileInfo calls unnecessary. Callers that didn't have the file info
will now need to call FindFileInfo on their own.
2017-06-13 22:40:57 +02:00