Commit Graph

920 Commits

Author SHA1 Message Date
JosJuice 3f9eec9ae4 DirectoryBlob: Use DiscContent for ReadNonPartition too
ReadPartition and ReadNonPartition are now unified into
ReadInternal, with a parameter controlling whether
the read is in a partition or not.
2017-08-01 11:36:40 +02:00
JosJuice a0fa350ca6 DirectoryBlob: Use DiscContent for everything in ReadPartition 2017-08-01 11:36:40 +02:00
JosJuice 936ef5b9dd DirectoryBlob: Let DiscContent be backed by memory instead of file 2017-08-01 11:36:40 +02:00
JosJuice e1321b131d DirectoryBlob: Create a DiscContent class for use in m_virtual_disc 2017-08-01 11:36:40 +02:00
JosJuice b56214789e DirectoryBlob: Make some functions private 2017-08-01 11:36:40 +02:00
JosJuice b8d8c7370f DirectoryBlob: Move constants to .cpp file 2017-08-01 11:36:40 +02:00
JosJuice 37c09343d8 Turn VolumeDirectory into DirectoryBlob
This lets VolumeDirectory/DirectoryBlob skip implementing
various volume functions like GetGameID, GetBanner, etc.
It also lets us view extracted discs in the game list.

This ends up breaking the boot process for Wii
DirectoryBlobs due to workarounds being removed from the
boot process, but that will be fixed later by adding
proper DirectoryBlob support for things like TMDs.

We now expect the directories to be laid out in a certain
format (based on the format that WIT uses) instead of requiring
the user to set the DVD root and apploader path settings.
2017-08-01 11:36:40 +02:00
JosJuice 5f8935932d Blob: Add interface for reading decrypted Wii data directly
This is useful for blob types that store Wii data unencrypted
(such as WIA and discs extracted to directories) so that
we don't have to waste CPU time encrypting in the blob code
just to decrypt right afterwards in the volume code.
2017-08-01 11:36:40 +02:00
Leo Lam 764c93f932 Merge pull request #5683 from JosJuice/volume-wii-defer
VolumeWii: Defer loading tickets, TMDs and keys until when needed
2017-08-01 17:11:30 +08:00
Leo Lam c207a1072d Merge pull request #5795 from JosJuice/gc-region-detection
Improve GameCube region detection
2017-07-20 14:13:11 +08:00
JosJuice be8e93f472 Ensure that Volume::GetRegion doesn't return invalid enum values 2017-07-16 14:52:17 +02:00
JosJuice db1c534b3e Add warning comments to RegionSwitch and code that uses it 2017-07-16 14:49:28 +02:00
JosJuice c8b4645039 Detect Korean GC releases as Korean when possible
According to http://scanlines16.com/en/blog-3/retro-gaming/game-cube/gamecube-korean-master-list/,
Korean GC releases use the following country codes:
- E or W for games in English
- K for games in Korean
- Unknown value for games in Japanese (my guess is that they might
  have made the discs bit-for-bit identical to Japanese releases
  because the regions of these games are already set to NTSC-J)

As far as I know, the GC has no Taiwanese releases, which is what
the W country code is used for on the Wii. But I could be wrong.

A small note: The country_byte == 'K' check in the code isn't
actually necessary as long as RegionSwitchGC returns NTSC_J
for 'K', but I thought it would be better to not rely on that.
2017-07-16 14:30:41 +02:00
JosJuice ebf0f64a01 Detect GC region based on the actual region value
The county code isn't 100% reliable for detecting the region.
For instance, some games released in Korea have the country
code E even though they're region-locked to NTSC-J consoles.

This commit makes the GC disc region detection match the Wii
disc region detection (apart from the region value being in
a different place on the disc).
2017-07-16 14:29:43 +02:00
JosJuice fe10e8aa6c Fix RegionSwitchGC for the value 'K'
Korean GameCube releases have their region set to NTSC-J.
2017-07-16 13:52:53 +02:00
JosJuice c64d668c3c Use find_first_not_of in FileSystemGCWii::FindFileInfo 2017-07-16 13:19:18 +02:00
Leo Lam b6c3479bb4 Merge pull request #5720 from JosJuice/file-metadata
FileUtil: Redesign Exists/IsDirectory/GetSize
2017-07-11 00:28:34 +02:00
JosJuice edc7a4c895 Fix reading FSTs for multi-disc games
It's a wonder that this managed to work for any discs at all...
2017-07-08 15:17:05 +02:00
JosJuice 78217532e5 VolumeWii: Defer loading tickets, TMDs and keys until when needed
This should make building the game list cache a tiny bit
faster, since we won't have to read anything from
partitions other than the game partition.
2017-07-04 21:31:24 +02:00
JosJuice 93e9740683 Drop unnecessary check from FileSystemGCWii::FindFileInfo
If the current file_info is a file, the ranged for loop will
run 0 times, and we'll reach the last line of the function.
2017-07-04 20:40:32 +02:00
JosJuice 6a31d2598e Support leading slashes in FileSystemGCWii::FindFileInfo 2017-07-04 20:40:08 +02:00
JosJuice 333b6d99d2 VolumeWii: Change the format of partition maps
Having a separate map for each type of data is a bit unnecessary.
2017-07-04 16:20:13 +02:00
JosJuice 5ca3aee00a FileUtil: Add a class for Exists/IsDirectory/GetSize
Some code was calling more than one of these functions in a row
(in particular, FileUtil.cpp itself did it a lot...), which is
a waste since it's possible to call stat a single time and then
read all three values from the stat struct. This commit adds a
File::FileInfo class that calls stat once on construction and
then lets Exists/IsDirectory/GetSize be executed very quickly.

The performance improvement mostly matters for functions that
can be handling a lot of files, such as File::ScanDirectoryTree.

I've also done some cleanup in code that uses these functions.
For instance, some code had checks like !Exists() || !IsDirectory(),
which is functionally equivalent to !IsDirectory(), and some
code was using File::GetSize even though there was an IOFile
object that the code could call GetSize on.
2017-06-29 19:07:29 +02:00
JosJuice a59edfe8cf FilesystemPanel: Put partitions in separate folders when extracting all partitions 2017-06-28 22:25:27 +02:00
JosJuice 478c4fd1f3 FileSystemGCWii: Use DiscExtractor's FST offset/size functions 2017-06-28 22:25:09 +02:00
JosJuice 1b6506f4e8 DiscExtractor: Add support for more things to extract 2017-06-28 22:24:33 +02:00
JosJuice 0b068d84d5 FilesystemPanel: Overhaul the right-click menu 2017-06-28 22:23:01 +02:00
JosJuice 39ff203c1e DiscExtractor: Don't create extra folders when extracting a folder
Before, if you extracted a directory like /map/Final/Release/,
Dolphin would create the nested folders map, Final and Release
in the output directory and put the files in Release instead of
just putting the files directly in the output directory.
2017-06-28 22:22:41 +02:00
JosJuice 6d51992648 Move ExtractDir from FilesystemPanel to DiscExtractor 2017-06-28 22:22:31 +02:00
JosJuice baf3a3b188 DiscExtractor: Don't hardcode names in ExportApploader and ExportDOL 2017-06-28 22:22:13 +02:00
JosJuice 89c901780e DiscScrubber: Deduplicate code for writing to file 2017-06-28 22:22:04 +02:00
JosJuice 1f31390b54 DiscExtractor: Improve error handling 2017-06-28 22:21:55 +02:00
JosJuice a17272c146 DiscExtractor: Minor cleanup 2017-06-28 22:21:48 +02:00
JosJuice d06b532150 DiscIO: Move parts of Filesystem to the new file DiscExtractor 2017-06-28 22:21:23 +02:00
JosJuice 23bb029250 DiscIO: Add convenience methods IsDisc and IsWii for Platform enum 2017-06-28 21:46:43 +02:00
JosJuice 7992c786b9 Fix references to CACHE_REVISION in DiscIO comments
The ISOFile.cpp one has been moved, and the GameFile.cpp one isn't
used for anything because DolphinQt2 doesn't support caching yet.
2017-06-27 21:32:14 +02:00
Léo Lam 6d821de2b9 IOS: Use a std::array for the title key instead of vector
The title key is always 16 bytes, so it doesn't make sense to make it
a std::vector.
2017-06-27 00:01:20 +02:00
JosJuice b440dbd998 FileSystemGCWii: Use case insensitive comparison in FindFileInfo
This was a regression in f49b64c. Some games seem to name the
banner file OPENING.BNR instead of opening.bnr.

Should fix https://bugs.dolphin-emu.org/issues/10354
2017-06-24 14:06:31 +02:00
Starsam80 5399311a13
NANDImporter: Improve certificate extraction 2017-06-19 01:23:58 -06:00
Leo Lam 8f460a1cda Merge pull request #5611 from JosJuice/reorganize-file-namespace
Reorganize File namespace
2017-06-15 23:28:36 +02:00
JosJuice f09ceaa735 Move IOFile to a separate file
Reduces the number of files that need to be recompiled
when making changes to FileUtil.h.
2017-06-15 21:33:50 +02:00
JosJuice 583406d900 Filesystem: Make destructors = default 2017-06-14 15:26:04 +02:00
JosJuice 0b7d2e7c68 Filesystem: Modernize variable names 2017-06-14 15:25:09 +02:00
JosJuice e3a2cd827a Filesystem: Do more validity checking in the constructor
This makes it possible to catch errors earlier so that file systems
simply fail to load instead of behaving oddly. It also makes it possible
to check for errors that weren't checkable before, like the end of a
directory being after the end of the parent directory.
2017-06-14 15:25:05 +02:00
JosJuice ee27be06d7 Filesystem: Add a cache for finding file info by disc offset
FileMonitor calls this every time a read happens, and there's no code that
only calls this a small number of times, so having a cache is worthwhile.
2017-06-14 15:25:02 +02:00
JosJuice 87916fe099 Filesystem: Replace GetFileList()
Instead of expecting callers to know how the size of directory file infos
relates to which files are in which directories, filesystems now offer a
GetRoot() method, and file infos offer a way to get their children. As
a bonus, m_FileInfoVector no longer has to be created and kept around
in RAM. Only the file info objects that actually are used are created.
2017-06-14 15:23:48 +02:00
JosJuice ee2b88ebb6 Filesystem: Store pointer to beginning of FST in file infos
Needed for the next commit.
2017-06-13 22:44:15 +02:00
JosJuice 1262f08ac1 Filesystem: Better validity checking
Now that the FST in read in the constructor, m_Valid
can be set to false when there are errors in the FST.
2017-06-13 22:44:11 +02:00
JosJuice afe2bc60f6 Filesystem: Initialize everything in constructor
Not initializing until the filesystem is used is good when
a filesystem is constructed and then never used, but nobody does that.
This simplifies the code a little and lets all methods be const.
2017-06-13 22:44:07 +02:00
JosJuice d6ee7ec32c Filesystem: Read the entire FST in one go
Instead of using lots of small scattered reads to read the FST,
only one big read is used, which is more efficient.

This also means that the FST only allocates memory once and stores all
strings close to each other - good for the CPU cache. The file info
objects use pointers to this FST memory of containing data themselves.
Keeping around the big m_FileInfoVector containing objects with only
pointers is a bit unnecessary, but that will be fixed soon.
2017-06-13 22:43:41 +02:00
JosJuice f49b64caff Filesystem: Rewrite finding file info by path for performance
Instead of calling GetPathFromFSTOffset for every file info, FindFileInfo
now only looks at names in directories that are included in the path.
For the common case of searching for "opening.bnr", this means that
only root-level files and directories have to be searched through.
2017-06-13 22:41:02 +02:00
JosJuice 7c45afecb2 Filesystem: Use file info in arguments instead of path
Some callers already have the file info, making the relatively slow
FindFileInfo calls unnecessary. Callers that didn't have the file info
will now need to call FindFileInfo on their own.
2017-06-13 22:40:57 +02:00
JosJuice 3d5ef948d0 Filesystem: Make FindFileInfo public 2017-06-13 22:40:53 +02:00
JosJuice 07d3a39aeb Filesystem: Replace file info's full path with name
Some callers (i.e. ISOProperties) don't want the full path, so giving them
it is unnecessary. Those that do want it can use GetPathFromFSTOffset.
Not storing full paths everywhere also saves a small bit of RAM and is
necessary for a later commit. The code isn't especially pretty right now
(callers need to use FST offsets...) but it'll become better later.
2017-06-13 22:38:31 +02:00
JosJuice 5021b4a567 Filesystem: Replace FileInfo struct with interface
GC/Wii filesystem internals shouldn't be exposed to other classes.
This change isn't especially useful by itself, but it opens up the
way for some neat stuff in the following commits.
2017-06-13 22:37:43 +02:00
JosJuice 95bc57cff3 DiscScrubber: Use FileInfo::IsDirectory instead of bit magic 2017-06-13 17:01:39 +02:00
JosJuice 2578a3553b VolumeDirectory: That's not what "4 byte" means 2017-06-09 20:00:36 +02:00
JosJuice 240cb0ca11 VolumeDirectory: Use correct endianness when writing apploader -1 2017-06-09 20:00:17 +02:00
JosJuice d2b69f963a VolumeWii: Don't set m_game_partition until we know partition is valid
Without this, we can end up in an inconsistent state where
m_game_partition is set to a partition that isn't in the
partition maps.
2017-06-09 18:39:30 +02:00
Shawn Hoffman e1a3e41bf3 fix various instances of -1 being assigned to unsigned types 2017-06-07 19:52:07 -07:00
Léo Lam 0ddb6f9328 DiscIO: Use the correct offset shift for DOL exports 2017-06-07 22:41:00 +02:00
Léo Lam 0b71b60ef0 DiscIO: Fix formatting 2017-06-07 22:40:51 +02:00
JosJuice b2af07a7b7 DiscIO: Remove C/I/S prefixes from class names
These prefixes were inconsistent with the rest of Dolphin.

I'm also renaming VolumeWiiCrypted to VolumeWii because of 1113b13.
2017-06-06 12:31:59 +02:00
JosJuice 69f01bac39 Reimplement support for unencrypted Wii discs
You may want to read the PR #2047 comments before reading this.

Dolphin attempts to support an unencrypted type of Wii discs
that apparently is identified by a 4-byte integer at 0x60
being non-zero. I don't know what discs (if any) would be
using that format, so I haven't been able to test Dolphin's
support for it, but it has probably been broken for a while.

The old implementation is very short but also strange.
In CreateVolumeFromFilename, we read a 4-byte integer from
0x60, and if it's non-zero, we create a CVolumeGC object
instead of a CVolumeWiiCrypted object. This might seem like
it makes no sense, but it presumably worked in the past
because IsVolumeWiiDisc used to check the volume type by
reading the magic word for Wii straight from the disc,
meaning that CVolumeGC objects representing unencrypted Wii
discs would be treated as Wii discs by pretty much all of
Dolphin's code except for the volume implementation code.
(It wasn't possible to simply use CVolumeWiiCrypted, because
that class only handled encrypted discs, like the name says.)

However, that stopped working as intended because of ace0607.
And furthermore, bb93336 made it even more broken by making
parts of Dolphin expect that data read from Wii discs needed
to be decrypted (rather than the volume implementation
implicitly deciding whether to decrypt when Read was called).
Disclaimer: Like I said before, I haven't been able to test
any of this because I don't have any discs that use this
unencrypted Wii disc format, so this is all theoretical.

Later, PR #2047 tried to remove Dolphin's support for
the unencrypted Wii disc format because seemingly no
discs used it, but the PR got closed without being merged.
At the end of that PR, I said that I would make a new PR
with a better implementation for the format after PR #2353
was merged. Now that PR #2353 is merged (two years later...)
and PR #5521 is merged, the new implementation was easy to
make, and here it is!

Untested.
2017-06-05 14:54:37 +02:00
JosJuice c3fa0d6edf DiscIO: Use std::optional in Volume and Blob 2017-06-05 13:26:51 +02:00
Léo Lam d9fd056803 Fix minor formatting issues
These were not caught by the lint script while it was broken.
2017-06-05 02:32:19 +02:00
JosJuice 2ce9281f77 Don't show FST size in game properties
Normal users don't care about it. In fact, people care so
little about it that the Wii implementation of it was broken
starting from when it was implemented (eb65601) to 7 years
later (e0a47c1), apparently without anyone reporting it.
2017-06-04 14:20:20 +02:00
JosJuice 703858e217 Merge pull request #5522 from leoetlino/simpler-gettitleid
DiscIO: Use std::optional for GetTitleID instead of pointer
2017-06-04 08:57:36 +02:00
Shawn Hoffman 397720a9fe might as well update yet some more pointless version numbers.. 2017-06-03 18:20:40 -07:00
Léo Lam 1a17c02d6f DiscIO: Use std::optional for GetTitleID instead of pointer
This makes the interface slightly cleaner and a bit more consistent
with the other getters. Still not fully the same, since the others
don't really handle failures with std::optional; but at least the
value is returned by value now, as opposed to having the function
take a pointer to a u64.
2017-06-03 21:56:33 +02:00
Leo Lam fbad958f03 Merge pull request #5518 from JosJuice/replace-weird-brace-thingy
Replace {{}} with {}
2017-06-03 21:16:36 +02:00
JosJuice 1c926dcae2 Replace {{}} with {}
IIRC we were only using {{}} because {} used to cause errors
on the FreeBSD buildbot.
2017-06-03 18:14:22 +02:00
Léo Lam 9d52ab5144 IOS/ES: Move ImportTicket write function
This commit moves the write function to where it should be (IOS),
especially when ES::ImportTicket() is the only place to use it.

Prevents misusing the ticket import function, and removes one unsafe
direct write to the NAND that does not go through IOS.

This also fixes the destination path: the session root is the one which
should be used for determining the ticket path, not the configured one.
2017-06-03 17:43:16 +02:00
Léo Lam 62d08e2d17 DiscIO: Fix the wrong NAND root being used
The whole NANDContentLoader stuff is truly awful and will be removed
as soon as possible.

For now, this fixes a bug that was exposed by std::optional::operator*.
2017-06-03 12:36:24 +02:00
Léo Lam 545006f666 Use std::optional for ESFormats/SharedContentMap 2017-06-03 12:36:24 +02:00
BhaaL 072c161445 upgrade to Windows SDK 10.0.15063.0
this is required for /permissive- to work, because some headers in the
Windows SDK use Microsoft extensions that are not allowed in standards mode
2017-05-28 13:37:31 +02:00
Léo Lam b5a2f34a37 Check TMD sizes using the actual constant
0x49e4 is the actual maximum TMD size (which is checked against in ES).
IsValidTMDSize is added to ESFormats to avoid duplicating the constant.
2017-05-26 11:27:43 +02:00
Pierre Bourdon d592bdd4d4 Migrate to Visual Studio 2017.
Auto-generated by the IDE, I'll trust it knows what it's doing.
2017-05-25 15:58:59 -07:00
Léo Lam cf0c4455cd Fix indirect includes 2017-05-22 10:39:36 +02:00
Silvan Jegen d8db6a7139 NANDImporter: fix printf warning
gcc complains that the printf %x formatting instruction expects an
'unsigned int' but we pass a 'size_t'.  We add the 'z' length formatting
specifier used for 'size_t'
2017-05-21 13:06:40 +02:00
JosJuice 1575020c3a VolumeWiiCrypted: Get title keys from TicketReader 2017-05-20 20:44:12 +02:00
JosJuice 26f5b53ecb VolumeWiiCrypted: Get title IDs from TicketReader 2017-05-20 20:43:31 +02:00
JosJuice 5c68a03ae4 VolumeWiiCrypted: Cache TMDs and tickets 2017-05-20 20:42:44 +02:00
Starsam80 3229cde292
NANDImporter: Add logging 2017-05-19 16:36:06 -06:00
Starsam80 618eb9f743
NANDImporter: Use a indeterminate progress meter 2017-05-19 16:31:47 -06:00
JosJuice 78effbabc6 Replace some usages of IVolume::ReadSwapped with IBlobReader::ReadSwapped
Skips needing to specify PARTITION_NONE.
2017-05-19 21:31:47 +02:00
JosJuice fcb2c5bd29 Remove CBlobBigEndianReader
We can simply put ReadSwapped directly in IBlobReader.
2017-05-19 21:23:00 +02:00
JosJuice ac5c82b36b DiscIO: Remove VolumeCreator
This file is pretty small now that it doesn't handle Wii
partitions anymore, so let's move its contents to Volume.cpp.
This is also more consistent with how blob creation works.
2017-05-19 18:33:21 +02:00
JosJuice 56b218a750 DiscIO: Simplify volume creation
Basically inlining GetDiscType into CreateVolumeFromFilename.
2017-05-19 18:18:43 +02:00
JosJuice fa06d10f4a Merge pull request #2353 from JosJuice/wii-partition-cleanup
VolumeWiiCrypted: Replace ChangePartition with a partition parameter
2017-05-16 23:06:32 +02:00
JosJuice 2bcad57225 Check file system validity before reading opening.bnr
This happened to work without any problems because the only way for a
file system to be invalid was to not have the right GC/Wii magic word
in the unencrypted area, and a volume could not be created without
having the right GC/Wii magic word there. Now that file systems read
the magic word from a partition instead, a fix is needed.
2017-05-16 22:59:01 +02:00
JosJuice 639ce6c484 FileSystemGCWii: Replace m_Wii with m_offset_shift
I replaced m_OffsetShift with m_Wii in bb93336 to support
the decrypt parameter for read functions. Doing that is no
longer necessary, so m_offset_shift is now used like before.
2017-05-16 22:58:57 +02:00
JosJuice 19b8f1c10a VolumeWiiCrypted: Replace ChangePartition with a partition parameter
By removing mutable state in VolumeWiiCrypted, this change makes
partition-related code simpler. It also gets rid of other ugly things,
like ISOProperties's "over 9000" loop that creates a list of
partitions by trying possible combinations, and DiscScrubber's
volume swapping that recreates the entire volume when it needs to
change partition.
2017-05-16 22:58:15 +02:00
JosJuice 74d84c5af2 VolumeWiiCrypted: Turn m_dataOffset into a constant 2017-05-16 22:56:53 +02:00
JosJuice 405643b902 VolumeWiiCrypted: Use std::min 2017-05-14 22:52:21 +02:00
JosJuice 5c46810a36 VolumeWiiCrypted: Store last decrypted offset instead of block number
This makes it unnecessary to reset m_last_decrypted_offset
when changing partition.
2017-05-14 22:52:21 +02:00
Léo Lam c8bffb0153 Reuse the IOS code for WAD installation
* Less code and logic duplication.

* Fixes a bug with the data dir not being created, steps being done in
  the wrong order.
2017-05-14 15:30:42 +02:00
Léo Lam afcda22da9 DiscIO: Add GetContent() for reading content from WADs
Direct access to the WAD bytes is required to read contents with proper
padding data (since they can sometimes end up being outside of the
data app section). Allowing the whole buffer to be accessed directly
would be error prone, so this commit adds GetContent() to WiiWAD
for getting raw content data by index.
2017-05-14 15:30:42 +02:00
Léo Lam bacf52384f IOS/ES: Implement DeleteTitleContent properly
Just like DeleteTitle, Using CNANDContentManager is overkill,
inefficient and useless. And it results in a few failures in
situations where a delete should just always work.

But here it gets bonus points, because it manages to actually use
the TMD for deleting contents, when IOS does none of that and just
deletes files ending with .app in the title content directory. :)
2017-05-13 00:34:58 +02:00
JosJuice 36b9e3dd35 Don't duplicate code for getting paths based on title IDs
I've seen the expression (u32)(title_id >> 32), (u32)title_id
a few more times in my life than I would've liked to...
2017-05-06 17:45:08 +02:00