Commit Graph

82 Commits

Author SHA1 Message Date
David Korth 95d3416305 VolumeVerifier: Use correct IOS filename for development discs.
Retail-signed discs use the format: IOS56-64-v5661.wad
Debug-signed discs use the format:  firmware.64.56.22.29.wad

Debug-signed discs usually have a 128 version of the firmware as well,
since some devkits have 128 MB MEM2. (Retail has 64 MB.)
2021-08-15 12:41:35 -04:00
Pierre Bourdon e149ad4f0a
treewide: convert GPLv2+ license info to SPDX tags
SPDX standardizes how source code conveys its copyright and licensing
information. See https://spdx.github.io/spdx-spec/1-rationale/ . SPDX
tags are adopted in many large projects, including things like the Linux
kernel.
2021-07-05 04:35:56 +02:00
Mateus B. Cassiano 2c598e46ab VolumeVerifier: increase problem severity for incorrectly signed TMDs 2021-05-26 05:52:21 -04:00
Pokechu22 004dfd1586 Replace uses of cassert with Common/Assert.h 2021-04-02 10:18:18 -07:00
JosJuice c0eb95481f VolumeVerifier: Align partition reads to groups
This improves the speed of verifying Wii WIA/RVZ files.
For me, the verification speed for LZMA2-compressed files
has gone from 11-12 MiB/s to 13-14 MiB/s.

One thing VolumeVerifier does to achieve parallelism is to
compute hashes for one chunk of data while reading the next
chunk of data. In master, when reading data from a Wii
partition, each such chunk is 32 KiB. This is normally fine,
but with WIA and RVZ it leads to rather lopsided read times
(without the compute times being lopsided): The first 32 KiB
of each 2 MiB takes a long time to read, and the remaining
part of the 2 MiB can be read nearly instantly. (The WIA/RVZ
code has to read the entire 2 MiB in order to compute hashes
which appear at the beginning of the 2 MiB, and then caches
the result afterwards.) This leads to us at times not doing
much reading and at other times not doing much computation.
To improve this, this change makes us use 2 MiB chunks
instead of 32 KiB chunks when reading from Wii partitions.

(block = 32 KiB, group = 2 MiB)
2021-03-22 21:07:01 +01:00
JosJuice bbacefeb75 VolumeVerifier: Handle contents overlapping
This can't actually happen in practice due to how WAD files work,
but it's very easy to add support for thanks to the last commit,
so we might as well add support for it.
2021-03-22 20:05:11 +01:00
JosJuice 10e1acf25c VolumeVerifier: Handle overlapping blocks more efficiently
The performance gains of doing this aren't too important since you
normally wouldn't run into any disc image that has overlapping blocks
(which by extension means overlapping partitions), but this change also
lets us get rid of things like VolumeVerifier's mutex that used to
exist just for the sake of handling overlapping blocks.
2021-03-22 20:05:11 +01:00
Léo Lam f44f20560d
Merge pull request #9573 from JosJuice/volumeverifier-cancel-crash
VolumeVerifier: Fix potential crash when cancelling
2021-03-16 11:10:47 +01:00
JosJuice 7d570f1edb DiscIO: Move magic constants for discs to DiscUtils 2021-03-10 00:16:37 +01:00
JosJuice b14bf82732 DiscIO: Move some code from VolumeVerifier to DiscUtils 2021-03-10 00:16:12 +01:00
JosJuice 49ccc77ebb DiscIO: Move some code from DiscExtractor to new file DiscUtils 2021-03-09 20:34:24 +01:00
JosJuice 96ebf01ea8 VolumeVerifier: Fix potential crash when cancelling
The async operations may contain references to class members, so
any running async operations must end before destroying the class.
2021-03-07 13:56:06 +01:00
Léo Lam 522cb6b137
IOS: Use less ambiguous names for classes
Some of the device names can be ambiguous and require fully or partly
qualifying the name (e.g. IOS::HLE::FS::) in a somewhat verbose way.

Additionally, insufficiently qualified names are prone to breaking.
Consider the example of IOS::HLE::FS:: (namespace) and
IOS::HLE::Device::FS (class). If we use FS::Foo in a file that doesn't
know about the class, everything will work fine. However, as soon as
Device::FS is declared via a header include or even just forward
declared, that code will cease to compile because FS:: now resolves
to Device::FS if FS::Foo was used in the Device namespace.

It also leads to having to write IOS::ES:: to access ES types and
utilities even for code that is already under the IOS namespace.

The fix for this is simple: rename the device classes and give them
a "device" suffix in their names if the existing ones may be ambiguous.
This makes it clear whether we're referring to the device class or to
something else.

This is not any longer to type, considering it lets us get rid of the
Device namespace, which is now wholly unnecessary.

There are no functional changes in this commit.

A future commit will fix unnecessarily qualified names.
2021-02-12 21:40:31 +01:00
Shawn Hoffman 84128d9532 rename Common/File to Common/IOFile 2021-01-27 14:29:48 -08:00
JosJuice 9a01c3fb9f Use positional arguments in all translatable fmt strings
We want to use positional arguments in translatable strings
that have more than one argument so that translators can change
the order of them, but the question is: Should we also use
positional arguments in translatable strings with only one
argument? I think it makes most sense that way, partially
so that translators don't even have to be aware of the
non-positional syntax and partially because "translatable
strings use positional arguments" is an easier rule for us
to remember than "transitional strings which have more than
one argument use positional arguments". But let me know if
you have a different opinion.
2020-11-16 13:28:11 +01:00
Lioncash 689eec5304 DiscIO: Make use of fmt-capable panic alerts
Migrates the DiscIO code over to fmt.
2020-11-11 01:09:42 -05:00
Léo Lam 285b926e92
Merge pull request #8782 from Pokechu22/missing-serial-version-data
Explain how to fix "Serial and/or version data is missing"
2020-10-23 20:44:06 +02:00
Lioncash e93fbb7c5e DiscIO: Migrate logging over to fmt
Eliminates quite a bit of the PRI* macros used for handling 64-bit
values.
2020-10-22 15:41:42 -04:00
Pokechu22 88bc32b638 Explain how to fix "Serial and/or version data is missing" 2020-10-21 15:39:26 -07:00
JosJuice bb03280462 VolumeVerifier: Make "no valid data in partition" hide some other errors
Not much use in knowing that the H3 hashes are wrong when the
partition consists of complete garbage.
2020-10-19 21:15:49 +02:00
JosJuice 1084cfec1a Add helper function for fmt::format(GetStringT(... 2020-10-19 14:36:08 +02:00
JosJuice d100c1dc37 DiscIO: Make use of fmt where applicable
Once nice benefit of fmt is that we can use positional arguments
in localizable strings. This a feature which has been
requested for the Korean translation of strings like
"Errors were found in %zu blocks in the %s partition."
and which will no doubt be useful for other languages too.
2020-10-19 14:30:47 +02:00
JosJuice 402643fe4c VolumeVerifier: Call CheckDiscSize from Finish 2020-09-23 11:19:03 +02:00
JosJuice 4f62960fb1 VolumeVerifier: Make high severity "too small" hide low severity "too small" 2020-09-21 21:00:04 +02:00
JMC47 e7e5175606
Merge pull request #8861 from JosJuice/netplay-hash
Make netplay's "same game" check more robust
2020-09-06 17:14:08 -04:00
JosJuice fc6c1931fa VolumeVerifier: Adjust problem descriptions for new netplay check
The new hash check catches essentially all desync problems
that VolumeVerifier can catch, so from the user's perspective,
such problems will result in Dolphin refusing to start the
game on netplay rather than actually getting a desync.
2020-08-19 17:21:38 +02:00
JosJuice 7ef0bc0359 VolumeVerifier: Split TMD error from ticket error for WADs
When I first made VolumeVerifier, I figured that the distinction
between an unsigned ticket and an unsigned TMD was a technical
detail that users would have no reason to care about. However,
while this might be true for discs, it isn't equally true for
WADs, due to the widespread practice of fakesigning tickets to
set the console ID to 0. This practice does not require
fakesigning the TMD (though apparently people do it anyway,
at least sometimes...), and the presence of a correctly signed
TMD is a useful indicator that the contents have not been
tampered with, even if the ticket isn't correctly signed.
2020-08-08 21:15:51 +02:00
JosJuice 2e8c5b4521 DolphinQt: Show a warning when launching an NKit disc image
It is my opinion that nobody should use NKit disc images without
being aware of the drawbacks of them. Since it seems like almost
nobody who is using NKit disc images knows what NKit is (hmm, now
how could that have happened...?), I am adding a warning to Dolphin
so that you can't run NKit disc images without finding out about the
drawbacks. In case someone really does want to use NKit disc images,
the warning has a "Don't show this again" option. Unfortunately, I
can't retroactively add the warning where it's most needed:
in Dolphin 5.0, which does not support Wii NKit disc images.
2020-07-29 21:05:57 +02:00
JosJuice 74f74ac928 DiscIO: Use Volume::IsDatelDisc in VolumeVerifier
I forgot to do this in the PR where I added Volume::IsDatelDisc.
2020-07-04 20:24:26 +02:00
JosJuice fb922781e7 RedumpVerifier: Treat Datel game IDs as empty
Replacement for the reverted commit
f9705fd117.
2020-07-04 15:00:02 +02:00
degasus f8f592c345 Externals: Update minizip search path. 2020-04-29 12:41:36 +02:00
JosJuice 5b722b775a VolumeVerifier: Treat IOS9 as always present
Fixes a false positive for Mario Party 8.
2020-04-28 23:40:01 +02:00
JMC47 116cef572b
Merge pull request #8738 from JosJuice/convert-dialog
Replace the compress/uncompress actions with a convert dialog
2020-04-28 12:37:17 -04:00
JosJuice bacf0d629d VolumeVerifier: Show desync warning for dual layer discs too 2020-04-25 19:47:08 +02:00
JosJuice 8a9597e32e DiscIO: Allow converting from formats other than ISO and GCZ
The constant DESIRED_BUFFER_SIZE was determined by multiplying the
old hardcoded value 32 with the default GCZ block size 16 KiB.
Not sure if it actually is the best value, but it seems fine.
2020-04-24 15:10:35 +02:00
Tilka 103b3abf66 Merge pull request #8621 from JosJuice/volumeverifier-invalid-partition
Fix VolumeVerifier not showing a problem for invalid partitions
2020-02-10 09:33:29 +00:00
JosJuice 829f3cff6f VolumeVerifier: Ignore invalid partitions in GetBiggestReferencedOffset
Otherwise GetBiggestReferencedOffset might treat garbage data as a valid
large offset, making Dolphin incorrectly say that the disc is too small.
2020-02-09 19:05:44 +01:00
JosJuice 07df6597a5 Fix VolumeVerifier not showing a problem for invalid partitions 2020-02-09 18:58:15 +01:00
JosJuice f8355d0f82 VolumeVerifier: Don't return early when file is too small
Not sure why that was there.
2020-02-02 17:09:58 +01:00
JosJuice e449d23929 VolumeVerifier: Don't show an assert for files that are too small 2020-02-02 17:09:58 +01:00
JosJuice de26fec0af VolumeVerifier: Report read errors to the user 2020-01-25 20:21:12 +01:00
Léo Lam ad75215bb0 Fix several warnings
A small, nonexhaustive set of warning fixes. The DiscIO Volume change
is a workaround for a GCC bug [1] that causes returning an unengaged
std::optional to emit annoying -Wmaybe-uninitialized warnings.
This last change alone fixes pages upon pages of warnings since
Volume.h is included from several files.

-Wstringop-truncation is another irrelevant warning for us, but
unfortunately there seems to be no way to disable it without
adding ugly pragmas wherever the warning appears.
2020-01-04 12:11:39 +01:00
David Korth afe2e7de0f VolumeVerifier.cpp: Verify that the string arrays are sorted before using std::binary_sort().
Debug builds only, since this is wrapped in assert().
2019-12-29 23:42:55 -05:00
David Korth d660aba20f VolumeVerifier.cpp: Use arrays of string_view objects instead of strings.
string_view is a thin wrapper around C strings, so it's more efficient
for constant strings than C++ strings.

The unordered_set<> also adds extra runtime overhead. For small arrays,
a simple linear search works. For larger arrays, std::binary_search()
works better than linear but without the unordered_set<> overhead.

ShouldBeDualLayer(): Removed a duplicate "SK8X52" entry.
2019-12-29 23:42:55 -05:00
JosJuice 59633f5309 VolumeVerifier: Detect broken Super Paper Mario
https://bugs.dolphin-emu.org/issues/11900
2019-11-27 18:26:22 +01:00
JosJuice 4e713238d9 VolumeVerifier: Fix a copypaste error
This was completely wrong, but I guess it happened to work correctly
anyway due to timing reasons (MD5 is faster than SHA-1 + AES).
2019-11-22 13:39:04 +01:00
JosJuice 56d37d773b RedumpVerifier: Show an error when datfile lacks serials or versions
This happens if someone manually downloads a regular datfile from
redump.org and puts it where Dolphin stores datfiles. Dolphin needs
"special" datfiles that contain fields for serials and versions.

Before this change, all discs (except Datel discs) would show up as
"Unknown disc" when using a regular datfile.
2019-11-02 16:49:20 +01:00
Mat M 78fad0aafa
Merge pull request #8443 from JosJuice/redumpverifier-datel-wii
RedumpVerifier: Fix handling of Datel Wii disc serials
2019-10-30 05:37:08 -04:00
JosJuice 42ec861469 RedumpVerifier: Fix handling of Datel Wii disc serials
GC Datel discs have empty serials, but Wii Datel discs have
serials starting with DS followed by some digits.
2019-10-30 10:11:24 +01:00
JosJuice 49977446dd RedumpVerifier: Don't crash on missing hyphen in serial 2019-10-30 09:58:37 +01:00