Commit Graph

1298 Commits

Author SHA1 Message Date
Léo Lam 83c5446d85
Fix static initialisation order fiasco issue for Version variables
Fixes a crash that could occur if the static constructor function for
the MainSettings.cpp TU happened to run before the variables in
Common/Version.cpp are initialised. (This is known as the static
initialisation order fiasco.)

By using wrapper functions, those variables are now guaranteed to be
constructed on first use.
2022-01-14 00:04:22 +01:00
Pokechu22 29d6dd609c Fix non-constexpr format strings 2022-01-13 11:11:08 -08:00
Léo Lam 7abb501e0a
Merge pull request #10334 from AdmiralCurtiss/json-maker
Core: Add a Maker field for GameModDescriptors.
2022-01-03 01:43:02 +01:00
Admiral H. Curtiss 2906889804
Core: Add a Maker field for GameModDescriptors. 2022-01-02 06:06:53 +01:00
Pokechu22 0c19f895d3 Replace remaining uses of zlib crc32 with Common/Hash.h 2022-01-01 10:36:38 -08:00
Admiral H. Curtiss e54657254a
Core: Make format of D_WIIROOT_IDX consistent with the rest of the user directories. 2021-12-25 20:21:32 +01:00
Léo Lam 185475fe03
Merge pull request #10187 from AdmiralCurtiss/json-gamelist
Support for GameModDescriptor files in Game List.
2021-12-14 11:08:38 +01:00
Admiral H. Curtiss a2a39cfcfb
Core: Add ability to serialize a GameModDescriptor to json. 2021-12-12 21:26:09 +01:00
Admiral H. Curtiss da161faff4
GameList: Show game mod descriptor .json files in game list. 2021-12-12 21:26:09 +01:00
Admiral H. Curtiss b50861ea44
Core: Require game mod descriptor json to self-identify as one. 2021-12-12 21:26:09 +01:00
Admiral H. Curtiss ea17dd8dba
RiivolutionParser: Create config xml directory if it does not exist yet. 2021-12-12 21:24:36 +01:00
Pokechu22 2025763420 Treewide: Adjust order of includes 2021-12-10 14:49:57 -08:00
ssdsnake 1aa8a4d46f DolphinTool: CLI utility interface and disc image tools 2021-12-03 15:40:19 -06:00
Admiral H. Curtiss 0fc563ee2e
RiivolutionPatcher: Use case-insensitive filename comparison when searching for files in a folder patch. 2021-11-20 01:15:18 +01:00
JosJuice 96a6d6fd95
Merge pull request #10184 from JosJuice/android-riivolution
Android: Allow starting game with Riivolution patches from the GUI
2021-10-31 23:44:54 +01:00
Léo Lam fb8b9f3251
Merge pull request #10194 from AdmiralCurtiss/riivolution-edge-cases
Fix some Riivolution file patching edge cases.
2021-10-31 21:24:03 +01:00
Sepalani e51119c4ce HLE: Fix hooks overlapping Riivolution patches 2021-11-01 00:07:14 +04:00
Admiral H. Curtiss e91e04fe1e
RiivolutionPatcher: More closely match behavior of filename searching file patches to actual Riivolution. 2021-10-31 05:47:39 +01:00
Admiral H. Curtiss 335a1b5cb5
RiivolutionPatcher: More closely match behavior of folder patches to actual Riivolution. 2021-10-31 05:47:39 +01:00
JosJuice 34021b5ebc Android: Allow starting game with Riivolution patches from the GUI 2021-10-30 23:24:36 +02:00
Admiral H. Curtiss 4f82810845
RiivolutionPatcher: Use FileInfo instead of IOFile when only checking for filesize. 2021-10-24 00:09:09 +02:00
Admiral H. Curtiss fe7f3a6f26
RiivolutionPatcher: Protect against three or more dots in path traversal. 2021-10-24 00:09:09 +02:00
Admiral H. Curtiss 22c6f2fe3b
RiivolutionPatcher: Handle the possibility of the FST already containing a main.dol file. 2021-10-24 00:09:09 +02:00
Admiral H. Curtiss ba3373b476
RiivolutionPatcher: Modify memory patching logic to be more accurate to actual Riivolution. 2021-10-24 00:09:09 +02:00
Admiral H. Curtiss 2be62d94fc
RiivolutionParser: Add code to read and write the Riivolution config XMLs that remember the last selection. 2021-10-24 00:09:08 +02:00
Admiral H. Curtiss 6394960f54
Core: Add ability to specify and launch a riivolution-modded game via a .json file. 2021-10-24 00:09:07 +02:00
Admiral H. Curtiss fe242f79ee
Core: Implement Wii NAND path redirects for Riivolution savegame patches. 2021-10-24 00:09:07 +02:00
Admiral H. Curtiss 588c31acb6
RiivolutionPatcher: Ignore last two bits of offset in file patches to match console behavior. 2021-10-24 00:09:07 +02:00
Admiral H. Curtiss 6ec4af7ea4
RiivolutionPatcher: Load external files with a layer of indirection during the patching process to properly resolve the paths given in the XML.
This also may eventually allow loading patches from sources other than the 1:1 expected file structure host file system, such as memory or an archive file.
2021-10-24 00:09:07 +02:00
Admiral H. Curtiss 783b180dc8
Core: Add RiivolutionPatcher to apply Riivolution-style patches to a game's file system and memory. 2021-10-24 00:09:06 +02:00
Admiral H. Curtiss e26b59bab3
Core: Add RiivolutionParser to parse a Riivolution XML. 2021-10-24 00:09:06 +02:00
Admiral H. Curtiss 09fc39e2e5
DiscIO: Add CreateDisc/WAD/Volume() overloads that take a BlobReader directly. 2021-10-24 00:09:05 +02:00
Admiral H. Curtiss e3f1de023f
DiscIO/DirectoryBlob: Add a callback that allows patching the main.dol and FST of the game partition during blob construction. 2021-10-24 00:09:05 +02:00
Admiral H. Curtiss 00ef9f2b4f
DiscIO/DirectoryBlob: Allow constructing a DirectoryBlobReader from a VolumeDisc. 2021-10-24 00:09:05 +02:00
Admiral H. Curtiss 3a72a39efd
DiscIO/DirectoryBlob: Allow constructing a DirectoryBlobPartition from a VolumeDisc. 2021-10-24 00:09:00 +02:00
Admiral H. Curtiss b1802f6daa
DiscIO/DirectoryBlob: Add a content source representing a run of padding bytes. 2021-10-23 23:51:23 +02:00
Admiral H. Curtiss f8611f7139
DiscIO/DirectoryBlob: Add a content source that reads data from a DiscIO::Volume. 2021-10-23 16:51:45 +02:00
Admiral H. Curtiss b7a9cc37b1
DiscIO/DirectoryBlob: Add ability to have an offset for a partition ContentSource. 2021-10-23 16:51:45 +02:00
Admiral H. Curtiss a14436fe3f
DiscIO/DirectoryBlob: Add ability to have a start-of-file offset for a file ContentSource. 2021-10-23 16:51:45 +02:00
Admiral H. Curtiss 885e6690c5
DiscIO/DirectoryBlob: Add explanations for DiscContent members. 2021-10-23 16:51:44 +02:00
Admiral H. Curtiss b988ab4441
DiscIO/DirectoryBlob: Consolidate functions that take data for ContentSource to just take a ContentSource. 2021-10-23 16:51:44 +02:00
Admiral H. Curtiss 7f57c24172
DiscIO/DirectoryBlob: Rename the vector-reference variant of DiscContentContainer::Add() to AddReference() to make it clearer at call sites that the given vector must remain in memory. 2021-10-23 16:51:44 +02:00
Pokechu22 78bfd25964 Fix all uninitialized variable warnings (C26495) 2021-10-13 12:32:16 -07:00
Admiral H. Curtiss 6684e26ae2
DiscIO/DirectoryBlob: Fix swapped DiscContent comparison operators. 2021-09-21 08:01:21 +02:00
Admiral H. Curtiss ad410009bb
Core/DiscIO: Extract disc and partition constants to DiscUtils.h. 2021-09-19 22:43:52 +02:00
Admiral H. Curtiss 06d41c53ce DiscIO/DirectoryBlob: Handle reads between files. 2021-09-14 07:09:55 +02:00
Léo Lam 815b590dfe
Update clang-format to version 12
12 is the latest release and various editors (VS, VSCode and CLion)
ship with clang-format 12 by default.
2021-08-30 00:20:18 +02:00
Tilka 0f628ff882
Merge pull request #10028 from GerbilSoft/bugfix/RVT-firmware-filenames
VolumeVerifier: Use correct IOS filename for development discs.
2021-08-16 20:47:07 +01:00
David Korth 95d3416305 VolumeVerifier: Use correct IOS filename for development discs.
Retail-signed discs use the format: IOS56-64-v5661.wad
Debug-signed discs use the format:  firmware.64.56.22.29.wad

Debug-signed discs usually have a 128 version of the firmware as well,
since some devkits have 128 MB MEM2. (Retail has 64 MB.)
2021-08-15 12:41:35 -04:00
Shawn Hoffman 4ef87194a4 revert workaround for msvc arm64 ICE in WIABlob 2021-08-14 03:57:53 -07:00
JosJuice 5fe23ccc3e DiscIO: Fix broken workaround for MSVC ARM64 ICE
The workaround added in 30f9f31 caused a regression where Dolphin
incorrectly replaced runs of one byte with runs of another byte
when writing WIA and RVZ files. ReuseID::operator< was always
returning false unless the ReuseIDs being compared had different
partition keys, which caused std::map<ReuseID, GroupEntry>
to treat all ReuseIDs with the same partition key as equal.
2021-07-24 13:29:03 +02:00
Léo Lam 39e14b6b5f
Merge pull request #9854 from JosJuice/wia-ice
DiscIO: Store partition key directly in ReuseID
2021-07-06 13:05:09 +02:00
Pierre Bourdon f4e34703c0
licensing: convert "public domain" to CC0 1.0
Public domain does not have an internationally agreed upon definition,
As such it's generally preferred to use an extremely liberal license,
which can explicitly list the rights granted by the copyright holder.
The CC0 license is the usual choice here.

This "relicensing" is done without hunting down copyright holders, since
it is presumed that their release of this work into the public domain
authorizes us to redistribute this code under any other license of our
choosing.
2021-07-05 04:43:55 +02:00
Pierre Bourdon e149ad4f0a
treewide: convert GPLv2+ license info to SPDX tags
SPDX standardizes how source code conveys its copyright and licensing
information. See https://spdx.github.io/spdx-spec/1-rationale/ . SPDX
tags are adopted in many large projects, including things like the Linux
kernel.
2021-07-05 04:35:56 +02:00
JosJuice eafa2784d6 DiscIO: Store partition key directly in ReuseID
This lets us reuse blocks in the case where different partition
entries use the same key (which probably isn't very common).
2021-06-30 09:06:20 +02:00
Shawn Hoffman 30f9f31a69 DiscIO: Add workaround for MSVC ARM64 ICE 2021-06-29 10:24:09 +02:00
Mateus B. Cassiano 2c598e46ab VolumeVerifier: increase problem severity for incorrectly signed TMDs 2021-05-26 05:52:21 -04:00
Léo Lam 1a9e72c9bb
DiscIO: Use MathUtil::SaturatingCast 2021-04-06 23:27:23 +02:00
Léo Lam e89ae71cec
Merge pull request #9388 from JosJuice/default-locale
Set console's default language/country/region based on computer settings
2021-04-06 12:34:58 +02:00
Pokechu22 004dfd1586 Replace uses of cassert with Common/Assert.h 2021-04-02 10:18:18 -07:00
JosJuice a1000afacc Add utility functions for GameCube language codes 2021-03-27 09:46:46 +01:00
Léo Lam 8d2b0fff8a
Merge pull request #9545 from leoetlino/es-launch-timings
IOS: Improve timing accuracy for title launches (both ARM and PPC)
2021-03-27 01:47:28 +01:00
JosJuice c0eb95481f VolumeVerifier: Align partition reads to groups
This improves the speed of verifying Wii WIA/RVZ files.
For me, the verification speed for LZMA2-compressed files
has gone from 11-12 MiB/s to 13-14 MiB/s.

One thing VolumeVerifier does to achieve parallelism is to
compute hashes for one chunk of data while reading the next
chunk of data. In master, when reading data from a Wii
partition, each such chunk is 32 KiB. This is normally fine,
but with WIA and RVZ it leads to rather lopsided read times
(without the compute times being lopsided): The first 32 KiB
of each 2 MiB takes a long time to read, and the remaining
part of the 2 MiB can be read nearly instantly. (The WIA/RVZ
code has to read the entire 2 MiB in order to compute hashes
which appear at the beginning of the 2 MiB, and then caches
the result afterwards.) This leads to us at times not doing
much reading and at other times not doing much computation.
To improve this, this change makes us use 2 MiB chunks
instead of 32 KiB chunks when reading from Wii partitions.

(block = 32 KiB, group = 2 MiB)
2021-03-22 21:07:01 +01:00
JosJuice bbacefeb75 VolumeVerifier: Handle contents overlapping
This can't actually happen in practice due to how WAD files work,
but it's very easy to add support for thanks to the last commit,
so we might as well add support for it.
2021-03-22 20:05:11 +01:00
JosJuice 10e1acf25c VolumeVerifier: Handle overlapping blocks more efficiently
The performance gains of doing this aren't too important since you
normally wouldn't run into any disc image that has overlapping blocks
(which by extension means overlapping partitions), but this change also
lets us get rid of things like VolumeVerifier's mutex that used to
exist just for the sake of handling overlapping blocks.
2021-03-22 20:05:11 +01:00
JosJuice d1946aded6 Avoid using panic alerts in DiscIO
Panic alerts in DiscIO can potentially be very annoying since
large amounts of them can pop up when loading the game list
if you have some particularly weird files in your game list.

This was a much bigger problem back in 5.0 with its
"Tried to decrypt data from a non-Wii volume" panic alert, but
I figured I would take it all the way and remove the remaining
panic alerts that can show up when loading the game list.

I have exempted uses of ASSERT/ASSERT_MSG since they indicate
a bug in Dolphin rather than a malformed file.
2021-03-20 12:58:54 +01:00
Léo Lam f44f20560d
Merge pull request #9573 from JosJuice/volumeverifier-cancel-crash
VolumeVerifier: Fix potential crash when cancelling
2021-03-16 11:10:47 +01:00
Léo Lam b7f931fc6b
Merge pull request #9570 from JosJuice/wia-partial-exception-list
DiscIO: Fix reading certain WIA chunks with many exceptions
2021-03-16 11:10:12 +01:00
JosJuice 7d570f1edb DiscIO: Move magic constants for discs to DiscUtils 2021-03-10 00:16:37 +01:00
JosJuice b14bf82732 DiscIO: Move some code from VolumeVerifier to DiscUtils 2021-03-10 00:16:12 +01:00
JosJuice 49ccc77ebb DiscIO: Move some code from DiscExtractor to new file DiscUtils 2021-03-09 20:34:24 +01:00
JosJuice 14bfc0be78 DiscIO: Fix reading certain WIA chunks with many exceptions
The loop in WIARVZFileReader::Chunk::Read could terminate
prematurely if the size argument was smaller than the size
of an exception list which had only been partially loaded.
2021-03-07 14:14:45 +01:00
JosJuice 96ebf01ea8 VolumeVerifier: Fix potential crash when cancelling
The async operations may contain references to class members, so
any running async operations must end before destroying the class.
2021-03-07 13:56:06 +01:00
Léo Lam aef0760efe
IOS/ES: Emulate /sys/launch.sys for more accurate timings
Also gets rid of one static variable
2021-03-04 18:41:13 +01:00
Léo Lam 522cb6b137
IOS: Use less ambiguous names for classes
Some of the device names can be ambiguous and require fully or partly
qualifying the name (e.g. IOS::HLE::FS::) in a somewhat verbose way.

Additionally, insufficiently qualified names are prone to breaking.
Consider the example of IOS::HLE::FS:: (namespace) and
IOS::HLE::Device::FS (class). If we use FS::Foo in a file that doesn't
know about the class, everything will work fine. However, as soon as
Device::FS is declared via a header include or even just forward
declared, that code will cease to compile because FS:: now resolves
to Device::FS if FS::Foo was used in the Device namespace.

It also leads to having to write IOS::ES:: to access ES types and
utilities even for code that is already under the IOS namespace.

The fix for this is simple: rename the device classes and give them
a "device" suffix in their names if the existing ones may be ambiguous.
This makes it clear whether we're referring to the device class or to
something else.

This is not any longer to type, considering it lets us get rid of the
Device namespace, which is now wholly unnecessary.

There are no functional changes in this commit.

A future commit will fix unnecessarily qualified names.
2021-02-12 21:40:31 +01:00
Shawn Hoffman 500a694ca8 msbuild: bundle all dolphin "core" code into single library 2021-01-27 14:29:49 -08:00
Shawn Hoffman 84128d9532 rename Common/File to Common/IOFile 2021-01-27 14:29:48 -08:00
JosJuice b43f7c85cc DiscIO: Fix recursive directory extraction
https://bugs.dolphin-emu.org/issues/12331
2020-12-03 21:13:53 +01:00
JosJuice 9a01c3fb9f Use positional arguments in all translatable fmt strings
We want to use positional arguments in translatable strings
that have more than one argument so that translators can change
the order of them, but the question is: Should we also use
positional arguments in translatable strings with only one
argument? I think it makes most sense that way, partially
so that translators don't even have to be aware of the
non-positional syntax and partially because "translatable
strings use positional arguments" is an easier rule for us
to remember than "transitional strings which have more than
one argument use positional arguments". But let me know if
you have a different opinion.
2020-11-16 13:28:11 +01:00
Léo Lam ec5313fe24
Merge pull request #9204 from fbastos1/master
fix emulator issue #12139
2020-11-12 14:43:35 +01:00
Lioncash 689eec5304 DiscIO: Make use of fmt-capable panic alerts
Migrates the DiscIO code over to fmt.
2020-11-11 01:09:42 -05:00
Felipe d7fa75ef64 modify file export to create folder in OS when explicitly extracting a directory 2020-10-29 15:20:46 -04:00
Léo Lam 285b926e92
Merge pull request #8782 from Pokechu22/missing-serial-version-data
Explain how to fix "Serial and/or version data is missing"
2020-10-23 20:44:06 +02:00
Lioncash e93fbb7c5e DiscIO: Migrate logging over to fmt
Eliminates quite a bit of the PRI* macros used for handling 64-bit
values.
2020-10-22 15:41:42 -04:00
Pokechu22 88bc32b638 Explain how to fix "Serial and/or version data is missing" 2020-10-21 15:39:26 -07:00
Léo Lam 70ab3bf7b3
Merge pull request #9025 from cyrozap/fix-build-with-older-cmake
Fix builds with system LZMA when using versions of CMake before 3.14
2020-10-20 14:51:15 +02:00
JosJuice 5b6f604d90 DiscIO: Make WiiEncryptionCache moveable
Fixes the following warning:

../../../../../../Core\DiscIO/DirectoryBlob.h:156:3: warning: explicitly defaulted move constructor is implicitly deleted [-Wdefaulted-function-deleted]
  DirectoryBlobReader(DirectoryBlobReader&&) = default;
  ^
../../../../../../Core\DiscIO/DirectoryBlob.h:205:22: note: move constructor of 'DirectoryBlobReader' is implicitly deleted because field 'm_encryption_cache' has a deleted move constructor
  WiiEncryptionCache m_encryption_cache;
                     ^
2020-10-20 11:33:38 +02:00
JosJuice bb03280462 VolumeVerifier: Make "no valid data in partition" hide some other errors
Not much use in knowing that the H3 hashes are wrong when the
partition consists of complete garbage.
2020-10-19 21:15:49 +02:00
JosJuice cb14b65aad CMake: Fix build errors exposed by making common dependent on fmt 2020-10-19 14:36:09 +02:00
JosJuice 1084cfec1a Add helper function for fmt::format(GetStringT(... 2020-10-19 14:36:08 +02:00
JosJuice d100c1dc37 DiscIO: Make use of fmt where applicable
Once nice benefit of fmt is that we can use positional arguments
in localizable strings. This a feature which has been
requested for the Korean translation of strings like
"Errors were found in %zu blocks in the %s partition."
and which will no doubt be useful for other languages too.
2020-10-19 14:30:47 +02:00
Sergei Trofimovich af2a881a85 Source/Core/DiscIO/WIACompression.cpp: add missing <limits> include
Noticed missing include as a build failure on gcc-11:

```
[ 26%] Building CXX object Source/Core/DiscIO/CMakeFiles/discio.dir/WIACompression.cpp.o
../../../../Source/Core/DiscIO/WIACompression.cpp: In lambda function:
../../../../Source/Core/DiscIO/WIACompression.cpp:170:31: error: 'numeric_limits' is not a member of 'std'
  170 |         std::min<size_t>(std::numeric_limits<unsigned int>().max(), x));
      |                               ^~~~~~~~~~~~~~
../../../../Source/Core/DiscIO/WIACompression.cpp:170:46: error: expected primary-expression before 'unsigned'
  170 |         std::min<size_t>(std::numeric_limits<unsigned int>().max(), x));
      |                                              ^~~~~~~~
```

Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
2020-10-15 19:56:53 +01:00
Léo Lam ab2e226df8
Merge pull request #9105 from JosJuice/volumeverifier-small-low-severity
VolumeVerifier: Make high severity "too small" hide low severity "too small"
2020-10-14 12:41:57 +02:00
JosJuice 3feea108db DiscIO: Decrease RAM usage during zstd compression
By calling ZSTD_CCtx_setPledgedSrcSize, we can let zstd know
how large a chunk is going to be before which start compressing
it, which lets zstd avoid allocating more memory than needed
for various internal buffers. This greatly reduces the RAM usage
when using a high compression level with a small chunk size,
and doesn't have much of an effect in other circumstances.

A side effect of calling ZSTD_CCtx_setPledgedSrcSize is that
zstd by default will write the uncompressed size into the
compressed data stream as metadata. In order to save space,
and since the decompressed size can be figured out through
the structure of the RVZ format anyway, we disable writing
the uncompressed size by setting ZSTD_c_contentSizeFlag to 0.
2020-10-07 16:40:06 +02:00
JosJuice 402643fe4c VolumeVerifier: Call CheckDiscSize from Finish 2020-09-23 11:19:03 +02:00
JosJuice 4f62960fb1 VolumeVerifier: Make high severity "too small" hide low severity "too small" 2020-09-21 21:00:04 +02:00
JosJuice ca46028cde DiscIO: Use std::function for compression callback 2020-09-16 18:36:50 +02:00
JMC47 e7e5175606
Merge pull request #8861 from JosJuice/netplay-hash
Make netplay's "same game" check more robust
2020-09-06 17:14:08 -04:00
JMC47 c8c4ec28ce
Merge pull request #8644 from JosJuice/rework-tgc
DiscIO: Rework the implementation of TGC reading
2020-09-06 07:06:26 -04:00
LC db067104ed
Merge pull request #9047 from JosJuice/wia-header-heuristic
DiscIO: Adjust WIA/RVZ header size heuristic
2020-09-01 17:58:51 -04:00