Commit Graph

44 Commits

Author SHA1 Message Date
JosJuice a87dffe52d DiscIO: Replace IsDataSizeAccurate with GetDataSizeType
Previously, we had WBFS and CISO which both returned an upper bound
of the size, and other formats which returned an accurate size. But
now we also have NFS, which returns a lower bound of the size. To
allow VolumeVerifier to make better informed decisions for NFS, let's
use an enum instead of a bool for the type of data size a blob has.
2022-08-04 22:00:59 +02:00
Shawn Hoffman dd29a54cf6 introduce wrapper for SHA1 functionality 2022-07-26 22:16:37 -07:00
Minty-Meeo deba9ce256 Added a way to check Block Size, Compression Method, and Compression Level flags to dolphin-tool
New dolphin-tool command: "header"
-b / --block_size
-c / --compression
-l / --compression_level

Informative RVZ/WIA header2 value "compression_level" is now a s32 instead of a u32, because negative compression is a thing.

Speaking of, it is now possible to use negative compression levels in dolphin-tool's convert command (not the GUI, though).
2022-03-19 17:13:28 -05:00
Pokechu22 78bfd25964 Fix all uninitialized variable warnings (C26495) 2021-10-13 12:32:16 -07:00
Shawn Hoffman 4ef87194a4 revert workaround for msvc arm64 ICE in WIABlob 2021-08-14 03:57:53 -07:00
JosJuice 5fe23ccc3e DiscIO: Fix broken workaround for MSVC ARM64 ICE
The workaround added in 30f9f31 caused a regression where Dolphin
incorrectly replaced runs of one byte with runs of another byte
when writing WIA and RVZ files. ReuseID::operator< was always
returning false unless the ReuseIDs being compared had different
partition keys, which caused std::map<ReuseID, GroupEntry>
to treat all ReuseIDs with the same partition key as equal.
2021-07-24 13:29:03 +02:00
Léo Lam 39e14b6b5f
Merge pull request #9854 from JosJuice/wia-ice
DiscIO: Store partition key directly in ReuseID
2021-07-06 13:05:09 +02:00
Pierre Bourdon e149ad4f0a
treewide: convert GPLv2+ license info to SPDX tags
SPDX standardizes how source code conveys its copyright and licensing
information. See https://spdx.github.io/spdx-spec/1-rationale/ . SPDX
tags are adopted in many large projects, including things like the Linux
kernel.
2021-07-05 04:35:56 +02:00
JosJuice eafa2784d6 DiscIO: Store partition key directly in ReuseID
This lets us reuse blocks in the case where different partition
entries use the same key (which probably isn't very common).
2021-06-30 09:06:20 +02:00
Shawn Hoffman 30f9f31a69 DiscIO: Add workaround for MSVC ARM64 ICE 2021-06-29 10:24:09 +02:00
JosJuice 14bfc0be78 DiscIO: Fix reading certain WIA chunks with many exceptions
The loop in WIARVZFileReader::Chunk::Read could terminate
prematurely if the size argument was smaller than the size
of an exception list which had only been partially loaded.
2021-03-07 14:14:45 +01:00
Shawn Hoffman 84128d9532 rename Common/File to Common/IOFile 2021-01-27 14:29:48 -08:00
JosJuice ca46028cde DiscIO: Use std::function for compression callback 2020-09-16 18:36:50 +02:00
JosJuice 047bb80337 DiscIO: Add parameters to BlobReader::SupportsReadWiiDecrypted
It's possible (but rare) for a WIA or RVZ file to support
this for some partitions but not all, and for the game and
the blob code to disagree on how large a partition is.
2020-08-29 15:18:52 +02:00
JosJuice d494e0230c Show file format details in game properties 2020-06-21 20:47:23 +02:00
JosJuice 660d81a10b RVZ: Bump version number to 1.0 2020-06-21 12:38:03 +02:00
JosJuice 224c6e799d RVZ: Extend GroupEntry 2020-06-17 13:48:45 +02:00
JosJuice f2c38c0e67 RVZ: Make m_rvz a template parameter
Needed for the next commit.
2020-06-17 13:48:45 +02:00
JosJuice ca4e4a6207 WIA/RVZ: Move (de)compression to a different file 2020-06-17 13:48:45 +02:00
JosJuice 3f753fc87d RVZ: Detect junk data in the same block as a file 2020-06-17 13:48:44 +02:00
JosJuice 4b74993374 RVZ: Store pseudorandom junk data efficiently 2020-06-17 13:48:42 +02:00
JosJuice 1e92b54bf5 WIA/RVZ: Skip some memory allocations when reusing chunks 2020-06-17 13:48:16 +02:00
JosJuice f5ef70fc76 RVZ: Don't store redundant exceptions when chunk size is < 2 MiB 2020-06-17 13:48:16 +02:00
JosJuice b06c50ed2e RVZ: Support chunk sizes between 32 KiB and 2 MiB
WIA doesn't support smaller than 2 MiB.
2020-06-17 13:48:15 +02:00
JosJuice 1f7c0b636f RVZ: Add Zstandard as a compression method 2020-06-17 13:48:12 +02:00
JosJuice e2ae2b3b0b Add new file format RVZ based on WIA 2020-06-17 13:47:34 +02:00
JosJuice f21a254042 WIA: Implement multithreaded compression 2020-06-17 13:47:34 +02:00
JosJuice 9dea8169e0 WIA: Write all headers at the start of the file
Gets rid of the need to seek to the end of the file
when opening a file.

The downside of this is that we waste a little space,
since we can't know in advance exactly how much
space the compressed parts of the headers will need.
2020-06-17 13:47:34 +02:00
JosJuice e5b9e1ba1f WIA: Reuse groups when writing
This is useful for the way Dolphin scrubs Wii discs.
The encrypted data is what gets zeroed out, but this
zeroed out data then gets decrypted before being stored,
and the resulting data does not compress well.
However, each block of decrypted scrubbed data is
identical given the same encryption key, and there's
nothing stopping us from making multiple group entries
point to the same offset in the file, so we only have
to store one copy of this data per partition.

For reference, wit zeroes out the decrypted data,
but Dolphin's WIA writer can't do this because it currently
doesn't know which parts of the disc are scrubbed.

This is also useful for things such as storing Datel discs
full of 0x55 blocks (repesenting unreadable blocks)
without compression enabled.
2020-06-17 13:47:34 +02:00
JosJuice e8b019ac29 WIA: Implement compression 2020-06-17 13:47:30 +02:00
JosJuice 3b8c44fd0e WIA: Decrypt Wii data when writing 2020-06-17 13:43:54 +02:00
JosJuice 115edea34e WIA: Add early support for WIA writing 2020-06-17 13:43:54 +02:00
JosJuice 791e363c9a WIA: Make use of the exception lists 2020-06-17 13:43:54 +02:00
JosJuice 47067f661a WIA: Properly check for overlapping data 2020-06-17 13:43:53 +02:00
JosJuice 04089f24f9 WIA: Implement re-encryption of Wii partition data 2020-06-17 13:43:53 +02:00
JosJuice e3d291a529 WIA: Check the internal WIA hashes 2020-06-17 13:43:53 +02:00
JosJuice 827437c036 WIA: Fix the handling of chunk sizes larger than 2 MiB 2020-06-17 13:43:53 +02:00
JosJuice 0b407228b7 WIA: Add documentation 2020-06-17 13:43:53 +02:00
JosJuice 01a77ae8a1 WIA: Implement caching and partial decompression 2020-06-17 13:43:53 +02:00
JosJuice b59ef81a7e WIA: Implement bzip2, LZMA, and LZMA2 decompression 2020-06-17 13:43:52 +02:00
JosJuice 36991e2dde WIA: Implement PURGE decompression 2020-06-17 13:43:52 +02:00
JosJuice 3672bd79f3 WIA: Implement ReadWiiDecrypted 2020-06-17 13:43:52 +02:00
JosJuice 2a5fcc9c25 WIA: Add reading raw data 2020-06-17 13:43:51 +02:00
JosJuice 8da5d0c4fe Add an early version of WIABlobReader
It can currently only read the first 0x80 bytes of a disc image,
which is enough for identifying it but not for doing anything else.
2020-06-17 13:43:51 +02:00