GameDB: Support case-insensitive serials and patch CRCs like existing GameDB impl.

GameDB: Regenerate with master's .dbf file


GameDB: Update with master's latest changes

bin/GameIndex.md

@@ -1,6 +1,7 @@
 # GameDB Documentation
 
 - [YAML Game Format](#yaml-game-format)
+- [A Note on Case Sensitivity](#a-note-on-case-sensitivity)
 - [Rounding Modes](#rounding-modes)
   - [Options](#options)
 - [Clamping Modes](#clamping-modes)
@@ -17,7 +18,7 @@
 The following is an annotated and comprehensive example of everything that can be defined for a single Game entry.
 
 ```yaml
-SERIAL-12345: # !required! Serial number for the game, this is how games are looked up
+SERIAL-12345: # !required! Serial number for the game, this is how games are looked up.  Case insensitive
   name: "A Sample Game" # !required!
   region: "NTSC-U" # !required!
   roundModes:
@@ -75,7 +76,7 @@ SERIAL-12345: # !required! Serial number for the game, this is how games are loo
       content: |- # !required! This allows for multi-line strings in YAML, this type preserves new-line characters
         comment=Sample Patch
         patch=1,EE,00000002,word,00000000
-    "crc-1": # Specific CRC Patch!
+    crc123: # Specific CRC Patch!
       author: "Some Person"
       content: |-
         comment=Another Sample
@@ -84,6 +85,18 @@ SERIAL-12345: # !required! Serial number for the game, this is how games are loo
 
 > Note that quoting strings in YAML is optional, but certain characters are reserved like '*' and require the string to be quoted, be aware / use a YAML linter to avoid confusion.
 
+## A Note on Case Sensitivity
+
+Both the serial numbers for the games, and the CRC patches are at the moment not case-sensitive and will be looked up with their lowercase representations.  **However, stylistically, uppercase is preferred and may be enforced and migrated to in the future**.
+
+For example:
+- `SLUS-123` will be stored and looked up in the GameDB as `slus-123`
+- Likewise, a CRC with upper-case hex `23AF6876` will be stored and looked up as `23af6876`
+
+However, YAML is case-sensitive and will allow multiple serials that only differ on casing.  To prevent mistakes, this will also throw a validation error and the first entry will be the one that wins.
+
+**Everything else can be safely assumed to be case sensitive!**
+
 ## Rounding Modes
 
 The rounding modes are numerically based.
@@ -172,7 +185,9 @@ This works fine for the vast majority of games, but fails in some cases, for whi
 
 ## Patches
 
-The patch that corresponds to the running game's CRC will take precedence over the `default`.  Multiple patches for the same CRC cannot be defined and this will throw an invalidation errors.
+The patch that corresponds to the running game's CRC will take precedence over the `default`.  Multiple patches using the same CRC cannot be defined and this will throw a validation error.
+
+> CRCs are case-insensitive, however uppercase is preferred stylistically!
 
 Patches should be defined as multi-line string blocks, where each line would correspond with a line in a conventional `*.pnach` file
 

bin/GameIndex.yaml

@@ -32,7 +32,6 @@ GUST-00009:
   roundModes:
     eeRoundMode: 0  # Fixes jump issue.
   gameFixes:
-    - DMABusyHack
     - GIFFIFOHack # Fixes flickering sprites.
 PAPX-90203:
   name: "Gran Turismo 2000 [Trial]"
@@ -631,12 +630,12 @@ SCAJ-20125:
   name: "Tekken 5"
   region: "NTSC-Unk"
   clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SCAJ-20126:
   name: "Tekken 5"
   region: "NTSC-Unk"
   clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SCAJ-20127:
   name: "EyeToy - Play 2 [with Camera]"
   region: "NTSC-Unk"
@@ -766,6 +765,8 @@ SCAJ-20158:
 SCAJ-20159:
   name: "Soul Calibur III"
   region: "NTSC-Ch-E-J"
+  gameFixes:
+    - EETimingHack # Fixes bad colours on charcter select when in Progressive Scan
 SCAJ-20160:
   name: "Yoshitsuneki"
   region: "NTSC-Unk"
@@ -912,7 +913,7 @@ SCAJ-20199:
   name: "Tekken 5 [PlayStation 2 The Best]"
   region: "NTSC-Unk"
   clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SCAJ-25002:
   name: "Shinobi"
   region: "NTSC-Unk"
@@ -1182,6 +1183,8 @@ SCES-50139:
   name: "World Rally Championship"
   region: "PAL-M7"
   compat: 5
+  roundModes:
+    eeRoundMode: 0  # Fixes crash when using the Subaru
   gameFixes:
     - EETimingHack # Fix in-game Freeze.
 SCES-50240:
@@ -2023,7 +2026,7 @@ SCES-53202:
   region: "PAL-M5"
   compat: 5
   clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SCES-53247:
   name: "WRC Rally Evolved"
   region: "PAL-M8"
@@ -2713,7 +2716,7 @@ SCKA-20049:
   name: "Tekken 5"
   region: "NTSC-K"
   clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SCKA-20050:
   name: "Tales of Legendia"
   region: "NTSC-K"
@@ -2755,6 +2758,8 @@ SCKA-20059:
   name: "Soul Calibur III"
   region: "NTSC-K"
   compat: 5
+  gameFixes:
+    - EETimingHack # Fixes bad colours on charcter select when in Progressive Scan
 SCKA-20060:
   name: "Ratchet - Deadlocked"
   region: "NTSC-K"
@@ -2823,7 +2828,7 @@ SCKA-20081:
   name: "Tekken 5 [PlayStation 2 Big Hit Series]"
   region: "NTSC-K"
   clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SCKA-20086:
   name: "Shin Onimusha - Dawn of Dreams [Disc1of2]"
   region: "NTSC-K"
@@ -8376,6 +8381,10 @@ SLES-51203:
 SLES-51208:
   name: "Rocky"
   region: "PAL-M5"
+  clampModes:
+    eeClampMode: 0  # Fixes boxers not appearing/disappearing
+  gameFixes:
+    - VIF1StallHack # Fixes freezes
 SLES-51209:
   name: "Haven - Call of the King"
   region: "PAL-M5"
@@ -16773,7 +16782,6 @@ SLES-55443:
   roundModes:
     eeRoundMode: 0  # Fixes jump issue.
   gameFixes:
-    - DMABusyHack
     - GIFFIFOHack # Fixes flickering sprites.
 SLES-55444:
   name: "Ar tonelico II: Melody of Metafalica"
@@ -18225,6 +18233,8 @@ SLPM-61120:
 SLPM-61133:
   name: "Soul Calibur III [Trial Version]"
   region: "NTSC-J"
+  gameFixes:
+    - EETimingHack # Fixes bad colours on charcter select when in Progressive Scan
 SLPM-61135:
   name: "Naruto - Narutimett Hero 3 [Trial Version]"
   region: "NTSC-J"
@@ -21494,6 +21504,10 @@ SLPM-65462:
 SLPM-65463:
   name: "Rocky"
   region: "NTSC-J"
+  clampModes:
+    eeClampMode: 0  # Fixes boxers not appearing/disappearing
+  gameFixes:
+    - VIF1StallHack # Fixes freezes
 SLPM-65464:
   name: "Wind - A Breath of Heart"
   region: "NTSC-J"
@@ -27865,6 +27879,8 @@ SLPS-25098:
 SLPS-25099:
   name: "World Rally Championship"
   region: "NTSC-J"
+  roundModes:
+    eeRoundMode: 0  # Fixes crash when using the Subaru
 SLPS-25100:
   name: "Tekken 4"
   region: "NTSC-J"
@@ -29133,7 +29149,7 @@ SLPS-25510:
   region: "NTSC-J"
   compat: 5
   clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SLPS-25511:
   name: "Rasetsu Alternative"
   region: "NTSC-J"
@@ -29336,6 +29352,8 @@ SLPS-25577:
   name: "Soul Calibur III"
   region: "NTSC-J"
   compat: 5
+  gameFixes:
+    - EETimingHack # Fixes bad colours on charcter select when in Progressive Scan
 SLPS-25578:
   name: "K-1 World Grand Prix 2005"
   region: "NTSC-J"
@@ -30600,7 +30618,7 @@ SLPS-73223:
   name: "Tekken 5 [PlayStation 2 The Best]"
   region: "NTSC-J"
   clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SLPS-73224:
   name: "Xenosaga Episode II - Jenseits von Gut und Bose [PlayStation 2 The Best] [Disc1of2]"
   region: "NTSC-J"
@@ -32488,6 +32506,8 @@ SLUS-20418:
 SLUS-20419:
   name: "World Rally Championship"
   region: "NTSC-U"
+  roundModes:
+    eeRoundMode: 0  # Fixes crash when using the Subaru
 SLUS-20420:
   name: "Star Wars - Bounty Hunter"
   region: "NTSC-U"
@@ -33050,6 +33070,10 @@ SLUS-20559:
   name: "Rocky"
   region: "NTSC-U"
   compat: 5
+  clampModes:
+    eeClampMode: 0  # Fixes boxers not appearing/disappearing
+  gameFixes:
+    - VIF1StallHack # Fixes freezes
 SLUS-20560:
   name: "Galerians - Ash"
   region: "NTSC-U"
@@ -35161,7 +35185,7 @@ SLUS-21059:
   region: "NTSC-U"
   compat: 5
   clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SLUS-21060:
   name: "WWE SmackDown! vs. RAW"
   region: "NTSC-U"
@@ -35560,7 +35584,7 @@ SLUS-21160:
   name: "Tekken 5 [Demo]"
   region: "NTSC-U"
   clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SLUS-21161:
   name: "Fight Night - Round 2"
   region: "NTSC-U"
@@ -35798,6 +35822,8 @@ SLUS-21216:
   name: "Soul Calibur III"
   region: "NTSC-U"
   compat: 5
+  gameFixes:
+    - EETimingHack # Fixes bad colours on charcter select when in Progressive Scan
 SLUS-21217:
   name: "Incredibles, The - Rise of the Underminers"
   region: "NTSC-U"
@@ -37974,7 +38000,6 @@ SLUS-21735:
   roundModes:
     eeRoundMode: 0  # Fixes jump issue.
   gameFixes:
-    - DMABusyHack
     - GIFFIFOHack # Fixes flickering sprites.
 SLUS-21736:
   name: "Wall-E"

pcsx2/GameDatabase.cpp

@@ -20,8 +20,24 @@
 #include "fmt/core.h"
 #include "yaml-cpp/yaml.h"
 #include <fstream>
+#include <algorithm>
 
-std::string GameDatabaseSchema::GameEntry::memcardFiltersAsString()
+std::string strToLower(std::string str)
+{
+	std::transform(str.begin(), str.end(), str.begin(),
+				   [](unsigned char c) { return std::tolower(c); });
+	return str;
+}
+
+bool compareStrNoCase(const std::string str1, const std::string str2)
+{
+	return std::equal(str1.begin(), str1.end(), str2.begin(),
+					  [](char a, char b) {
+						  return tolower(a) == tolower(b);
+					  });
+}
+
+std::string GameDatabaseSchema::GameEntry::memcardFiltersAsString() const
 {
 	if (memcardFilters.empty())
 		return "";
@@ -37,6 +53,26 @@ std::string GameDatabaseSchema::GameEntry::memcardFiltersAsString()
 	return filters;
 }
 
+bool GameDatabaseSchema::GameEntry::findPatch(const std::string crc, Patch& patch) const
+{
+	std::string crcLower = strToLower(crc);
+	Console.WriteLn(fmt::format("[GameDB] Searching for patch with CRC '{}'", crc));
+	if (patches.count(crcLower) == 1)
+	{
+		Console.WriteLn(fmt::format("[GameDB] Found patch with CRC '{}'", crc));
+		patch = patches.at(crcLower);
+		return true;
+	}
+	else if (patches.count("default") == 1)
+	{
+		Console.WriteLn("[GameDB] Found and falling back to default patch");
+		patch = patches.at("default");
+		return true;
+	}
+	Console.WriteLn("[GameDB] No CRC-specific patch or default patch found");
+	return false;
+}
+
 std::vector<std::string> YamlGameDatabaseImpl::convertMultiLineStringToVector(const std::string multiLineString)
 {
 	std::vector<std::string> lines;
@@ -111,7 +147,12 @@ GameDatabaseSchema::GameEntry YamlGameDatabaseImpl::entryFromYaml(const std::str
 		{
 			for (const auto& entry : patches)
 			{
-				std::string crc = entry.first.as<std::string>();
+				std::string crc = strToLower(entry.first.as<std::string>());
+				if (gameEntry.patches.count(crc) == 1)
+				{
+					Console.Error(fmt::format("[GameDB] Duplicate CRC '{}' found for serial: '{}'. Skipping, CRCs are case-insensitive!", crc, serial));
+					continue;
+				}
 				YAML::Node patchNode = entry.second;
 
 				GameDatabaseSchema::Patch patchCol;
@@ -124,12 +165,12 @@ GameDatabaseSchema::GameEntry YamlGameDatabaseImpl::entryFromYaml(const std::str
 	}
 	catch (const YAML::RepresentationException& e)
 	{
-		Console.Error(fmt::format("[GameDB] Invalid GameDB syntax detected on serial: '{}'.  Error Details - {}", serial, e.msg));
+		Console.Error(fmt::format("[GameDB] Invalid GameDB syntax detected on serial: '{}'. Error Details - {}", serial, e.msg));
 		gameEntry.isValid = false;
 	}
 	catch (const std::exception& e)
 	{
-		Console.Error(fmt::format("[GameDB] Unexpected error occurred when reading serial: '{}'.  Error Details - {}", serial, e.what()));
+		Console.Error(fmt::format("[GameDB] Unexpected error occurred when reading serial: '{}'. Error Details - {}", serial, e.what()));
 		gameEntry.isValid = false;
 	}
 	return gameEntry;
@@ -137,9 +178,15 @@ GameDatabaseSchema::GameEntry YamlGameDatabaseImpl::entryFromYaml(const std::str
 
 GameDatabaseSchema::GameEntry YamlGameDatabaseImpl::findGame(const std::string serial)
 {
-	if (gameDb.count(serial) == 1)
+	std::string serialLower = strToLower(serial);
-		return gameDb[serial];
+	Console.WriteLn(fmt::format("[GameDB] Searching for '{}' in GameDB", serialLower));
+	if (gameDb.count(serialLower) == 1)
+	{
+		Console.WriteLn(fmt::format("[GameDB] Found '{}' in GameDB", serialLower));
+		return gameDb[serialLower];
+	}
 
+	Console.Error(fmt::format("[GameDB] Could not find '{}' in GameDB", serialLower));
 	GameDatabaseSchema::GameEntry entry;
 	entry.isValid = false;
 	return entry;
@@ -168,12 +215,21 @@ bool YamlGameDatabaseImpl::initDatabase(std::ifstream& stream)
 			// but we do want to yell about it so it can be corrected
 			try
 			{
-				std::string serial = entry.first.as<std::string>();
+				// Serials and CRCs must be inserted as lower-case, as that is how they are retrieved
+				// this is because the application may pass a lowercase CRC or serial along
+				//
+				// However, YAML's keys are as expected case-sensitive, so we have to explicitly do our own duplicate checking
+				std::string serial = strToLower(entry.first.as<std::string>());
+				if (gameDb.count(serial) == 1)
+				{
+					Console.Error(fmt::format("[GameDB] Duplicate serial '{}' found in GameDB. Skipping, Serials are case-insensitive!", serial));
+					continue;
+				}
 				gameDb[serial] = entryFromYaml(serial, entry.second);
 			}
 			catch (const YAML::RepresentationException& e)
 			{
-				Console.Error(fmt::format("[GameDB] Invalid GameDB syntax detected.  Error Details - {}", e.msg));
+				Console.Error(fmt::format("[GameDB] Invalid GameDB syntax detected. Error Details - {}", e.msg));
 			}
 		}
 	}

pcsx2/GameDatabase.h

@@ -77,7 +77,8 @@ public:
 		std::vector<std::string> memcardFilters;
 		std::unordered_map<std::string, Patch> patches;
 
-		std::string memcardFiltersAsString();
+		std::string memcardFiltersAsString() const;
+		bool findPatch(const std::string crc, Patch& patch) const;
 	};
 };
 
@@ -104,3 +105,5 @@ private:
 };
 
 extern IGameDatabase* AppHost_GetGameDatabase();
+extern std::string strToLower(std::string str);
+extern bool compareStrNoCase(const std::string str1, const std::string str2);

pcsx2/Patch.cpp

@@ -128,16 +128,8 @@ int LoadPatchesFromGamesDB(const wxString& crc, const GameDatabaseSchema::GameEn
 	if (game.isValid)
 	{
 		GameDatabaseSchema::Patch patch;
-		if (game.patches.count(std::string(crc)) == 1)
+		bool patchFound = game.findPatch(std::string(crc), patch);
-		{
+		if (patchFound && patch.patchLines.size() > 0)
-			patch = game.patches.at(std::string(crc));
-		}
-		else if (game.patches.count("default") == 1)
-		{
-			patch = game.patches.at("default");
-		}
-
-		if (patch.patchLines.size() > 0)
 		{
 			for (auto line : patch.patchLines)
 			{