GameDB: Support case-insensitive serials and patch CRCs like existing GameDB impl.

GameDB: Regenerate with master's .dbf file GameDB: Update with master's latest changes
2020-12-09 20:10:02 -05:00 · 2020-12-09 20:10:02 -05:00 · 34fe57d84b
parent 4b975efa23
commit 34fe57d84b
5 changed files with 126 additions and 35 deletions
--- a/bin/GameIndex.md
+++ b/bin/GameIndex.md
@ -1,6 +1,7 @@
 # GameDB Documentation

 - [YAML Game Format](#yaml-game-format)
+- [A Note on Case Sensitivity](#a-note-on-case-sensitivity)
 - [Rounding Modes](#rounding-modes)
  - [Options](#options)
 - [Clamping Modes](#clamping-modes)
@ -17,7 +18,7 @@
 The following is an annotated and comprehensive example of everything that can be defined for a single Game entry.

 ```yaml
-SERIAL-12345: # !required! Serial number for the game, this is how games are looked up
+SERIAL-12345: # !required! Serial number for the game, this is how games are looked up.  Case insensitive
  name: "A Sample Game" # !required!
  region: "NTSC-U" # !required!
  roundModes:
@ -75,7 +76,7 @@ SERIAL-12345: # !required! Serial number for the game, this is how games are loo
      content: |- # !required! This allows for multi-line strings in YAML, this type preserves new-line characters
        comment=Sample Patch
        patch=1,EE,00000002,word,00000000
-    "crc-1": # Specific CRC Patch!
+    crc123: # Specific CRC Patch!
      author: "Some Person"
      content: |-
        comment=Another Sample
@ -84,6 +85,18 @@ SERIAL-12345: # !required! Serial number for the game, this is how games are loo

 > Note that quoting strings in YAML is optional, but certain characters are reserved like '*' and require the string to be quoted, be aware / use a YAML linter to avoid confusion.

+## A Note on Case Sensitivity
+
+Both the serial numbers for the games, and the CRC patches are at the moment not case-sensitive and will be looked up with their lowercase representations.  **However, stylistically, uppercase is preferred and may be enforced and migrated to in the future**.
+
+For example:
+- `SLUS-123` will be stored and looked up in the GameDB as `slus-123`
+- Likewise, a CRC with upper-case hex `23AF6876` will be stored and looked up as `23af6876`
+
+However, YAML is case-sensitive and will allow multiple serials that only differ on casing.  To prevent mistakes, this will also throw a validation error and the first entry will be the one that wins.
+
+**Everything else can be safely assumed to be case sensitive!**
+
 ## Rounding Modes

 The rounding modes are numerically based.
@ -172,7 +185,9 @@ This works fine for the vast majority of games, but fails in some cases, for whi

 ## Patches

-The patch that corresponds to the running game's CRC will take precedence over the `default`.  Multiple patches for the same CRC cannot be defined and this will throw an invalidation errors.
+The patch that corresponds to the running game's CRC will take precedence over the `default`.  Multiple patches using the same CRC cannot be defined and this will throw a validation error.
+
+> CRCs are case-insensitive, however uppercase is preferred stylistically!

 Patches should be defined as multi-line string blocks, where each line would correspond with a line in a conventional `*.pnach` file

--- a/bin/GameIndex.yaml
+++ b/bin/GameIndex.yaml
@ -32,7 +32,6 @@ GUST-00009:
  roundModes:
    eeRoundMode: 0  # Fixes jump issue.
  gameFixes:
-    - DMABusyHack
    - GIFFIFOHack # Fixes flickering sprites.
 PAPX-90203:
  name: "Gran Turismo 2000 [Trial]"
@ -631,12 +630,12 @@ SCAJ-20125:
  name: "Tekken 5"
  region: "NTSC-Unk"
  clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SCAJ-20126:
  name: "Tekken 5"
  region: "NTSC-Unk"
  clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SCAJ-20127:
  name: "EyeToy - Play 2 [with Camera]"
  region: "NTSC-Unk"
@ -766,6 +765,8 @@ SCAJ-20158:
 SCAJ-20159:
  name: "Soul Calibur III"
  region: "NTSC-Ch-E-J"
+  gameFixes:
+    - EETimingHack # Fixes bad colours on charcter select when in Progressive Scan
 SCAJ-20160:
  name: "Yoshitsuneki"
  region: "NTSC-Unk"
@ -912,7 +913,7 @@ SCAJ-20199:
  name: "Tekken 5 [PlayStation 2 The Best]"
  region: "NTSC-Unk"
  clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SCAJ-25002:
  name: "Shinobi"
  region: "NTSC-Unk"
@ -1182,6 +1183,8 @@ SCES-50139:
  name: "World Rally Championship"
  region: "PAL-M7"
  compat: 5
+  roundModes:
+    eeRoundMode: 0  # Fixes crash when using the Subaru
  gameFixes:
    - EETimingHack # Fix in-game Freeze.
 SCES-50240:
@ -2023,7 +2026,7 @@ SCES-53202:
  region: "PAL-M5"
  compat: 5
  clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SCES-53247:
  name: "WRC Rally Evolved"
  region: "PAL-M8"
@ -2713,7 +2716,7 @@ SCKA-20049:
  name: "Tekken 5"
  region: "NTSC-K"
  clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SCKA-20050:
  name: "Tales of Legendia"
  region: "NTSC-K"
@ -2755,6 +2758,8 @@ SCKA-20059:
  name: "Soul Calibur III"
  region: "NTSC-K"
  compat: 5
+  gameFixes:
+    - EETimingHack # Fixes bad colours on charcter select when in Progressive Scan
 SCKA-20060:
  name: "Ratchet - Deadlocked"
  region: "NTSC-K"
@ -2823,7 +2828,7 @@ SCKA-20081:
  name: "Tekken 5 [PlayStation 2 Big Hit Series]"
  region: "NTSC-K"
  clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SCKA-20086:
  name: "Shin Onimusha - Dawn of Dreams [Disc1of2]"
  region: "NTSC-K"
@ -8376,6 +8381,10 @@ SLES-51203:
 SLES-51208:
  name: "Rocky"
  region: "PAL-M5"
+  clampModes:
+    eeClampMode: 0  # Fixes boxers not appearing/disappearing
+  gameFixes:
+    - VIF1StallHack # Fixes freezes
 SLES-51209:
  name: "Haven - Call of the King"
  region: "PAL-M5"
@ -16773,7 +16782,6 @@ SLES-55443:
  roundModes:
    eeRoundMode: 0  # Fixes jump issue.
  gameFixes:
-    - DMABusyHack
    - GIFFIFOHack # Fixes flickering sprites.
 SLES-55444:
  name: "Ar tonelico II: Melody of Metafalica"
@ -18225,6 +18233,8 @@ SLPM-61120:
 SLPM-61133:
  name: "Soul Calibur III [Trial Version]"
  region: "NTSC-J"
+  gameFixes:
+    - EETimingHack # Fixes bad colours on charcter select when in Progressive Scan
 SLPM-61135:
  name: "Naruto - Narutimett Hero 3 [Trial Version]"
  region: "NTSC-J"
@ -21494,6 +21504,10 @@ SLPM-65462:
 SLPM-65463:
  name: "Rocky"
  region: "NTSC-J"
+  clampModes:
+    eeClampMode: 0  # Fixes boxers not appearing/disappearing
+  gameFixes:
+    - VIF1StallHack # Fixes freezes
 SLPM-65464:
  name: "Wind - A Breath of Heart"
  region: "NTSC-J"
@ -27865,6 +27879,8 @@ SLPS-25098:
 SLPS-25099:
  name: "World Rally Championship"
  region: "NTSC-J"
+  roundModes:
+    eeRoundMode: 0  # Fixes crash when using the Subaru
 SLPS-25100:
  name: "Tekken 4"
  region: "NTSC-J"
@ -29133,7 +29149,7 @@ SLPS-25510:
  region: "NTSC-J"
  compat: 5
  clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SLPS-25511:
  name: "Rasetsu Alternative"
  region: "NTSC-J"
@ -29336,6 +29352,8 @@ SLPS-25577:
  name: "Soul Calibur III"
  region: "NTSC-J"
  compat: 5
+  gameFixes:
+    - EETimingHack # Fixes bad colours on charcter select when in Progressive Scan
 SLPS-25578:
  name: "K-1 World Grand Prix 2005"
  region: "NTSC-J"
@ -30600,7 +30618,7 @@ SLPS-73223:
  name: "Tekken 5 [PlayStation 2 The Best]"
  region: "NTSC-J"
  clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SLPS-73224:
  name: "Xenosaga Episode II - Jenseits von Gut und Bose [PlayStation 2 The Best] [Disc1of2]"
  region: "NTSC-J"
@ -32488,6 +32506,8 @@ SLUS-20418:
 SLUS-20419:
  name: "World Rally Championship"
  region: "NTSC-U"
+  roundModes:
+    eeRoundMode: 0  # Fixes crash when using the Subaru
 SLUS-20420:
  name: "Star Wars - Bounty Hunter"
  region: "NTSC-U"
@ -33050,6 +33070,10 @@ SLUS-20559:
  name: "Rocky"
  region: "NTSC-U"
  compat: 5
+  clampModes:
+    eeClampMode: 0  # Fixes boxers not appearing/disappearing
+  gameFixes:
+    - VIF1StallHack # Fixes freezes
 SLUS-20560:
  name: "Galerians - Ash"
  region: "NTSC-U"
@ -35161,7 +35185,7 @@ SLUS-21059:
  region: "NTSC-U"
  compat: 5
  clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SLUS-21060:
  name: "WWE SmackDown! vs. RAW"
  region: "NTSC-U"
@ -35560,7 +35584,7 @@ SLUS-21160:
  name: "Tekken 5 [Demo]"
  region: "NTSC-U"
  clampModes:
-    eeClampMode: 1
+    eeClampMode: 2  # Fixes camera and stops constant coin noises on Pirates Cove
 SLUS-21161:
  name: "Fight Night - Round 2"
  region: "NTSC-U"
@ -35798,6 +35822,8 @@ SLUS-21216:
  name: "Soul Calibur III"
  region: "NTSC-U"
  compat: 5
+  gameFixes:
+    - EETimingHack # Fixes bad colours on charcter select when in Progressive Scan
 SLUS-21217:
  name: "Incredibles, The - Rise of the Underminers"
  region: "NTSC-U"
@ -37974,7 +38000,6 @@ SLUS-21735:
  roundModes:
    eeRoundMode: 0  # Fixes jump issue.
  gameFixes:
-    - DMABusyHack
    - GIFFIFOHack # Fixes flickering sprites.
 SLUS-21736:
  name: "Wall-E"
--- a/pcsx2/GameDatabase.cpp
+++ b/pcsx2/GameDatabase.cpp
@ -20,8 +20,24 @@
 #include "fmt/core.h"
 #include "yaml-cpp/yaml.h"
 #include <fstream>
+#include <algorithm>

-std::string GameDatabaseSchema::GameEntry::memcardFiltersAsString()
+std::string strToLower(std::string str)
+{
+	std::transform(str.begin(), str.end(), str.begin(),
+				   [](unsigned char c) { return std::tolower(c); });
+	return str;
+}
+
+bool compareStrNoCase(const std::string str1, const std::string str2)
+{
+	return std::equal(str1.begin(), str1.end(), str2.begin(),
+					  [](char a, char b) {
+						  return tolower(a) == tolower(b);
+					  });
+}
+
+std::string GameDatabaseSchema::GameEntry::memcardFiltersAsString() const
 {
 	if (memcardFilters.empty())
 		return "";
@ -37,6 +53,26 @@ std::string GameDatabaseSchema::GameEntry::memcardFiltersAsString()
 	return filters;
 }

+bool GameDatabaseSchema::GameEntry::findPatch(const std::string crc, Patch& patch) const
+{
+	std::string crcLower = strToLower(crc);
+	Console.WriteLn(fmt::format("[GameDB] Searching for patch with CRC '{}'", crc));
+	if (patches.count(crcLower) == 1)
+	{
+		Console.WriteLn(fmt::format("[GameDB] Found patch with CRC '{}'", crc));
+		patch = patches.at(crcLower);
+		return true;
+	}
+	else if (patches.count("default") == 1)
+	{
+		Console.WriteLn("[GameDB] Found and falling back to default patch");
+		patch = patches.at("default");
+		return true;
+	}
+	Console.WriteLn("[GameDB] No CRC-specific patch or default patch found");
+	return false;
+}
+
 std::vector<std::string> YamlGameDatabaseImpl::convertMultiLineStringToVector(const std::string multiLineString)
 {
 	std::vector<std::string> lines;
@ -111,7 +147,12 @@ GameDatabaseSchema::GameEntry YamlGameDatabaseImpl::entryFromYaml(const std::str
 		{
 			for (const auto& entry : patches)
 			{
-				std::string crc = entry.first.as<std::string>();
+				std::string crc = strToLower(entry.first.as<std::string>());
+				if (gameEntry.patches.count(crc) == 1)
+				{
+					Console.Error(fmt::format("[GameDB] Duplicate CRC '{}' found for serial: '{}'. Skipping, CRCs are case-insensitive!", crc, serial));
+					continue;
+				}
 				YAML::Node patchNode = entry.second;

 				GameDatabaseSchema::Patch patchCol;
@ -137,9 +178,15 @@ GameDatabaseSchema::GameEntry YamlGameDatabaseImpl::entryFromYaml(const std::str

 GameDatabaseSchema::GameEntry YamlGameDatabaseImpl::findGame(const std::string serial)
 {
-	if (gameDb.count(serial) == 1)
-		return gameDb[serial];
+	std::string serialLower = strToLower(serial);
+	Console.WriteLn(fmt::format("[GameDB] Searching for '{}' in GameDB", serialLower));
+	if (gameDb.count(serialLower) == 1)
+	{
+		Console.WriteLn(fmt::format("[GameDB] Found '{}' in GameDB", serialLower));
+		return gameDb[serialLower];
+	}

+	Console.Error(fmt::format("[GameDB] Could not find '{}' in GameDB", serialLower));
 	GameDatabaseSchema::GameEntry entry;
 	entry.isValid = false;
 	return entry;
@ -168,7 +215,16 @@ bool YamlGameDatabaseImpl::initDatabase(std::ifstream& stream)
 			// but we do want to yell about it so it can be corrected
 			try
 			{
-				std::string serial = entry.first.as<std::string>();
+				// Serials and CRCs must be inserted as lower-case, as that is how they are retrieved
+				// this is because the application may pass a lowercase CRC or serial along
+				//
+				// However, YAML's keys are as expected case-sensitive, so we have to explicitly do our own duplicate checking
+				std::string serial = strToLower(entry.first.as<std::string>());
+				if (gameDb.count(serial) == 1)
+				{
+					Console.Error(fmt::format("[GameDB] Duplicate serial '{}' found in GameDB. Skipping, Serials are case-insensitive!", serial));
+					continue;
+				}
 				gameDb[serial] = entryFromYaml(serial, entry.second);
 			}
 			catch (const YAML::RepresentationException& e)
--- a/pcsx2/GameDatabase.h
+++ b/pcsx2/GameDatabase.h
@ -77,7 +77,8 @@ public:
 		std::vector<std::string> memcardFilters;
 		std::unordered_map<std::string, Patch> patches;

-		std::string memcardFiltersAsString();
+		std::string memcardFiltersAsString() const;
+		bool findPatch(const std::string crc, Patch& patch) const;
 	};
 };

@ -104,3 +105,5 @@ private:
 };

 extern IGameDatabase* AppHost_GetGameDatabase();
+extern std::string strToLower(std::string str);
+extern bool compareStrNoCase(const std::string str1, const std::string str2);
--- a/pcsx2/Patch.cpp
+++ b/pcsx2/Patch.cpp
@ -128,16 +128,8 @@ int LoadPatchesFromGamesDB(const wxString& crc, const GameDatabaseSchema::GameEn
 	if (game.isValid)
 	{
 		GameDatabaseSchema::Patch patch;
-		if (game.patches.count(std::string(crc)) == 1)
-		{
-			patch = game.patches.at(std::string(crc));
-		}
-		else if (game.patches.count("default") == 1)
-		{
-			patch = game.patches.at("default");
-		}
-
-		if (patch.patchLines.size() > 0)
+		bool patchFound = game.findPatch(std::string(crc), patch);
+		if (patchFound && patch.patchLines.size() > 0)
 		{
 			for (auto line : patch.patchLines)
 			{