GameDatabase: Switch to YAML

This commit is contained in:
Stenzek 2024-02-04 02:36:25 +10:00
parent dded138d4a
commit 6aeec82a0c
No known key found for this signature in database
17 changed files with 1077 additions and 735 deletions

View File

@ -25,7 +25,6 @@ add_library(rapidyaml
include/c4/substr_fwd.hpp
include/c4/szconv.hpp
include/c4/types.hpp
include/c4/utf.cpp
include/c4/utf.hpp
include/c4/windows.hpp
include/c4/windows_pop.hpp
@ -69,3 +68,8 @@ target_include_directories(rapidyaml PRIVATE
target_include_directories(rapidyaml INTERFACE
"${CMAKE_CURRENT_SOURCE_DIR}/include"
)
target_compile_definitions(rapidyaml PUBLIC
"C4_NO_DEBUG_BREAK"
)

View File

@ -1,23 +0,0 @@
Copyright (c) 2011-2016, Scott Tsai
All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

View File

@ -1,33 +0,0 @@
# How to use `debugbreak-gdb.py`
Just add `-x debugbreak-gdb.py` to your usual GDB invocation or add `source debugbreak-gdb.py` to your `$HOME/.gdbinit`.
Here's a sample session:
```
$ cd debugbreak
$ make
$ gdb -q -x debugbreak-gdb.py test/break
Reading symbols from test/break...done.
(gdb) set disassemble-next-line 1
(gdb) run
Starting program: /home/fedora/debugbreak/test/break
Program received signal SIGTRAP, Trace/breakpoint trap.
main () at test/break.c:6
6 debug_break();
Program received signal SIGTRAP, Trace/breakpoint trap.
main () at test/break.c:6
6 debug_break();
(gdb) debugbreak-step
7 printf("hello world\n");
(gdb) debugbreak-continue
hello world
[Inferior 1 (process 12533) exited normally]
(gdb)
```
On ARM and POWER, trying to use `step` or `stepi` in place of `debugbreak-step` in the sesion above wouldn't have worked as execution would be stock on the breakpoint instruction.

View File

@ -1,127 +0,0 @@
# Debug Break
[debugbreak.h](https://github.com/scottt/debugbreak/blob/master/debugbreak.h) allows you to put breakpoints in your C/C++ code with a call to **debug_break()**:
```C
#include <stdio.h>
#include "debugbreak.h"
int main()
{
debug_break(); /* will break into debugger */
printf("hello world\n");
return 0;
}
```
* Include one header file and insert calls to `debug_break()` in the code where you wish to break into the debugger.
* Supports GCC, Clang and MSVC.
* Works well on ARM, AArch64, i686, x86-64, POWER and has a fallback code path for other architectures.
* Works like the **DebugBreak()** fuction provided by [Windows](http://msdn.microsoft.com/en-us/library/ea9yy3ey.aspx) and [QNX](http://www.qnx.com/developers/docs/6.3.0SP3/neutrino/lib_ref/d/debugbreak.html).
**License**: the very permissive [2-Clause BSD](https://github.com/scottt/debugbreak/blob/master/COPYING).
Known Problem: if continuing execution after a debugbreak breakpoint hit doesn't work (e.g. on ARM or POWER), see [HOW-TO-USE-DEBUGBREAK-GDB-PY.md](HOW-TO-USE-DEBUGBREAK-GDB-PY.md) for a workaround.
Implementation Notes
================================
The requirements for the **debug_break()** function are:
* Act as a compiler code motion barrier
* Don't cause the compiler optimizers to think the code following it can be removed
* Trigger a software breakpoint hit when executed (e.g. **SIGTRAP** on Linux)
* GDB commands like **continue**, **next**, **step**, **stepi** must work after a **debug_break()** hit
Ideally, both GCC and Clang would provide a **__builtin_debugtrap()** that satisfies the above on all architectures and operating systems. Unfortunately, that is not the case (yet).
GCC's [__builtin_trap()](http://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html#index-g_t_005f_005fbuiltin_005ftrap-3278) causes the optimizers to think the code follwing can be removed ([test/trap.c](https://github.com/scottt/debugbreak/blob/master/test/trap.c)):
```C
#include <stdio.h>
int main()
{
__builtin_trap();
printf("hello world\n");
return 0;
}
```
compiles to:
```
main
0x0000000000400390 <+0>: 0f 0b ud2
```
Notice how the call to `printf()` is not present in the assembly output.
Further, on i386 / x86-64 **__builtin_trap()** generates an **ud2** instruction which triggers **SIGILL** instead of **SIGTRAP**. This makes it necessary to change GDB's default behavior on **SIGILL** to not terminate the process being debugged:
```
(gdb) handle SIGILL stop nopass
```
Even after this, continuing execution in GDB doesn't work well on some GCC, GDB combinations. See [GCC Bugzilla 84595](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84595).
On ARM, **__builtin_trap()** generates a call to **abort()**, making it even less suitable.
**debug_break()** generates an **int3** instruction on i386 / x86-64 ([test/break.c](https://github.com/scottt/debugbreak/blob/master/test/break.c)):
```C
#include <stdio.h>
#include "debugbreak.h"
int main()
{
debug_break();
printf("hello world\n");
return 0;
}
```
compiles to:
```
main
0x00000000004003d0 <+0>: 50 push %rax
0x00000000004003d1 <+1>: cc int3
0x00000000004003d2 <+2>: bf a0 05 40 00 mov $0x4005a0,%edi
0x00000000004003d7 <+7>: e8 d4 ff ff ff callq 0x4003b0 <puts@plt>
0x00000000004003dc <+12>: 31 c0 xor %eax,%eax
0x00000000004003de <+14>: 5a pop %rdx
0x00000000004003df <+15>: c3 retq
```
which correctly trigges **SIGTRAP** and single-stepping in GDB after a **debug_break()** hit works well.
Clang / LLVM also has a **__builtin_trap()** that generates **ud2** but further provides **__builtin_debugtrap()** that generates **int3** on i386 / x86-64 ([original LLVM intrinsic](http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20120507/142621.html), [further fixes](https://reviews.llvm.org/rL166300#96cef7d3), [Clang builtin support](https://reviews.llvm.org/rL166298)).
On ARM, **debug_break()** generates **.inst 0xe7f001f0** in ARM mode and **.inst 0xde01** in Thumb mode which correctly triggers **SIGTRAP** on Linux. Unfortunately, stepping in GDB after a **debug_break()** hit doesn't work and requires a workaround like:
```
(gdb) set $l = 2
(gdb) tbreak *($pc + $l)
(gdb) jump *($pc + $l)
(gdb) # Change $l from 2 to 4 for ARM mode
```
to jump over the instruction.
A new GDB command, **debugbreak-step**, is defined in [debugbreak-gdb.py](https://github.com/scottt/debugbreak/blob/master/debugbreak-gdb.py) to automate the above. See [HOW-TO-USE-DEBUGBREAK-GDB-PY.md](HOW-TO-USE-DEBUGBREAK-GDB-PY.md) for sample usage.
```
$ arm-none-linux-gnueabi-gdb -x debugbreak-gdb.py test/break-c++
<...>
(gdb) run
Program received signal SIGTRAP, Trace/breakpoint trap.
main () at test/break-c++.cc:6
6 debug_break();
(gdb) debugbreak-step
7 std::cout << "hello, world\n";
```
On AArch64, **debug_break()** generates **.inst 0xd4200000**.
See table below for the behavior of **debug_break()** on other architecturs.
Behavior on Different Architectures
----------------
| Architecture | debug_break() |
| ------------- | ------------- |
| x86/x86-64 | `int3` |
| ARM mode, 32-bit | `.inst 0xe7f001f0` |
| Thumb mode, 32-bit | `.inst 0xde01` |
| AArch64, ARMv8 | `.inst 0xd4200000` |
| POWER | `.4byte 0x7d821008` |
| RISC-V | `.4byte 0x00100073` |
| MSVC compiler | `__debugbreak` |
| Apple compiler on AArch64 | `__builtin_trap()` |
| Otherwise | `raise(SIGTRAP)` |

View File

@ -1,183 +0,0 @@
# Copyright (c) 2013, Scott Tsai
#
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
# Usage: gdb -x debugbreak-gdb.py
# (gdb) debugbreak-step
# (gdb) debugbreak-continue
#
# To debug:
# (gdb) set python print-stack full
import gdb
import re
def _gdb_show_version_parse(version_str):
'''
>>> s0 = 'This GDB was configured as "x86_64-redhat-linux-gnu".'
>>> s1 = 'This GDB was configured as "--host=i686-build_pc-linux-gnu --target=arm-linux-gnueabihf".'
>>> s2 = 'This GDB was configured as "x86_64-unknown-linux-gnu".'
>>> _gdb_show_version_parse(s0) == dict(target='x86_64-redhat-linux-gnu')
True
>>> _gdb_show_version_parse(s1) == dict(host='i686-build_pc-linux-gnu', target='arm-linux-gnueabihf')
True
>>> _gdb_show_version_parse(s2) == dict(target='x86_64-unknown-linux-gnu')
True
'''
t = version_str
msg = 'This GDB was configured as "'
s = t.find(msg)
if s == -1:
raise ValueError
s += len(msg)
e = t.find('".', s)
if e == -1:
raise ValueError
config = t[s:e]
d = {}
for i in config.split():
i = i.strip()
if i.startswith('--'):
(k, v) = i[2:].split('=')
d[k] = v
else:
if not i:
continue
d['target'] = i
return d
def _target_triplet():
'''
-> 'arm-linux-gnueabihf' or 'x86_64-redhat-linux-gnu' or ...
>>> import re
>>> not not re.match(r'\w*-\w*-\w*', target_triplet())
True
'''
t = gdb.execute('show version', to_string=True)
return _gdb_show_version_parse(t)['target']
temp_breakpoint_num = None
def on_stop_event(e):
global temp_breakpoint_num
if not isinstance(e, gdb.BreakpointEvent):
return
for bp in e.breakpoints:
if bp.number == temp_breakpoint_num:
bp.delete()
gdb.events.stop.disconnect(on_stop_event)
l = gdb.find_pc_line(int(gdb.parse_and_eval('$pc'))).line
gdb.execute('list %d, %d' % (l, l))
break
def _next_instn_jump_len(gdb_frame):
'-> None means don\'t jump'
try:
arch_name = gdb_frame.architecture().name()
except AttributeError:
arch_name = None
if arch_name.startswith('powerpc:'):
# 'powerpc:common64' on ppc64 big endian
i = bytes(gdb.selected_inferior().read_memory(gdb.parse_and_eval('$pc'), 4))
if (i == b'\x7d\x82\x10\x08') or (i == b'\x08\x10\x82\x7d'):
return 4
else: # not stopped on a breakpoint instruction
return None
triplet = _target_triplet()
if re.match(r'^arm-', triplet):
i = bytes(gdb.selected_inferior().read_memory(gdb.parse_and_eval('$pc'), 4))
if i == b'\xf0\x01\xf0\xe7':
return 4
elif i.startswith(b'\x01\xde'):
return 2
elif i == b'\xf0\xf7\x00\xa0 ':
# 'arm_linux_thumb2_le_breakpoint' from arm-linux-tdep.c in GDB
return 4
else: # not stopped on a breakpoint instruction
return None
return None
def _debugbreak_step():
global temp_breakpoint_num
try:
frame = gdb.selected_frame()
except gdb.error as e:
# 'No frame is currently selected.'
gdb.write(e.args[0] + '\n', gdb.STDERR)
return
instn_len = _next_instn_jump_len(frame)
if instn_len is None:
gdb.execute('stepi')
else:
loc = '*($pc + %d)' % (instn_len,)
bp = gdb.Breakpoint(loc, gdb.BP_BREAKPOINT, internal=True)
bp.silent = True
temp_breakpoint_num = bp.number
gdb.events.stop.connect(on_stop_event)
gdb.execute('jump ' + loc)
def _debugbreak_continue():
try:
frame = gdb.selected_frame()
except gdb.error as e:
# 'No frame is currently selected.'
gdb.write(e.args[0] + '\n', gdb.STDERR)
return
instn_len = _next_instn_jump_len(frame)
if instn_len is None:
gdb.execute('continue')
else:
loc = '*($pc + %d)' % (instn_len,)
gdb.execute('jump ' + loc)
class _DebugBreakStep(gdb.Command):
'''Usage: debugbreak-step
Step one instruction after a debug_break() breakpoint hit'''
def __init__(self):
gdb.Command.__init__(self, 'debugbreak-step', gdb.COMMAND_BREAKPOINTS, gdb.COMPLETE_NONE)
def invoke(self, arg, from_tty):
_debugbreak_step()
class _DebugBreakContinue(gdb.Command):
'''Usage: debugbreak-continue
Continue execution after a debug_break() breakpoint hit'''
def __init__(self):
gdb.Command.__init__(self, 'debugbreak-continue', gdb.COMMAND_BREAKPOINTS, gdb.COMPLETE_NONE)
def invoke(self, arg, from_tty):
_debugbreak_continue()
_DebugBreakStep()
_DebugBreakContinue()

View File

@ -1,174 +0,0 @@
/* Copyright (c) 2011-2021, Scott Tsai
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef DEBUG_BREAK_H
#define DEBUG_BREAK_H
#ifdef _MSC_VER
#define debug_break __debugbreak
#else
#ifdef __cplusplus
extern "C" {
#endif
#define DEBUG_BREAK_USE_TRAP_INSTRUCTION 1
#define DEBUG_BREAK_USE_BULTIN_TRAP 2
#define DEBUG_BREAK_USE_SIGTRAP 3
#if defined(__i386__) || defined(__x86_64__)
#define DEBUG_BREAK_IMPL DEBUG_BREAK_USE_TRAP_INSTRUCTION
__inline__ static void trap_instruction(void)
{
__asm__ volatile("int $0x03");
}
#elif defined(__thumb__)
#define DEBUG_BREAK_IMPL DEBUG_BREAK_USE_TRAP_INSTRUCTION
/* FIXME: handle __THUMB_INTERWORK__ */
__attribute__((always_inline))
__inline__ static void trap_instruction(void)
{
/* See 'arm-linux-tdep.c' in GDB source.
* Both instruction sequences below work. */
#if 1
/* 'eabi_linux_thumb_le_breakpoint' */
__asm__ volatile(".inst 0xde01");
#else
/* 'eabi_linux_thumb2_le_breakpoint' */
__asm__ volatile(".inst.w 0xf7f0a000");
#endif
/* Known problem:
* After a breakpoint hit, can't 'stepi', 'step', or 'continue' in GDB.
* 'step' would keep getting stuck on the same instruction.
*
* Workaround: use the new GDB commands 'debugbreak-step' and
* 'debugbreak-continue' that become available
* after you source the script from GDB:
*
* $ gdb -x debugbreak-gdb.py <... USUAL ARGUMENTS ...>
*
* 'debugbreak-step' would jump over the breakpoint instruction with
* roughly equivalent of:
* (gdb) set $instruction_len = 2
* (gdb) tbreak *($pc + $instruction_len)
* (gdb) jump *($pc + $instruction_len)
*/
}
#elif defined(__arm__) && !defined(__thumb__)
#define DEBUG_BREAK_IMPL DEBUG_BREAK_USE_TRAP_INSTRUCTION
__attribute__((always_inline))
__inline__ static void trap_instruction(void)
{
/* See 'arm-linux-tdep.c' in GDB source,
* 'eabi_linux_arm_le_breakpoint' */
__asm__ volatile(".inst 0xe7f001f0");
/* Known problem:
* Same problem and workaround as Thumb mode */
}
#elif defined(__aarch64__) && defined(__APPLE__)
#define DEBUG_BREAK_IMPL DEBUG_BREAK_USE_BULTIN_DEBUGTRAP
#elif defined(__aarch64__)
#define DEBUG_BREAK_IMPL DEBUG_BREAK_USE_TRAP_INSTRUCTION
__attribute__((always_inline))
__inline__ static void trap_instruction(void)
{
/* See 'aarch64-tdep.c' in GDB source,
* 'aarch64_default_breakpoint' */
__asm__ volatile(".inst 0xd4200000");
}
#elif defined(__powerpc__)
/* PPC 32 or 64-bit, big or little endian */
#define DEBUG_BREAK_IMPL DEBUG_BREAK_USE_TRAP_INSTRUCTION
__attribute__((always_inline))
__inline__ static void trap_instruction(void)
{
/* See 'rs6000-tdep.c' in GDB source,
* 'rs6000_breakpoint' */
__asm__ volatile(".4byte 0x7d821008");
/* Known problem:
* After a breakpoint hit, can't 'stepi', 'step', or 'continue' in GDB.
* 'step' stuck on the same instruction ("twge r2,r2").
*
* The workaround is the same as ARM Thumb mode: use debugbreak-gdb.py
* or manually jump over the instruction. */
}
#elif defined(__riscv)
/* RISC-V 32 or 64-bit, whether the "C" extension
* for compressed, 16-bit instructions are supported or not */
#define DEBUG_BREAK_IMPL DEBUG_BREAK_USE_TRAP_INSTRUCTION
__attribute__((always_inline))
__inline__ static void trap_instruction(void)
{
/* See 'riscv-tdep.c' in GDB source,
* 'riscv_sw_breakpoint_from_kind' */
__asm__ volatile(".4byte 0x00100073");
}
#else
#define DEBUG_BREAK_IMPL DEBUG_BREAK_USE_SIGTRAP
#endif
#ifndef DEBUG_BREAK_IMPL
#error "debugbreak.h is not supported on this target"
#elif DEBUG_BREAK_IMPL == DEBUG_BREAK_USE_TRAP_INSTRUCTION
__attribute__((always_inline))
__inline__ static void debug_break(void)
{
trap_instruction();
}
#elif DEBUG_BREAK_IMPL == DEBUG_BREAK_USE_BULTIN_DEBUGTRAP
__attribute__((always_inline))
__inline__ static void debug_break(void)
{
__builtin_debugtrap();
}
#elif DEBUG_BREAK_IMPL == DEBUG_BREAK_USE_BULTIN_TRAP
__attribute__((always_inline))
__inline__ static void debug_break(void)
{
__builtin_trap();
}
#elif DEBUG_BREAK_IMPL == DEBUG_BREAK_USE_SIGTRAP
#include <signal.h>
__attribute__((always_inline))
__inline__ static void debug_break(void)
{
raise(SIGTRAP);
}
#else
#error "invalid DEBUG_BREAK_IMPL value"
#endif
#ifdef __cplusplus
}
#endif
#endif /* ifdef _MSC_VER */
#endif /* ifndef DEBUG_BREAK_H */

View File

@ -1,60 +0,0 @@
#include "c4/utf.hpp"
#include "c4/charconv.hpp"
namespace c4 {
C4_SUPPRESS_WARNING_GCC_CLANG_WITH_PUSH("-Wold-style-cast")
size_t decode_code_point(uint8_t *C4_RESTRICT buf, size_t buflen, const uint32_t code)
{
C4_UNUSED(buflen);
C4_ASSERT(buflen >= 4);
if (code <= UINT32_C(0x7f))
{
buf[0] = (uint8_t)code;
return 1u;
}
else if(code <= UINT32_C(0x7ff))
{
buf[0] = (uint8_t)(UINT32_C(0xc0) | (code >> 6)); /* 110xxxxx */
buf[1] = (uint8_t)(UINT32_C(0x80) | (code & UINT32_C(0x3f))); /* 10xxxxxx */
return 2u;
}
else if(code <= UINT32_C(0xffff))
{
buf[0] = (uint8_t)(UINT32_C(0xe0) | ((code >> 12))); /* 1110xxxx */
buf[1] = (uint8_t)(UINT32_C(0x80) | ((code >> 6) & UINT32_C(0x3f))); /* 10xxxxxx */
buf[2] = (uint8_t)(UINT32_C(0x80) | ((code ) & UINT32_C(0x3f))); /* 10xxxxxx */
return 3u;
}
else if(code <= UINT32_C(0x10ffff))
{
buf[0] = (uint8_t)(UINT32_C(0xf0) | ((code >> 18))); /* 11110xxx */
buf[1] = (uint8_t)(UINT32_C(0x80) | ((code >> 12) & UINT32_C(0x3f))); /* 10xxxxxx */
buf[2] = (uint8_t)(UINT32_C(0x80) | ((code >> 6) & UINT32_C(0x3f))); /* 10xxxxxx */
buf[3] = (uint8_t)(UINT32_C(0x80) | ((code ) & UINT32_C(0x3f))); /* 10xxxxxx */
return 4u;
}
return 0;
}
substr decode_code_point(substr out, csubstr code_point)
{
C4_ASSERT(out.len >= 4);
C4_ASSERT(!code_point.begins_with("U+"));
C4_ASSERT(!code_point.begins_with("\\x"));
C4_ASSERT(!code_point.begins_with("\\u"));
C4_ASSERT(!code_point.begins_with("\\U"));
C4_ASSERT(!code_point.begins_with('0'));
C4_ASSERT(code_point.len <= 8);
C4_ASSERT(code_point.len > 0);
uint32_t code_point_val;
C4_CHECK(read_hex(code_point, &code_point_val));
size_t ret = decode_code_point((uint8_t*)out.str, out.len, code_point_val);
C4_ASSERT(ret <= 4);
return out.first(ret);
}
C4_SUPPRESS_WARNING_GCC_CLANG_POP
} // namespace c4

View File

@ -0,0 +1,200 @@
#ifndef C4_YML_DETAIL_CHECKS_HPP_
#define C4_YML_DETAIL_CHECKS_HPP_
#include "c4/yml/tree.hpp"
#ifdef __clang__
# pragma clang diagnostic push
#elif defined(__GNUC__)
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wtype-limits" // error: comparison of unsigned expression >= 0 is always true
#elif defined(_MSC_VER)
# pragma warning(push)
# pragma warning(disable: 4296/*expression is always 'boolean_value'*/)
#endif
namespace c4 {
namespace yml {
void check_invariants(Tree const& t, size_t node=NONE);
void check_free_list(Tree const& t);
void check_arena(Tree const& t);
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
inline void check_invariants(Tree const& t, size_t node)
{
if(node == NONE)
{
if(t.size() == 0) return;
node = t.root_id();
}
auto const& n = *t._p(node);
#ifdef RYML_DBG
if(n.m_first_child != NONE || n.m_last_child != NONE)
{
printf("check(%zu): fc=%zu lc=%zu\n", node, n.m_first_child, n.m_last_child);
}
else
{
printf("check(%zu)\n", node);
}
#endif
C4_CHECK(n.m_parent != node);
if(n.m_parent == NONE)
{
C4_CHECK(t.is_root(node));
}
else //if(n.m_parent != NONE)
{
C4_CHECK(t.has_child(n.m_parent, node));
auto const& p = *t._p(n.m_parent);
if(n.m_prev_sibling == NONE)
{
C4_CHECK(p.m_first_child == node);
C4_CHECK(t.first_sibling(node) == node);
}
else
{
C4_CHECK(p.m_first_child != node);
C4_CHECK(t.first_sibling(node) != node);
}
if(n.m_next_sibling == NONE)
{
C4_CHECK(p.m_last_child == node);
C4_CHECK(t.last_sibling(node) == node);
}
else
{
C4_CHECK(p.m_last_child != node);
C4_CHECK(t.last_sibling(node) != node);
}
}
C4_CHECK(n.m_first_child != node);
C4_CHECK(n.m_last_child != node);
if(n.m_first_child != NONE || n.m_last_child != NONE)
{
C4_CHECK(n.m_first_child != NONE);
C4_CHECK(n.m_last_child != NONE);
}
C4_CHECK(n.m_prev_sibling != node);
C4_CHECK(n.m_next_sibling != node);
if(n.m_prev_sibling != NONE)
{
C4_CHECK(t._p(n.m_prev_sibling)->m_next_sibling == node);
C4_CHECK(t._p(n.m_prev_sibling)->m_prev_sibling != node);
}
if(n.m_next_sibling != NONE)
{
C4_CHECK(t._p(n.m_next_sibling)->m_prev_sibling == node);
C4_CHECK(t._p(n.m_next_sibling)->m_next_sibling != node);
}
size_t count = 0;
for(size_t i = n.m_first_child; i != NONE; i = t.next_sibling(i))
{
#ifdef RYML_DBG
printf("check(%zu): descend to child[%zu]=%zu\n", node, count, i);
#endif
auto const& ch = *t._p(i);
C4_CHECK(ch.m_parent == node);
C4_CHECK(ch.m_next_sibling != i);
++count;
}
C4_CHECK(count == t.num_children(node));
if(n.m_prev_sibling == NONE && n.m_next_sibling == NONE)
{
if(n.m_parent != NONE)
{
C4_CHECK(t.num_children(n.m_parent) == 1);
C4_CHECK(t.num_siblings(node) == 1);
}
}
if(node == t.root_id())
{
C4_CHECK(t.size() == t.m_size);
C4_CHECK(t.capacity() == t.m_cap);
C4_CHECK(t.m_cap == t.m_size + t.slack());
check_free_list(t);
check_arena(t);
}
for(size_t i = t.first_child(node); i != NONE; i = t.next_sibling(i))
{
check_invariants(t, i);
}
}
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
inline void check_free_list(Tree const& t)
{
if(t.m_free_head == NONE)
{
C4_CHECK(t.m_free_tail == t.m_free_head);
return;
}
C4_CHECK(t.m_free_head >= 0 && t.m_free_head < t.m_cap);
C4_CHECK(t.m_free_tail >= 0 && t.m_free_tail < t.m_cap);
auto const& head = *t._p(t.m_free_head);
//auto const& tail = *t._p(t.m_free_tail);
//C4_CHECK(head.m_prev_sibling == NONE);
//C4_CHECK(tail.m_next_sibling == NONE);
size_t count = 0;
for(size_t i = t.m_free_head, prev = NONE; i != NONE; i = t._p(i)->m_next_sibling)
{
auto const& elm = *t._p(i);
if(&elm != &head)
{
C4_CHECK(elm.m_prev_sibling == prev);
}
prev = i;
++count;
}
C4_CHECK(count == t.slack());
}
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
inline void check_arena(Tree const& t)
{
C4_CHECK(t.m_arena.len == 0 || (t.m_arena_pos >= 0 && t.m_arena_pos <= t.m_arena.len));
C4_CHECK(t.arena_size() == t.m_arena_pos);
C4_CHECK(t.arena_slack() + t.m_arena_pos == t.m_arena.len);
}
} /* namespace yml */
} /* namespace c4 */
#ifdef __clang__
# pragma clang diagnostic pop
#elif defined(__GNUC__)
# pragma GCC diagnostic pop
#elif defined(_MSC_VER)
# pragma warning(pop)
#endif
#endif /* C4_YML_DETAIL_CHECKS_HPP_ */

View File

@ -0,0 +1,130 @@
#ifndef C4_YML_DETAIL_PRINT_HPP_
#define C4_YML_DETAIL_PRINT_HPP_
#include "c4/yml/tree.hpp"
#include "c4/yml/node.hpp"
namespace c4 {
namespace yml {
C4_SUPPRESS_WARNING_GCC_CLANG_WITH_PUSH("-Wold-style-cast")
inline size_t print_node(Tree const& p, size_t node, int level, size_t count, bool print_children)
{
printf("[%zd]%*s[%zd] %p", count, (2*level), "", node, (void const*)p.get(node));
if(p.is_root(node))
{
printf(" [ROOT]");
}
printf(" %s:", p.type_str(node));
if(p.has_key(node))
{
if(p.has_key_anchor(node))
{
csubstr ka = p.key_anchor(node);
printf(" &%.*s", (int)ka.len, ka.str);
}
if(p.has_key_tag(node))
{
csubstr kt = p.key_tag(node);
csubstr k = p.key(node);
printf(" %.*s '%.*s'", (int)kt.len, kt.str, (int)k.len, k.str);
}
else
{
csubstr k = p.key(node);
printf(" '%.*s'", (int)k.len, k.str);
}
}
else
{
RYML_ASSERT( ! p.has_key_tag(node));
}
if(p.has_val(node))
{
if(p.has_val_tag(node))
{
csubstr vt = p.val_tag(node);
csubstr v = p.val(node);
printf(" %.*s '%.*s'", (int)vt.len, vt.str, (int)v.len, v.str);
}
else
{
csubstr v = p.val(node);
printf(" '%.*s'", (int)v.len, v.str);
}
}
else
{
if(p.has_val_tag(node))
{
csubstr vt = p.val_tag(node);
printf(" %.*s", (int)vt.len, vt.str);
}
}
if(p.has_val_anchor(node))
{
auto &a = p.val_anchor(node);
printf(" valanchor='&%.*s'", (int)a.len, a.str);
}
printf(" (%zd sibs)", p.num_siblings(node));
++count;
if(p.is_container(node))
{
printf(" %zd children:\n", p.num_children(node));
if(print_children)
{
for(size_t i = p.first_child(node); i != NONE; i = p.next_sibling(i))
{
count = print_node(p, i, level+1, count, print_children);
}
}
}
else
{
printf("\n");
}
return count;
}
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
inline void print_node(ConstNodeRef const& p, int level=0)
{
print_node(*p.tree(), p.id(), level, 0, true);
}
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------
inline size_t print_tree(Tree const& p, size_t node=NONE)
{
printf("--------------------------------------\n");
size_t ret = 0;
if(!p.empty())
{
if(node == NONE)
node = p.root_id();
ret = print_node(p, node, 0, 0, true);
}
printf("#nodes=%zd vs #printed=%zd\n", p.size(), ret);
printf("--------------------------------------\n");
return ret;
}
C4_SUPPRESS_WARNING_GCC_CLANG_POP
} /* namespace yml */
} /* namespace c4 */
#endif /* C4_YML_DETAIL_PRINT_HPP_ */

View File

@ -4,19 +4,6 @@
<PropertyGroup Label="Globals">
<ProjectGuid>{1AD23A8A-4C20-434C-AE6B-0E07759EEB1E}</ProjectGuid>
</PropertyGroup>
<ItemGroup>
<ClCompile Include="src\c4\base64.cpp" />
<ClCompile Include="src\c4\error.cpp" />
<ClCompile Include="src\c4\format.cpp" />
<ClCompile Include="src\c4\language.cpp" />
<ClCompile Include="src\c4\memory_util.cpp" />
<ClCompile Include="src\c4\utf.cpp" />
<ClCompile Include="src\c4\yml\common.cpp" />
<ClCompile Include="src\c4\yml\node.cpp" />
<ClCompile Include="src\c4\yml\parse.cpp" />
<ClCompile Include="src\c4\yml\preprocess.cpp" />
<ClCompile Include="src\c4\yml\tree.cpp" />
</ItemGroup>
<ItemGroup>
<ClInclude Include="include\c4\base64.hpp" />
<ClInclude Include="include\c4\blob.hpp" />
@ -49,7 +36,9 @@
<ClInclude Include="include\c4\windows_pop.hpp" />
<ClInclude Include="include\c4\windows_push.hpp" />
<ClInclude Include="include\c4\yml\common.hpp" />
<ClInclude Include="include\c4\yml\detail\checks.hpp" />
<ClInclude Include="include\c4\yml\detail\parser_dbg.hpp" />
<ClInclude Include="include\c4\yml\detail\print.hpp" />
<ClInclude Include="include\c4\yml\detail\stack.hpp" />
<ClInclude Include="include\c4\yml\emit.def.hpp" />
<ClInclude Include="include\c4\yml\emit.hpp" />
@ -64,16 +53,33 @@
<ClInclude Include="include\c4\yml\tree.hpp" />
<ClInclude Include="include\c4\yml\writer.hpp" />
<ClInclude Include="include\c4\yml\yml.hpp" />
<ClInclude Include="include\ryml.hpp" />
<ClInclude Include="include\ryml_std.hpp" />
</ItemGroup>
<ItemGroup>
<Natvis Include="include\c4\c4core.natvis" />
<Natvis Include="include\ryml.natvis" />
</ItemGroup>
<ItemGroup>
<ClCompile Include="src\c4\base64.cpp" />
<ClCompile Include="src\c4\error.cpp" />
<ClCompile Include="src\c4\format.cpp" />
<ClCompile Include="src\c4\language.cpp" />
<ClCompile Include="src\c4\memory_util.cpp" />
<ClCompile Include="src\c4\utf.cpp" />
<ClCompile Include="src\c4\yml\common.cpp" />
<ClCompile Include="src\c4\yml\node.cpp" />
<ClCompile Include="src\c4\yml\parse.cpp" />
<ClCompile Include="src\c4\yml\preprocess.cpp" />
<ClCompile Include="src\c4\yml\tree.cpp" />
</ItemGroup>
<Import Project="..\msvc\vsprops\StaticLibrary.props" />
<ItemDefinitionGroup>
<ClCompile>
<WarningLevel>TurnOffAllWarnings</WarningLevel>
<AdditionalIncludeDirectories>$(ProjectDir)src;$(ProjectDir)include;$(ProjectDir)..\fast_float\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<PreprocessorDefinitions>C4_NO_DEBUG_BREAK;%(PreprocessorDefinitions)</PreprocessorDefinitions>
</ClCompile>
</ItemDefinitionGroup>
<Import Project="..\msvc\vsprops\Targets.props" />
</Project>
</Project>

View File

@ -4,8 +4,192 @@
<Filter Include="yml">
<UniqueIdentifier>{c002fb0c-6a19-4827-8a7a-19e0a309fb0c}</UniqueIdentifier>
</Filter>
<Filter Include="yml\detail">
<UniqueIdentifier>{a3c32dd2-c04e-4122-8aa7-75c26954f557}</UniqueIdentifier>
</Filter>
<Filter Include="yml\std">
<UniqueIdentifier>{df1d37c9-df20-47f2-9031-60d4fa1f8c69}</UniqueIdentifier>
</Filter>
<Filter Include="c4">
<UniqueIdentifier>{6df19960-803e-4406-a9e6-43eaa256d94d}</UniqueIdentifier>
</Filter>
<Filter Include="c4\std">
<UniqueIdentifier>{c53d424e-00a4-4c3c-89f4-f67c57cdee8f}</UniqueIdentifier>
</Filter>
</ItemGroup>
<ItemGroup>
<ClInclude Include="include\c4\yml\detail\parser_dbg.hpp">
<Filter>yml\detail</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\detail\print.hpp">
<Filter>yml\detail</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\detail\stack.hpp">
<Filter>yml\detail</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\detail\checks.hpp">
<Filter>yml\detail</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\std\string.hpp">
<Filter>yml\std</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\std\vector.hpp">
<Filter>yml\std</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\std\map.hpp">
<Filter>yml\std</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\std\std.hpp">
<Filter>yml\std</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\yml.hpp">
<Filter>yml</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\common.hpp">
<Filter>yml</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\emit.def.hpp">
<Filter>yml</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\emit.hpp">
<Filter>yml</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\export.hpp">
<Filter>yml</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\node.hpp">
<Filter>yml</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\parse.hpp">
<Filter>yml</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\preprocess.hpp">
<Filter>yml</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\tree.hpp">
<Filter>yml</Filter>
</ClInclude>
<ClInclude Include="include\c4\yml\writer.hpp">
<Filter>yml</Filter>
</ClInclude>
<ClInclude Include="include\c4\base64.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\blob.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\charconv.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\compiler.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\config.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\cpu.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\dump.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\error.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\export.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\format.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\language.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\memory_util.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\platform.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\preprocessor.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\substr.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\substr_fwd.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\szconv.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\types.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\utf.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\windows.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\windows_pop.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\windows_push.hpp">
<Filter>c4</Filter>
</ClInclude>
<ClInclude Include="include\c4\std\string.hpp">
<Filter>c4\std</Filter>
</ClInclude>
<ClInclude Include="include\c4\std\string_fwd.hpp">
<Filter>c4\std</Filter>
</ClInclude>
<ClInclude Include="include\c4\std\string_view.hpp">
<Filter>c4\std</Filter>
</ClInclude>
<ClInclude Include="include\c4\std\tuple.hpp">
<Filter>c4\std</Filter>
</ClInclude>
<ClInclude Include="include\c4\std\vector.hpp">
<Filter>c4\std</Filter>
</ClInclude>
<ClInclude Include="include\c4\std\vector_fwd.hpp">
<Filter>c4\std</Filter>
</ClInclude>
<ClInclude Include="include\c4\std\std.hpp">
<Filter>c4\std</Filter>
</ClInclude>
<ClInclude Include="include\c4\std\std_fwd.hpp">
<Filter>c4\std</Filter>
</ClInclude>
<ClInclude Include="include\ryml.hpp" />
<ClInclude Include="include\ryml_std.hpp" />
</ItemGroup>
<ItemGroup>
<Natvis Include="include\c4\c4core.natvis">
<Filter>c4</Filter>
</Natvis>
<Natvis Include="include\ryml.natvis" />
</ItemGroup>
<ItemGroup>
<ClCompile Include="src\c4\base64.cpp">
<Filter>c4</Filter>
</ClCompile>
<ClCompile Include="src\c4\error.cpp">
<Filter>c4</Filter>
</ClCompile>
<ClCompile Include="src\c4\format.cpp">
<Filter>c4</Filter>
</ClCompile>
<ClCompile Include="src\c4\language.cpp">
<Filter>c4</Filter>
</ClCompile>
<ClCompile Include="src\c4\memory_util.cpp">
<Filter>c4</Filter>
</ClCompile>
<ClCompile Include="src\c4\utf.cpp">
<Filter>c4</Filter>
</ClCompile>
<ClCompile Include="src\c4\yml\parse.cpp">
<Filter>yml</Filter>
</ClCompile>

View File

@ -131,7 +131,7 @@ target_precompile_headers(core PRIVATE "pch.h")
target_include_directories(core PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/..")
target_include_directories(core PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}/..")
target_link_libraries(core PUBLIC Threads::Threads common util ZLIB::ZLIB)
target_link_libraries(core PRIVATE stb xxhash imgui rapidjson rcheevos)
target_link_libraries(core PRIVATE stb xxhash imgui rapidyaml rapidjson rcheevos)
if(CPU_ARCH_X64)
target_compile_definitions(core PUBLIC "ENABLE_RECOMPILER=1" "ENABLE_NEWREC=1" "ENABLE_MMAP_FASTMEM=1")

View File

@ -11,6 +11,10 @@
<PreprocessorDefinitions Condition="('$(Platform)'=='x64' Or '$(Platform)'=='ARM64')">ENABLE_NEWREC=1;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>%(AdditionalIncludeDirectories);$(SolutionDir)dep\zlib\include;$(SolutionDir)dep\rcheevos\include;$(SolutionDir)dep\rapidjson\include;$(SolutionDir)dep\discord-rpc\include</AdditionalIncludeDirectories>
<PreprocessorDefinitions>%(PreprocessorDefinitions);C4_NO_DEBUG_BREAK=1</PreprocessorDefinitions>
<AdditionalIncludeDirectories>%(AdditionalIncludeDirectories);$(SolutionDir)dep\rapidyaml\include;$(SolutionDir)dep\rapidjson\include</AdditionalIncludeDirectories>
<AdditionalIncludeDirectories Condition="'$(Platform)'!='ARM64'">%(AdditionalIncludeDirectories);$(SolutionDir)dep\rainterface</AdditionalIncludeDirectories>
<AdditionalIncludeDirectories Condition="'$(Platform)'=='x64'">%(AdditionalIncludeDirectories);$(SolutionDir)dep\xbyak\xbyak</AdditionalIncludeDirectories>

View File

@ -172,6 +172,9 @@
<ProjectReference Include="..\..\dep\rainterface\rainterface.vcxproj" Condition="'$(Platform)'!='ARM64'">
<Project>{e4357877-d459-45c7-b8f6-dcbb587bb528}</Project>
</ProjectReference>
<ProjectReference Include="..\..\dep\rapidyaml\rapidyaml.vcxproj">
<Project>{1ad23a8a-4c20-434c-ae6b-0e07759eeb1e}</Project>
</ProjectReference>
<ProjectReference Include="..\..\dep\rcheevos\rcheevos.vcxproj">
<Project>{4ba0a6d4-3ae1-42b2-9347-096fd023ff64}</Project>
</ProjectReference>

View File

@ -18,19 +18,33 @@
#include "rapidjson/document.h"
#include "rapidjson/error/en.h"
#include "ryml.hpp"
#include "ryml_std.hpp" // TODO: Remove this?
#include <iomanip>
#include <memory>
#include <optional>
#include <sstream>
#include <type_traits>
#include "IconsFontAwesome5.h"
Log_SetChannel(GameDatabase);
#ifdef _WIN32
#include "common/windows_headers.h"
#endif
ALWAYS_INLINE std::string_view to_stringview(const c4::csubstr& s)
{
return std::string_view(s.data(), s.size());
}
ALWAYS_INLINE std::string_view to_stringview(const c4::substr& s)
{
return std::string_view(s.data(), s.size());
}
ALWAYS_INLINE c4::csubstr to_csubstr(const std::string_view& sv)
{
return c4::csubstr(sv.data(), sv.length());
}
namespace GameDatabase {
@ -46,12 +60,51 @@ static const Entry* GetEntryForId(const std::string_view& code);
static bool LoadFromCache();
static bool SaveToCache();
static void SetRymlCallbacks();
static bool LoadGameDBJson();
static bool LoadGameDBYaml();
static bool ParseJsonEntry(Entry* entry, const rapidjson::Value& value);
static bool ParseJsonCodes(u32 index, const rapidjson::Value& value);
static bool ParseYamlEntry(Entry* entry, const ryml::ConstNodeRef& value);
static bool ParseYamlCodes(u32 index, const ryml::ConstNodeRef& value, std::string_view serial);
static bool LoadTrackHashes();
static std::array<const char*, static_cast<int>(CompatibilityRating::Count)> s_compatibility_rating_names = {
{"Unknown", "DoesntBoot", "CrashesInIntro", "CrashesInGame", "GraphicalAudioIssues", "NoIssues"}};
static constexpr std::array<const char*, static_cast<size_t>(CompatibilityRating::Count)>
s_compatibility_rating_display_names = {{TRANSLATE_NOOP("GameListCompatibilityRating", "Unknown"),
TRANSLATE_NOOP("GameListCompatibilityRating", "Doesn't Boot"),
TRANSLATE_NOOP("GameListCompatibilityRating", "Crashes In Intro"),
TRANSLATE_NOOP("GameListCompatibilityRating", "Crashes In-Game"),
TRANSLATE_NOOP("GameListCompatibilityRating", "Graphical/Audio Issues"),
TRANSLATE_NOOP("GameListCompatibilityRating", "No Issues")}};
static std::array<const char*, static_cast<u32>(GameDatabase::Trait::Count)> s_trait_names = {{
"forceInterpreter",
"forceSoftwareRenderer",
"forceSoftwareRendererForReadbacks",
"forceInterlacing",
"disableTrueColor",
"disableUpscaling",
"disableTextureFiltering",
"disableScaledDithering",
"disableForceNTSCTimings",
"disableWidescreen",
"disablePGXP",
"disablePGXPCulling",
"disablePGXPTextureCorrection",
"disablePGXPColorCorrection",
"disablePGXPDepthBuffer",
"forcePGXPVertexCache",
"forcePGXPCPUMode",
"forceRecompilerMemoryExceptions",
"forceRecompilerICache",
"forceRecompilerLUTFastmem",
"isLibCryptProtected",
}};
static std::array<const char*, static_cast<u32>(GameDatabase::Trait::Count)> s_jsontrait_names = {{
"ForceInterpreter",
"ForceSoftwareRenderer",
"ForceSoftwareRendererForReadbacks",
@ -93,6 +146,61 @@ void GameDatabase::EnsureLoaded()
s_loaded = true;
LoadGameDBYaml();
auto entries = std::move(s_entries);
auto code_lookup = std::move(s_code_lookup);
s_entries = {};
s_code_lookup = {};
LoadGameDBJson();
Assert(code_lookup.size() == s_code_lookup.size());
for (const auto& it : code_lookup)
{
const auto oit = s_code_lookup.find(it.first);
Assert(oit != s_code_lookup.end() && oit->second == it.second);
}
Assert(entries.size() == s_entries.size());
for (size_t i = 0; i < entries.size(); i++)
{
const GameDatabase::Entry& ee = s_entries[i];
const GameDatabase::Entry& oe = entries[i];
#define CHE(f) Assert(ee.f == oe.f)
CHE(serial);
CHE(title);
CHE(genre);
CHE(developer);
CHE(publisher);
CHE(release_date);
CHE(min_players);
CHE(max_players);
CHE(min_blocks);
CHE(max_blocks);
CHE(supported_controllers);
CHE(compatibility);
CHE(traits);
CHE(display_active_start_offset);
CHE(display_active_end_offset);
CHE(display_line_start_offset);
CHE(display_line_end_offset);
CHE(dma_max_slice_ticks);
CHE(dma_halt_ticks);
CHE(gpu_fifo_size);
CHE(gpu_max_run_ahead);
CHE(gpu_pgxp_tolerance);
CHE(gpu_pgxp_depth_threshold);
CHE(disc_set_name);
CHE(disc_set_serials);
#undef CHE
}
LoadTrackHashes();
#if 0
if (!LoadFromCache())
{
s_entries = {};
@ -101,6 +209,7 @@ void GameDatabase::EnsureLoaded()
LoadGameDBJson();
SaveToCache();
}
#endif
Log_InfoPrintf("Database load took %.2f ms", timer.GetTimeMilliseconds());
}
@ -198,30 +307,16 @@ GameDatabase::Entry* GameDatabase::GetMutableEntry(const std::string_view& seria
return nullptr;
}
const char* GameDatabase::GetTraitName(Trait trait)
{
DebugAssert(trait < Trait::Count);
return s_trait_names[static_cast<u32>(trait)];
}
const char* GameDatabase::GetCompatibilityRatingName(CompatibilityRating rating)
{
static std::array<const char*, static_cast<int>(CompatibilityRating::Count)> names = {
{"Unknown", "DoesntBoot", "CrashesInIntro", "CrashesInGame", "GraphicalAudioIssues", "NoIssues"}};
return names[static_cast<int>(rating)];
return s_compatibility_rating_names[static_cast<int>(rating)];
}
const char* GameDatabase::GetCompatibilityRatingDisplayName(CompatibilityRating rating)
{
static constexpr std::array<const char*, static_cast<size_t>(CompatibilityRating::Count)> names = {
{TRANSLATE_NOOP("GameListCompatibilityRating", "Unknown"),
TRANSLATE_NOOP("GameListCompatibilityRating", "Doesn't Boot"),
TRANSLATE_NOOP("GameListCompatibilityRating", "Crashes In Intro"),
TRANSLATE_NOOP("GameListCompatibilityRating", "Crashes In-Game"),
TRANSLATE_NOOP("GameListCompatibilityRating", "Graphical/Audio Issues"),
TRANSLATE_NOOP("GameListCompatibilityRating", "No Issues")}};
return (rating >= CompatibilityRating::Unknown && rating < CompatibilityRating::Count) ?
Host::TranslateToCString("GameListCompatibilityRating", names[static_cast<int>(rating)]) :
Host::TranslateToCString("GameListCompatibilityRating",
s_compatibility_rating_display_names[static_cast<int>(rating)]) :
"";
}
@ -580,7 +675,7 @@ bool GameDatabase::LoadFromCache()
return false;
}
const u64 gamedb_ts = Host::GetResourceFileTimestamp("gamedb.json", false).value_or(0);
const u64 gamedb_ts = Host::GetResourceFileTimestamp("gamedb.yaml", false).value_or(0);
u32 signature, version, num_entries, num_codes;
u64 file_gamedb_ts;
@ -674,7 +769,7 @@ bool GameDatabase::LoadFromCache()
bool GameDatabase::SaveToCache()
{
const u64 gamedb_ts = Host::GetResourceFileTimestamp("gamedb.json", false).value_or(0);
const u64 gamedb_ts = Host::GetResourceFileTimestamp("gamedb.yaml", false).value_or(0);
std::unique_ptr<ByteStream> stream(
ByteStream::OpenFile(GetCacheFile().c_str(), BYTESTREAM_OPEN_CREATE | BYTESTREAM_OPEN_WRITE |
@ -757,6 +852,21 @@ static bool GetStringFromObject(const rapidjson::Value& object, const char* key,
return true;
}
static bool GetStringFromObject(const ryml::ConstNodeRef& object, std::string_view key, std::string* dest)
{
dest->clear();
const ryml::ConstNodeRef member = object.find_child(to_csubstr(key));
if (!member.valid())
return false;
const c4::csubstr val = member.val();
if (!val.empty())
dest->assign(val.data(), val.size());
return true;
}
static bool GetBoolFromObject(const rapidjson::Value& object, const char* key, bool* dest)
{
*dest = false;
@ -769,6 +879,32 @@ static bool GetBoolFromObject(const rapidjson::Value& object, const char* key, b
return true;
}
static bool GetBoolFromObject(const ryml::ConstNodeRef& object, std::string_view key, bool* dest)
{
*dest = false;
const ryml::ConstNodeRef member = object.find_child(to_csubstr(key));
if (!member.valid())
return false;
const c4::csubstr val = member.val();
if (val.empty())
{
Log_ErrorFmt("Unexpected empty value in {}", key);
return false;
}
const std::optional<bool> opt_value = StringUtil::FromChars<bool>(to_stringview(val));
if (!opt_value.has_value())
{
Log_ErrorFmt("Unexpected non-bool value in {}", key);
return false;
}
*dest = opt_value.value();
return true;
}
template<typename T>
static bool GetUIntFromObject(const rapidjson::Value& object, const char* key, T* dest)
{
@ -782,20 +918,31 @@ static bool GetUIntFromObject(const rapidjson::Value& object, const char* key, T
return true;
}
static bool GetArrayOfStringsFromObject(const rapidjson::Value& object, const char* key, std::vector<std::string>* dest)
template<typename T>
static bool GetUIntFromObject(const ryml::ConstNodeRef& object, std::string_view key, T* dest)
{
dest->clear();
auto member = object.FindMember(key);
if (member == object.MemberEnd() || !member->value.IsArray())
*dest = 0;
const ryml::ConstNodeRef member = object.find_child(to_csubstr(key));
if (!member.valid())
return false;
for (const rapidjson::Value& str : member->value.GetArray())
const c4::csubstr val = member.val();
if (val.empty())
{
if (str.IsString())
{
dest->emplace_back(str.GetString(), str.GetStringLength());
}
Log_ErrorFmt("Unexpected empty value in {}", key);
return false;
}
// TODO: Check for int??
const std::optional<T> opt_value = StringUtil::FromChars<T>(to_stringview(val));
if (!opt_value.has_value())
{
Log_ErrorFmt("Unexpected non-uint value in {}", key);
return false;
}
*dest = opt_value.value();
return true;
}
@ -828,6 +975,35 @@ static std::optional<float> GetOptionalFloatFromObject(const rapidjson::Value& o
return member->value.GetFloat();
}
template<typename T>
static std::optional<T> GetOptionalTFromObject(const ryml::ConstNodeRef& object, std::string_view key)
{
std::optional<T> ret;
const ryml::ConstNodeRef member = object.find_child(to_csubstr(key));
if (member.valid())
{
const c4::csubstr val = member.val();
if (!val.empty())
{
ret = StringUtil::FromChars<T>(to_stringview(val));
if (!ret.has_value())
{
if constexpr (std::is_floating_point_v<T>)
Log_ErrorFmt("Unexpected non-float value in {}", key);
else if constexpr (std::is_integral_v<T>)
Log_ErrorFmt("Unexpected non-int value in {}", key);
}
}
else
{
Log_ErrorFmt("Unexpected empty value in {}", key);
}
}
return ret;
}
bool GameDatabase::LoadGameDBJson()
{
std::optional<std::string> gamedb_data(Host::ReadResourceFileToString("gamedb.json", false));
@ -874,6 +1050,264 @@ bool GameDatabase::LoadGameDBJson()
return true;
}
void GameDatabase::SetRymlCallbacks()
{
ryml::Callbacks callbacks = ryml::get_callbacks();
callbacks.m_error = [](const char* msg, size_t msg_len, ryml::Location loc, void* userdata) {
Log_ErrorFmt("Parse error at {}:{} (bufpos={}): {}", loc.line, loc.col, loc.offset, std::string_view(msg, msg_len));
};
ryml::set_callbacks(callbacks);
c4::set_error_callback(
[](const char* msg, size_t msg_size) { Log_ErrorFmt("C4 error: {}", std::string_view(msg, msg_size)); });
}
bool GameDatabase::LoadGameDBYaml()
{
Common::Timer timer;
const std::optional<std::string> gamedb_data = Host::ReadResourceFileToString("gamedb.yaml", false);
if (!gamedb_data.has_value())
{
Log_ErrorPrint("Failed to read game database");
return false;
}
SetRymlCallbacks();
const ryml::Tree tree = ryml::parse_in_arena(c4::to_csubstr(gamedb_data.value()));
const ryml::ConstNodeRef root = tree.rootref();
s_entries.reserve(root.num_children());
for (const ryml::ConstNodeRef& current : root.children())
{
// TODO: binary sort
const u32 index = static_cast<u32>(s_entries.size());
Entry& entry = s_entries.emplace_back();
if (!ParseYamlEntry(&entry, current))
{
s_entries.pop_back();
continue;
}
ParseYamlCodes(index, current, entry.serial);
}
ryml::reset_callbacks();
Log_InfoFmt("Loaded {} entries and {} codes from database in {:.0f}ms.", s_entries.size(), s_code_lookup.size(),
timer.GetTimeMilliseconds());
return !s_entries.empty();
}
bool GameDatabase::ParseYamlEntry(Entry* entry, const ryml::ConstNodeRef& value)
{
entry->serial = to_stringview(value.key());
if (entry->serial.empty())
{
Log_ErrorPrint("Missing serial for entry.");
return false;
}
GetStringFromObject(value, "name", &entry->title);
if (const ryml::ConstNodeRef metadata = value.find_child(to_csubstr("metadata")); metadata.valid())
{
GetStringFromObject(metadata, "genre", &entry->genre);
GetStringFromObject(metadata, "developer", &entry->developer);
GetStringFromObject(metadata, "publisher", &entry->publisher);
GetUIntFromObject(metadata, "minPlayers", &entry->min_players);
GetUIntFromObject(metadata, "maxPlayers", &entry->max_players);
GetUIntFromObject(metadata, "minBlocks", &entry->min_blocks);
GetUIntFromObject(metadata, "maxBlocks", &entry->max_blocks);
entry->release_date = 0;
{
std::string release_date;
if (GetStringFromObject(metadata, "releaseDate", &release_date))
{
std::istringstream iss(release_date);
struct tm parsed_time = {};
iss >> std::get_time(&parsed_time, "%Y-%m-%d");
if (!iss.fail())
{
parsed_time.tm_isdst = 0;
#ifdef _WIN32
entry->release_date = _mkgmtime(&parsed_time);
#else
entry->release_date = timegm(&parsed_time);
#endif
}
}
}
}
entry->supported_controllers = static_cast<u16>(~0u);
if (const ryml::ConstNodeRef controllers = value.find_child(to_csubstr("controllers"));
controllers.valid() && controllers.has_children())
{
bool first = true;
for (const ryml::ConstNodeRef& controller : controllers.children())
{
const std::string_view controller_str = to_stringview(controller.val());
if (controller_str.empty())
{
Log_WarningFmt("controller is not a string in {}", entry->serial);
return false;
}
std::optional<ControllerType> ctype = Settings::ParseControllerTypeName(controller_str);
if (!ctype.has_value())
{
Log_WarningFmt("Invalid controller type {} in {}", controller_str, entry->serial);
continue;
}
if (first)
{
entry->supported_controllers = 0;
first = false;
}
entry->supported_controllers |= (1u << static_cast<u16>(ctype.value()));
}
}
if (const ryml::ConstNodeRef compatibility = value.find_child(to_csubstr("compatibility"));
compatibility.valid() && compatibility.has_children())
{
const ryml::ConstNodeRef rating = compatibility.find_child(to_csubstr("rating"));
if (rating.valid())
{
const std::string_view rating_str = to_stringview(rating.val());
const auto iter = std::find(s_compatibility_rating_names.begin(), s_compatibility_rating_names.end(), rating_str);
if (iter != s_compatibility_rating_names.end())
{
const size_t rating_idx = static_cast<size_t>(std::distance(s_compatibility_rating_names.begin(), iter));
DebugAssert(rating_idx < static_cast<size_t>(CompatibilityRating::Count));
entry->compatibility = static_cast<CompatibilityRating>(rating_idx);
}
else
{
Log_WarningFmt("Unknown compatibility rating {} in {}", rating_str, entry->serial);
}
}
}
if (const ryml::ConstNodeRef traits = value.find_child(to_csubstr("traits")); traits.valid() && traits.has_children())
{
for (const ryml::ConstNodeRef& trait : traits.children())
{
const std::string_view trait_str = to_stringview(trait.val());
if (trait_str.empty())
{
Log_WarningFmt("Empty trait in {}", entry->serial);
continue;
}
const auto iter = std::find(s_trait_names.begin(), s_trait_names.end(), trait_str);
if (iter == s_trait_names.end())
{
Log_WarningFmt("Unknown trait {} in {}", trait_str, entry->serial);
continue;
}
const size_t trait_idx = static_cast<size_t>(std::distance(s_trait_names.begin(), iter));
DebugAssert(trait_idx < static_cast<size_t>(Trait::Count));
entry->traits[trait_idx] = true;
}
}
if (const ryml::ConstNodeRef settings = value.find_child(to_csubstr("settings"));
settings.valid() && settings.has_children())
{
entry->display_active_start_offset = GetOptionalTFromObject<s16>(settings, "displayActiveStartOffset");
entry->display_active_end_offset = GetOptionalTFromObject<s16>(settings, "displayActiveEndOffset");
entry->display_line_start_offset = GetOptionalTFromObject<s8>(settings, "displayLineStartOffset");
entry->display_line_end_offset = GetOptionalTFromObject<s8>(settings, "displayLineEndOffset");
entry->dma_max_slice_ticks = GetOptionalTFromObject<u32>(settings, "dmaMaxSliceTicks");
entry->dma_halt_ticks = GetOptionalTFromObject<u32>(settings, "dmaHaltTicks");
entry->gpu_fifo_size = GetOptionalTFromObject<u32>(settings, "gpuFIFOSize");
entry->gpu_max_run_ahead = GetOptionalTFromObject<u32>(settings, "gpuMaxRunAhead");
entry->gpu_pgxp_tolerance = GetOptionalTFromObject<float>(settings, "gpuPGXPTolerance");
entry->gpu_pgxp_depth_threshold = GetOptionalTFromObject<float>(settings, "gpuPGXPDepthThreshold");
}
if (const ryml::ConstNodeRef disc_set = value.find_child("discSet"); disc_set.valid() && disc_set.has_children())
{
GetStringFromObject(disc_set, "name", &entry->disc_set_name);
if (const ryml::ConstNodeRef set_serials = disc_set.find_child("serials");
set_serials.valid() && set_serials.has_children())
{
entry->disc_set_serials.reserve(set_serials.num_children());
for (const ryml::ConstNodeRef& serial : set_serials)
{
const std::string_view serial_str = to_stringview(serial.val());
if (serial_str.empty())
{
Log_WarningFmt("Empty disc set serial in {}", entry->serial);
continue;
}
if (std::find(entry->disc_set_serials.begin(), entry->disc_set_serials.end(), serial_str) !=
entry->disc_set_serials.end())
{
Log_WarningFmt("Duplicate serial {} in disc set serials for {}", serial_str, entry->serial);
continue;
}
entry->disc_set_serials.emplace_back(serial_str);
}
}
}
return true;
}
bool GameDatabase::ParseYamlCodes(u32 index, const ryml::ConstNodeRef& value, std::string_view serial)
{
const ryml::ConstNodeRef& codes = value.find_child(to_csubstr("codes"));
if (!codes.valid() || !codes.has_children())
{
// use serial instead
auto iter = s_code_lookup.find(serial);
if (iter != s_code_lookup.end())
{
Log_WarningFmt("Duplicate code '{}'", serial);
return false;
}
s_code_lookup.emplace(serial, index);
return true;
}
u32 added = 0;
for (const ryml::ConstNodeRef& current_code : codes)
{
const std::string_view current_code_str = to_stringview(current_code.val());
if (current_code_str.empty())
{
Log_WarningFmt("code is not a string in {}", serial);
continue;
}
auto iter = s_code_lookup.find(current_code_str);
if (iter != s_code_lookup.end())
{
Log_WarningFmt("Duplicate code '{}' in {}", current_code_str, serial);
continue;
}
s_code_lookup.emplace(current_code_str, index);
added++;
}
return (added > 0);
}
bool GameDatabase::ParseJsonEntry(Entry* entry, const rapidjson::Value& value)
{
if (!value.IsObject())
@ -982,7 +1416,7 @@ bool GameDatabase::ParseJsonEntry(Entry* entry, const rapidjson::Value& value)
for (u32 trait = 0; trait < static_cast<u32>(Trait::Count); trait++)
{
bool bvalue;
if (GetBoolFromObject(traitsobj, s_trait_names[trait], &bvalue) && bvalue)
if (GetBoolFromObject(traitsobj, s_jsontrait_names[trait], &bvalue) && bvalue)
entry->traits[trait] = bvalue;
}
@ -1082,105 +1516,84 @@ void GameDatabase::EnsureTrackHashesMapLoaded()
bool GameDatabase::LoadTrackHashes()
{
std::optional<std::string> gamedb_data(Host::ReadResourceFileToString("gamedb.json", false));
Common::Timer load_timer;
std::optional<std::string> gamedb_data(Host::ReadResourceFileToString("discdb.yaml", false));
if (!gamedb_data.has_value())
{
Log_ErrorPrintf("Failed to read game database");
Log_ErrorPrint("Failed to read game database");
return false;
}
SetRymlCallbacks();
// TODO: Parse in-place, avoid string allocations.
std::unique_ptr<rapidjson::Document> json = std::make_unique<rapidjson::Document>();
json->Parse(gamedb_data->c_str(), gamedb_data->size());
if (json->HasParseError())
{
Log_ErrorPrintf("Failed to parse game database: %s at offset %zu",
rapidjson::GetParseError_En(json->GetParseError()), json->GetErrorOffset());
return false;
}
if (!json->IsArray())
{
Log_ErrorPrintf("Document is not an array");
return false;
}
const ryml::Tree tree = ryml::parse_in_arena(c4::to_csubstr(gamedb_data.value()));
const ryml::ConstNodeRef root = tree.rootref();
s_track_hashes_map = {};
for (const rapidjson::Value& current : json->GetArray())
size_t serials = 0;
for (const ryml::ConstNodeRef& current : root.children())
{
if (!current.IsObject())
const std::string_view serial = to_stringview(current.key());
if (serial.empty() || !current.has_children())
{
Log_WarningPrintf("entry is not an object");
Log_WarningPrint("entry is not an object");
continue;
}
std::vector<std::string> codes;
if (!GetArrayOfStringsFromObject(current, "codes", &codes))
const ryml::ConstNodeRef track_data = current.find_child(to_csubstr("trackData"));
if (!track_data.valid() || !track_data.has_children())
{
Log_WarningPrintf("codes member is missing");
Log_WarningFmt("trackData is missing in {}", serial);
continue;
}
auto track_data = current.FindMember("track_data");
if (track_data == current.MemberEnd())
u32 revision = 0;
for (const ryml::ConstNodeRef& track_revisions : track_data.children())
{
Log_WarningPrintf("track_data member is missing");
continue;
}
if (!track_data->value.IsArray())
{
Log_WarningPrintf("track_data is not an array");
continue;
}
uint32_t revision = 0;
for (const rapidjson::Value& track_revisions : track_data->value.GetArray())
{
if (!track_revisions.IsObject())
const ryml::ConstNodeRef tracks = track_revisions.find_child(to_csubstr("tracks"));
if (!tracks.valid() || !tracks.has_children())
{
Log_WarningPrintf("track_data is not an array of object");
Log_WarningFmt("tracks member is missing in {}", serial);
continue;
}
auto tracks = track_revisions.FindMember("tracks");
if (tracks == track_revisions.MemberEnd())
{
Log_WarningPrintf("tracks member is missing");
continue;
}
std::string revision_string;
GetStringFromObject(track_revisions, "version", &revision_string);
if (!tracks->value.IsArray())
for (const ryml::ConstNodeRef& track : tracks)
{
Log_WarningPrintf("tracks is not an array");
continue;
}
std::string revisionString;
GetStringFromObject(track_revisions, "version", &revisionString);
for (const rapidjson::Value& track : tracks->value.GetArray())
{
auto md5_field = track.FindMember("md5");
if (md5_field == track.MemberEnd() || !md5_field->value.IsString())
const ryml::ConstNodeRef md5 = track.find_child("md5");
std::string_view md5_str;
if (!md5.valid() || (md5_str = to_stringview(md5.val())).empty())
{
Log_WarningFmt("md5 is missing in track in {}", serial);
continue;
}
auto md5 = CDImageHasher::HashFromString(
std::string_view(md5_field->value.GetString(), md5_field->value.GetStringLength()));
if (md5)
const std::optional<CDImageHasher::Hash> md5o = CDImageHasher::HashFromString(md5_str);
if (md5o.has_value())
{
s_track_hashes_map.emplace(std::piecewise_construct, std::forward_as_tuple(md5.value()),
std::forward_as_tuple(codes, revisionString, revision));
s_track_hashes_map.emplace(std::piecewise_construct, std::forward_as_tuple(md5o.value()),
std::forward_as_tuple(std::string(serial), revision_string, revision));
}
else
{
Log_WarningFmt("invalid md5 in {}", serial);
}
}
revision++;
}
serials++;
}
return true;
ryml::reset_callbacks();
Log_InfoFmt("Loaded {} track hashes from {} serials in {:.0f}ms.", s_track_hashes_map.size(), serials,
load_timer.GetTimeMilliseconds());
return !s_track_hashes_map.empty();
}
const GameDatabase::TrackHashesMap& GameDatabase::GetTrackHashesMap()

View File

@ -98,16 +98,14 @@ const Entry* GetEntryForSerial(const std::string_view& serial);
std::string GetSerialForDisc(CDImage* image);
std::string GetSerialForPath(const char* path);
const char* GetTraitName(Trait trait);
const char* GetCompatibilityRatingName(CompatibilityRating rating);
const char* GetCompatibilityRatingDisplayName(CompatibilityRating rating);
/// Map of track hashes for image verification
struct TrackData
{
TrackData(std::vector<std::string> codes, std::string revisionString, uint32_t revision)
: codes(std::move(codes)), revisionString(revisionString), revision(revision)
TrackData(std::string serial_, std::string revision_str_, uint32_t revision_)
: serial(std::move(serial_)), revision_str(std::move(revision_str_)), revision(revision_)
{
}
@ -115,12 +113,12 @@ struct TrackData
{
// 'revisionString' is deliberately ignored in comparisons as it's redundant with comparing 'revision'! Do not
// change!
return left.codes == right.codes && left.revision == right.revision;
return left.serial == right.serial && left.revision == right.revision;
}
std::vector<std::string> codes;
std::string revisionString;
uint32_t revision;
std::string serial;
std::string revision_str;
u32 revision;
};
using TrackHashesMap = std::multimap<CDImageHasher::Hash, TrackData>;

View File

@ -1,4 +1,4 @@
// SPDX-FileCopyrightText: 2019-2022 Connor McLaughlin <stenzek@gmail.com>
// SPDX-FileCopyrightText: 2019-2024 Connor McLaughlin <stenzek@gmail.com>
// SPDX-License-Identifier: (GPL-3.0 OR CC-BY-NC-ND-4.0)
#include "gamesummarywidget.h"
@ -13,7 +13,6 @@
#include "fmt/format.h"
#include <QtConcurrent/QtConcurrent>
#include <QtCore/QFuture>
#include <QtWidgets/QMessageBox>
@ -219,13 +218,6 @@ void GameSummaryWidget::onComputeHashClicked()
return;
}
#ifndef _DEBUGFAST
// Kick off hash preparation asynchronously, as building the map of results may take a while
// This breaks for DebugFast because of the iterator debug level mismatch.
QFuture<const GameDatabase::TrackHashesMap*> result =
QtConcurrent::run([]() { return &GameDatabase::GetTrackHashesMap(); });
#endif
QtModalProgressCallback progress_callback(this);
progress_callback.SetProgressRange(image->GetTrackCount());
@ -259,6 +251,7 @@ void GameSummaryWidget::onComputeHashClicked()
if (calculate_hash_success)
{
std::string found_revision;
std::string found_serial;
m_redump_search_keyword = CDImageHasher::HashToString(track_hashes.front());
progress_callback.SetStatusText("Verifying hashes...");
@ -270,11 +263,7 @@ void GameSummaryWidget::onComputeHashClicked()
// 2. For each data track match, try to match all audio tracks
// If all match, assume this revision. Else, try other revisions,
// and accept the one with the most matches.
#ifndef _DEBUGFAST
const GameDatabase::TrackHashesMap& hashes_map = *result.result();
#else
const GameDatabase::TrackHashesMap& hashes_map = GameDatabase::GetTrackHashesMap();
#endif
auto data_track_matches = hashes_map.equal_range(track_hashes[0]);
if (data_track_matches.first != data_track_matches.second)
@ -317,13 +306,24 @@ void GameSummaryWidget::onComputeHashClicked()
}
}
found_revision = best_data_match->second.revisionString;
found_revision = best_data_match->second.revision_str;
found_serial = best_data_match->second.serial;
}
QString text;
if (!found_revision.empty())
text = tr("Revision: %1").arg(found_revision.empty() ? tr("N/A") : QString::fromStdString(found_revision));
if (found_serial != m_ui.serial->text().toStdString())
{
m_ui.revision->setText(
tr("Revision: %1").arg(found_revision.empty() ? tr("N/A") : QString::fromStdString(found_revision)));
text =
tr("Serial Mismatch: %1 vs %2%3").arg(QString::fromStdString(found_serial)).arg(m_ui.serial->text()).arg(text);
}
if (!text.isEmpty())
{
m_ui.revision->setText(text);
m_ui.revision->setVisible(true);
}
}