Dolphin's JITs have a minor terminology problem: The term "fastmem" can
refer to either the system of switching between a fast path and a slow
path using backpatching, or to the fast path itself. To hopefully make
things clearer, I'm adding some new terms, defining the old and new
terms as follows:
Fastmem: The system of switching from a fast path to a slow path by
backpatching when an invalid memory access occurs.
Fast access: A code path that accesses guest memory without calling C++
code.
Slow access: A code path that accesses guest memory by calling C++ code.
With this, situations where multiple arguments need to be moved
from multiple registers become easy to handle, and we also get
compile-time checking that the number of arguments is correct.
At the end of each frame automatically update the Current Value for
visible table rows in the selected and visible CheatSearchWidget (if
any). Also update all Current Values in all CheatSearchWidgets when the
State changes to Paused.
Only updating visible table rows serves to minimize the performance cost
of this feature. If the user scrolls to an un-updated cell it will
promptly be updated by either the next VIEndFieldEvent or the State
transitioning to Paused.
If dcache is enabled when the game starts, initializing the fastmem
arena is still useful in case the user changes the dcache setting.
And initializing it doesn't really cost anything.
Preparation for the next commit.
JitArm64 has been conflating these two flags. Most of the stuff that's
been guarded by fastmem_arena checks in fact requires fastmem.
When we have fastmem_arena without fastmem, it would be possible to do
things a bit more efficiently than what this commit does, but it's
non-trivial and therefore I will leave it out of this PR. With this
commit, we effectively have the same behavior as before this PR - plus
the added ability to toggle fastmem with a cache clear.
This is needed so that the checks added in the previous commit will be
reevaluated if the value of m_enable_dcache changes.
JitArm64 was already recompiling its asm routines on cache clear by
necessity. It doesn't have the same setup as Jit64 where the asm
routines are in a separate region, so clearing the JitArm64 cache
results in the asm routines being cleared too.
Some code paths in EmuCodeBlock.cpp that were checking fastmem_arena
should really also be checking m_enable_dcache.
Because JitArm64 centralizes more or less all memory access to the
EmitBackpatchRoutine function and because that function already
contained a check, JitArm64 works fine without the additional checks
added by this commit. Regardless, I added the checks to MMU.cpp instead
of EmuCodeBlock.cpp where applicable so they would be available to
JitArm64. Maybe one day JitArm64 will need them if its code gets
restructured.
The table only needs to be recreated when the displayed addresses might
change. If we're just refreshing the current values then update those
table cells and leave the rest of the table alone.
The code previously did this indirectly via `std::map<double, int>`, the key being the timestamp, which required a questionable workaround for the case where multiple states have the same timestamp. By having a particular combination of timestamps in the on-disk savestates, you could cause this workaround to infinitely loop, locking up Dolphin. This avoids this completely by refactoring the logic and just using `std::vector` instead.
Since ccf92a3e56, recording fifologs multiple times after launching dolphin caused all initial state to not be saved (the initial contents of bpmem, cpmem, etc were all zeroed out). For some games, this was not noticeable, as most registers were set each frame, but for others, this resulted in completely broken fifologs. (Note that recording fifologs also required 05181f6b88 and 9e0755a598 to be cherry-picked due to other, since fixed, regressions.)
This was because previously, `Renderer::CheckFifoRecording` was called every frame, but ccf92a3e56 changed it into a callback (`m_end_of_frame_event`) that was removed when recording ended. Thus, before, `OpcodeDecoder::g_record_fifo_data = IsRecording()` was called when `IsRecording()` returned false, but after that commit `g_record_fifo_data` never got changed back to false, so the check for `was_recording` only ever passed on the first fifolog recorded (even after stopping and starting a game).
There may still be another issue lurking, as I'm not sure if all broken fifologs were caused by recording multiple fifologs (for instance, on https://bugs.dolphin-emu.org/issues/13377, only one fifolog was initially uploaded, but it was affected by an issue with the same symptoms as this).
Instructions referencing registers r8-r15 take an additional byte to
encode. `reg_downcount` may be assigned to one of these registers, so it
is a small size win to store the downcount value in `RSCRATCH` first.
Before:
33 D2 xor edx,edx
44 8B 6D 64 mov r13d,dword ptr [rbp+64h]
45 85 ED test r13d,r13d
7E 30 jle 0000023546B43F6D
44 8B B5 D4 02 00 00 mov r14d,dword ptr [rbp+2D4h]
41 8B C5 mov eax,r13d
BF 07 00 00 00 mov edi,7
F7 F7 div eax,edi
After:
33 D2 xor edx,edx
8B 45 64 mov eax,dword ptr [rbp+64h]
85 C0 test eax,eax
7E 30 jle 000001AFBBAE359D
44 8B B5 D4 02 00 00 mov r14d,dword ptr [rbp+2D4h]
44 8B E8 mov r13d,eax
BF 07 00 00 00 mov edi,7
F7 F7 div eax,edi
This value is used in a multiplication. The result of this
multiplication is then subtracted from m_base. By negating m_dec, we are
free to use an addition instead.
On x64, this saves an instruction.
Split the "welcome" messages letting players know achievements are active into a separate method that gets called (currently) after a number of frames to ensure that the emulator has started properly and has somewhere to display the messages.
A new tab is added to the Achievements dialog to chart out the leaderboards in a table. Each row of the table contains the leaderboard information and up to four relevant entries, varying based on how many entries are in the leaderboard, whether or not the player has a submitted score, and where in the leaderboard the player's score is.
FetchBoardInfo is called (via the work queue asynchronously) on a leaderboard every time it is activated or submitted to. It makes two calls to the RetroAchievements API for fetching leaderboard info, one that requests the top four entries in the leaderboard and another that requests the player's entry, the two entries above the player and the two entries below. All of these are inserted into a single map (resolving any overlaps) so the result can be exposed to the UI.
The leaderboard map created here contains information useful to displaying leaderboard stats in the Achievement dialog, including each leaderboard's name and description and a partial list of entries for display. The entire map is exposed to the UI in a single call for simplicity.
Both of these functions access `m_game_data` and don't lock themselves, so they must be called in a way that guarantees that `m_game_data` is not modified during the call.
Scope issue in the event callback from `rc_runtime_do_frame()`. The pointer points to a variable on the stack from inside `rc_runtime_do_frame()`, so that's a race condition between the thread calling `rc_runtime_do_frame()` and the event queue thread.
Added handling to Achievement Progress Events, which are generated when an achievement with a Measured field updates in value. For example, an achievement for collecting 120 stars will throw this event when the player collects each star, and with this handling, the player will get a message on screen informing them of this progress. This message will only appear if the newly added RA_PROGRESS_ENABLED setting is true.
A deep-copy method CopyReader has been added to BlobReader (virtual) and all of its subclasses (override). This should create a second BlobReader to open the same set of data but with an independent read pointer so that it doesn't interfere with any reads done on the original Reader.
As part of this, IOFile has added code to create a deep copy IOFile pointer onto the same file, with code based on the platform in question to find the file ID from the file pointer and open a new one. There has also been a small piece added to FileInfo to enable a deep copy, but its only subclass at this time already had a copy constructor so this was relatively minor.
The achievement badges will now have a blue or gold border to identify whether they have been unlocked in softcore or hardcore mode. Similarly, the game badge will have a blue border if all achievements have been unlocked in either mode or a gold border if all achievements have been unlocked in hardcore mode.
Provided the badges are turned on in the settings, each achievement will have a badge next to it on the progress tab. There are different badges for locked and unlocked (usually locked is grayscale while unlocked is in color but not necessarily) and the badge chosen depends on the player's current unlock and hardcore status.
Provided badges are turned on, if there's a player logged in their RetroAchievements icon will appear next to their player info in the header of the Achievements dialog. If they're playing a game, so will the icon for the game. Also performed some refactoring and reorganizing to the header as a whole so that it looks consistent whether a game is running or not.
FetchBadges fetches all available badges (player, game, achievements) and stores them in AchievementManager's memory. New fields and accessors have been added as necessary. Badges for individual achievements are stored in the UnlockStatus map. The method is public so that the settings dialog can call it if badges are turned on after a game is started. Badges are deleted at game close and logout.
When evaluating whether jo.fastmem should be set to true, we check the
value of jo.fastmem_arena. However, due to a change made in 28e8117b90,
jo.fastmem_arena wasn't set until after the first time we set
jo.fastmem, so jo.fastmem would end up always being false until the next
time RefreshConfig was called.
Fixes https://bugs.dolphin-emu.org/issues/13364.
Image requests from RetroAchievements have a slightly different format from other requests, on account of being GET calls that return strings of image data, so this is a separate function that makes such calls. To handle this, I create a BadgeStatus object that ties each badge to a boolean flag for whether it is loaded and thus usable.
And in order to avoid a double dereference in the dispatcher, directly store
the normalEntry in the map.
The index to the block map becomes ((((DR<<1) | IR) << 30) | (address >> 2)).
This has been chosen since the msr bits change less often than the address,
thus we keep nearby entries together.
Also do not call the C dispatcher in case the assembly dispatcher didn't
find a block, since it wouldn't find a block either due to the 1:1 mapping,
except when falling back to the non shm segment lookup table.
We had two nearly identical copies of the gather pipe exception checking
code right next to each other, with the only difference being the
register used. Let's unify them, like in Jit64.
This bug has been lurking in the code ever since I added the discard
functionality. It doesn't seem to be triggered all that often,
and on top of that the emitted code only runs conditionally, so I'm not
sure if people have been affected by this bug in practice or not.
We need to check for the address of the *previous* instruction, because
checking fifoWriteAddresses happens not at the end of the instruction
that triggered it but at the start of the next instruction.
This refactors the Rich Presence generation to store to a member field that can be exposed to the UI to display the Rich Presence in the achievement header. It still updates at its original rate of once per two minutes, but calls an update on the dialog when that ticks.
Moved AchievementManager Init further down in the MainWindow constructor; its original position was because it had an impact on the contents of the menu bar, and this is no longer the case.
Whenever JitBaseBlockCache::Clear() got called, it threw away the memory mapping for the fast block map and created a new one. This new mapping typically got mapped at the same address at the old one, but this is not guaranteed. The pointer to the mapping gets embedded in the generated dispatcher code in Jit64AsmRoutineManager::Generate(), which is only called once on game boot, so if the new mapping ended up at a different address than the old one, the pointer in the ASM pointed at garbage, leading to a crash.
This fixes the issue by guaranteeing that the new mapping is mapped at the same address.
While both fastmem and the BLR optimization depend on fault handling,
the BLR optimization doesn't depend on fastmem, and there are cases
where you might want the BLR optimization but not fastmem. For me
personally, it's useful when I try to use a debugger on Android and have
to disable fastmem so I don't get SIGSEGVs all the time, but it would be
especially useful for iOS users.
When we execute a JIT block, we have to make sure that both the PC and
the DR/IR bits of MSR are the same as they were when the block was
compiled. When jumping to a block from the dispatcher, this is done in
the way you would expect: By checking the PC and the relevant MSR bits.
However, when returning to a block using the BLR optimization, we only
check the PC. Checking the MSR bits is done by instead resetting the
stack when the MSR changes, making PC checks afterwards fail.
Except... We were only resetting the stack on rfi instructions. There
are actually many more ways for the MSR to change, and we weren't
covering those at all. I looked into resetting the stack on all of them,
but it would be pretty cumbersome both in terms of writing the code and
in terms of how often at runtime we'd have to reset the stack, so I
think the better option would be to check the MSR bits along with the
PC. That's what this commit implements.
Because CPU thread config changed callbacks are no longer instant,
g_Config.iEFBScale doesn't yet contain the new value when the hotkey OSD
code tries to read it.
Should fix https://bugs.dolphin-emu.org/issues/13343.
There's no reason not to allow this now that these settings are
cleanly integrated into the new config system. (Actually, maybe
we could even have done this before the previous commit...)
This fixes a problem where changing the JIT debug settings on
Android while a game was running wouldn't cause the changed settings
to apply to code blocks that already had been compiled.
Resolve warning caused by using values from two different enums in a
conditional expression which was deprecated in c++20.
The warning in question is clang -Wdeprecated-anon-enum-enum-conversion
and gcc -Wenum-compare.
We had one implementation of this type of data structure in Arm64Emitter
and one in VideoCommon. This moves the Arm64Emitter implementation to
its own file and adds begin and end functions to it, so that VideoCommon
can use it.
You may notice that the license header for the new file is CC0. I wrote
the Arm64Emitter implementation of SmallVector, so this should be no
problem.
Modify PPCSTATE_OFF and PPCSTATE_OFF_ARRAY macros when using GCC to
avoid useless log spam. Specifically, use a consteval lambda with gcc
_Pragma statements to disable the -Winvalid-offsetof warning inside the
macros.
Each successful build (and many failing ones) on the Android buildbot
generates almost 300 cases of -Winvalid-offsetof, resulting in thousands
of lines of log spam per build. In addition to bloating the log filesize
these spurious warnings make it harder to find actual warnings.
These warnings are generated by calls to the macros PPCSTATE_OFF and
PPCSTATE_OFF_ARRAY, which in turn are used by many other macros used by
the JIT. The ultimate cause is that offsetof is only conditionally
supported on non-standard-layout types, which includes the PowerPCState
struct.
To address potential questions of whether there's a better way to handle
this:
The obvious solution would be to modify PowerPCState so that it does
have a standard layout. This is unfortunately impractical.
To have a standard layout a type can only contain other types with
standard layouts. None of the stl containers are guaranteed to have
standard layouts, and PowerPCState contains a std::tuple and std::array.
PowerPCState also contains a PowerPC::Cache and InstructionCache which
themselves contain std:arrays and std::vectors.
Furthermore InstructionCache derives from Cache, and a derived class can
only have standard layout if at most one class in its hierarchy has a
non-static data member, but both classes have such members. Making
InstructionCache have a standard layout would require duplicating all
the functionality of Cache so it no longer derived from it, as well as
replacing the stl containers. This might require having a raw pointer to
said containers, with the manual memory management that implies.
All of that would be much more disruptive than would be justified to get
rid of some warnings (however annoying they might be). This is
compounded by the fact that PowerPCState hasn't had a standard layout
for a long time, if ever, and if the PPCSTATE_OFF macros weren't working
reliably it would have become obvious a long time ago.
As to why I picked the lambda solution over other potential changes:
- Keeping the define as-is and wrapping some gcc #pragmas around it
doesn't work because the pragmas don't get included when the define is
substituted to the call site.
- Keeping the define as a non-lambda expression and using inline
_Pragma() statements would ideally be better and works fine for msvc,
but fails for GCC with "'#pragma' is not allowed here".
- Turning off -Winvalid-offsetof globally for gcc would work, but there
might be other contexts where offsetof is problematic and GCC seems to
be the only compiler warning about it.
This does require another register, but we skip having to use shifted
ADD, which takes two cycles on some CPUs, and we gain instruction-level
parallelism.
This was because the shader uniforms between the pixel and vertex shaders
were willingly left different, to avoid filling the vertex shader with unnecessary
params. Turns out all backends are fine with this except OGL.
The new behaviour is now much more consistent and well explained,
the "default" shaders are the ones that always run, and the non default
ones are the user selected ones (if any).
Turns out that we have two subsystems that want to register CPU thread
callbacks from a different thread than the CPU thread: FreeLookConfig
and VideoConfig. Both seem to happen from the host thread before the CPU
thread starts, so there's no thread safety issue. But ideally, if we
want to allow registering callbacks from threads other than the CPU
thread, we should make registering callbacks actually thread-safe. This
is an unsolved problem for the regular Config system, so I would like to
leave it outside the scope of this PR.
This fixes a problem that started happening in CoreTimingTest after the
previous commit. CPUThreadConfigCallback registers a Config callback
only once per run of the process, but CoreTimingTest calls
Config::Shutdown after each test, and Config::Shutdown was clearing all
callbacks, preventing the callback from running after that.
In theory, our config system supports calling Set from any thread. But
because we have config callbacks that call RunAsCPUThread, it's a lot
more restricted in practice. Calling Set from any thread other than the
host thread or the CPU thread is formally thread unsafe, and calling Set
on the host thread while the CPU thread is showing a panic alert causes
a deadlock. This is especially a problem because 04072f0 made the
"Ignore for this session" button in panic alerts call Set.
Because so many of our config callbacks want their code to run on the
CPU thread, I thought it would make sense to have a centralized way to
move execution to the CPU thread for config callbacks. To solve the
deadlock problem, this new way is non-blocking. This means that threads
other than the CPU thread might continue executing before the CPU thread
is informed of the new config, but I don't think there's any problem
with that.
Intends to fix https://bugs.dolphin-emu.org/issues/13108.
In std::string, you can store strings using any encoding, but in Dolphin
we have decided to use UTF-8. The problem is that if you convert between
std::string and std::filesystem::path using the built-in methods, the
standard library will make up its own assumption of what encoding you're
using in the std::string. On most OSes this is UTF-8, but on Windows
it's whatever the user's code page is.
What I believe is the C++ standard authors' intended solution to this is
to use std::u8string instead of std::string, but that's a big hassle to
move over to, because there's no convenient way to convert between
std::string and std::u8string. Instead, in Dolphin, we have added helper
functions that convert between std::string and std::filesystem::path in
the manner we want. You *always* have to use these when converting
between std::string and std::filesystem::path, otherwise we get these
kinds of encoding problems that we've been having with custom textures.
Fixes https://bugs.dolphin-emu.org/issues/13328.
The previous list had some issues. A lot of variant id's were set to 0x0000. Althought this works for some figures, on a technicallity implemented into the games, they are technically wrong and don't result in exactly the same experience as the real figures. For example, the previous small fry got a "series 1" text in the summon screen. The real small fry does not have this. I also added figure types so I can add seperate generation logic later.
The Kaos element only applies to 3 items. So, I decided to throw it under others since it's not listed as an element in the manual and you can easily search for Kaos
Android interprets char as unsigned char, so comparing with 0 triggers a
tautological-unsigned-char-zero-compare warning.
Casting c to an unsigned char and removing the comparison with 0
resolves the warning while needing one less comparison on all platforms.
Verify that DXFramebuffer's integer RTV descriptor's cpu_handle has been
allocated before using it, and if it hasn't use the non-integer RTV
descriptor instead. This fixes a Dolphin crash in Twilight Princess, and
possibly other games (Issue 13312).
As an optimization to save space in the descriptor heap, DXFramebuffer's
integer descriptor is only initialized if the given abstract texture
format has different integer and non-integer RTV formats. This
previously wasn't accounted for by GetIntRTVDescriptorArray, which could
cause DX12::Gfx::BindFramebuffer to call OMSetRenderTargets with an
invalid descriptor which would lead to a crash.
Triggering the bug was fortunately rare because integer formats are only
used when blending is disabled and logic ops are enabled. Furthermore,
the standard integer abstract format is RGBA8 which has different
integer and non-integer RTV formats, causing the integer descriptor to
be initialized and avoiding the bug.
The crash started appearing in a2702c6 because it changed the
swapchain's abstract texture format from RGBA8 to RGB10_A2. Unlike
RGBA8, RGB10_A2 has the same integer and non-integer RTV formats and so
the bug can be triggered if the other requirements are met.
Should fix https://bugs.dolphin-emu.org/issues/13178.
Jit64 doesn't need to do this because it stores PC directly into
ppcState instead of first storing it in a register.
Now block link nearcode is back to a length of three instructions.
Unfortunately, the code I'm adding to Jit.cpp ends up being a bit messy
because we need to handle the case of already being in farcode...
Jumping between linked blocks currently works as follows: First, at the
end of the first block, we check if the downcount is greater than zero.
If it is, we jump to the `normalEntry` of the block. So far so good. But
if the downcount wasn't greater than zero, we jump to the `checkedEntry`
of the block, which checks the downcount *again* and then jumps to
`do_timing` if it's less than zero (which seems like an off by one error
- Jit64 doesn't do anything like this). This second check is rather
redundant. Let's jump to `do_timing` where we previously jumped to
`checkedEntry`.
Jit64 doesn't check the downcount on block entry. See 5236dc3.
Sketch