ShuriZma
/
redream
mirror of https://github.com/inolen/redream.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

updated memory access docs

This commit is contained in:
Anthony Pesch 2016-12-23 01:31:35 -08:00
parent dfe4304e5c
commit f3fe32d6ae
8 changed files with 298 additions and 215 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
.gitignore vendored
View File

@ -1,6 +1,5 @@
.
/build
debug
Makefile
.*
build/
*.swo
*.swp
Makefile

3
docs/.gitignore vendored
View File

@ -1,3 +1,4 @@
_site/
.*
Gemfile
Gemfile.lock
_site/

4
docs/_config.yml
View File

@ -25,6 +25,10 @@ defaults:
markdown: kramdown
kramdown:
input: GFM
syntax_highlighter: rouge
exclude:
- Gemfile
- Gemfile.lock

77
docs/_docs/memory-design.md → docs/_docs/memory-access.md
View File

@ -1,15 +1,15 @@
---
title: Memory Design
title: Memory Access
---
Memory operations are the most frequently occuring type of instruction. From a quick sample of code running during Sonic Adventure gameplay, a whopping 37% of the instructions were memory operations. Due to their high frequency, this is the single most important operation to optimize.
Memory operations are the most frequently occuring type of instruction. From a quick sample of code running during Sonic Adventure gameplay, a whopping 37% of the instructions are memory operations. Due to their high frequency, this is the single most important operation to optimize.
# The beginning
In the beginning, all memory operations resulted in a function call, for example:
In the beginning, all memory access went through a function call, e.g.:
```
uint32_t load_i8(uint32_t addr) {
```c
uint32_t load_i32(uint32_t addr) {
if (addr < 0x04000000) {
return system_ram[addr];
} else if (addr < 0x08000000) {
@ -23,10 +23,10 @@ uint32_t load_i8(uint32_t addr) {
This function would be called by the compiled x64 code like so:
```
mov rax, <guest_address>
call load_i8
mov <dst_reg>, rax
```nasm
mov edi, <guest_address>
call load_i32
mov <dst_reg>, eax
```
This is simple and straight forward, but now a single guest instruction has transformed into ~10-20 host instructions: a function call, multiples moves to copy arguments / result, stack push / pop, multiple comparisons, multiple branches and a few arithmetic ops.
@ -41,7 +41,7 @@ To illustrate with code:
```c
/* memory_base is the base address in the virtual address space where there's
a 4 GB block of unreserved pages to map the shared memory object into */
a 4 GB block of unused pages to map the shared memory into */
void init_memory(void *memory_base) {
/* create the shared memory object representing the guest address space */
int handle = shm_open("/redream_memory", O_RDWR | O_CREAT | O_EXCL, S_IREAD | S_IWRITE);
@ -56,33 +56,33 @@ void init_memory(void *memory_base) {
With the entire guest address space being mapped into the host address space relative to `memory_base`, each memory lookup now looks like:
```
```nasm
mov rax, memory_base
mov <dst_reg>, [rax + <guest_address>]
```
In redream's JIT, an extra register is reserved to hold `memory_base`, so the initial mov is avoided:
```
```nasm
mov <dst_reg>, [<mem_base_reg> + <guest_address>]
```
# MMIO access
# Handling MMIO accesses
The above example glosses over the part where this design doesn't at all work for MMIO accesses, which still need to call a higher-level callback function on each access.
The above example glosses over the part where this solution doesn't handle MMIO accesses, which still need to call a higher-level callback function on each access.
Initially, it may seem that the only option is to conditionally branch at each memory access:
```
```nasm
cmp <guest_addr>, 0x08000000
jge .slowmem
# fastmem path when guest_addr < 0x08000000
# fast path when guest_addr < 0x08000000
mov <dst_reg>, [<mem_base_reg> + <guest_address>]
jmp .end
# slowmem path when guest_addr >= 0x08000000
# slow path when guest_addr >= 0x08000000
.slowmem:
sub <guest_addr>, 0x08000000
call [mmio_callbacks + <guest_addr>]
mov rdi, <guest_address>
call load_i32
.end:
```
@ -90,29 +90,29 @@ This isn't _awful_ and is much improved over the original implementation. While
Fortunately, we can have our cake and eat it too thanks to segfaults.
## Abusing segfaults
## Generating segfaults
The idea with segfaults is to disable access to the pages representing the MMIO region of the guest address space, and _always_ optimistically emit the fastmem code.
Extending on the above `init_memory` function, this would look like:
Extending on the above `init_memory` function, the code to disable access would look like:
```
/* disable access to the mmio region*/
```c
/* disable access to the mmio region */
mprotect(memory_base + 0x08000000, 0x04000000, PROT_NONE);
```
Now, when a fastmem piece of code tries to perform an MMIO access, a `SIGSEGV` signal will be raised at which point we can either:
Now, when a fastmem-optimized piece of code tries to perform an MMIO access, a segfault will be generated at which point we can either:
* "backpatch" the original code
* recompile the original code with fastmem optimizations disabled
* recompile the code with fastmem optimizations disabled
### Backpatching
This is the easier to implement of the two options, and what redream did originally. The technique involves always writing out the fastmem code with enough padding such that, inside of the signal handler, it can be overwritten with the slowmem code.
This is the easier to implement of the two options, and what redream did originally. The technique involves always writing out the fastmem code with enough padding such that, inside of the signal handler, it can be overwritten with the "slowmem" code.
All memory accesses would by default look like:
```
```nasm
mov <dst_reg>, [<mem_base_reg> + <guest_addr_reg>]
nop
nop
@ -121,22 +121,29 @@ nop
...
```
Then, when the signal handler is ran, the current pc would be extracted from the signal handler and the code would be overwritten with:
Then, when the signal handler is entered, the current PC would be extracted from the thread context and the code would be overwritten with:
```
sub <guest_addr_reg>, 0x08000000
call [mmio_callbacks + <guest_addr_reg>]
```c
mov edi, <guest_addr_reg>
call load_i32
mov <dst_reg>, eax
```
When the signal handler returns, the program would now resume at the `sub` instruction and all would be well. This approach works well, but the added `nop` instructions can add up depending on how your MMIO callbacks are invoked, which can very negatively impact performance.
When the signal handler returns, the thread will resume at the same PC that originally generated the segfault, but it will now execute the backpatched slow path. This approach works well, but the `nop` padding can add up depending on the size of the slow path code, which can very negatively impact performance.
### Recompiling
This is what redream currently does. The idea is simple: when the signal is raised, recompile the block to not use any fastmem optimizations and execute the recompiled block. However, while this sounds easy, the devil truly is in the details.
This is what redream currently does. The idea itself is simple: when the signal is raised, recompile the block to not use any fastmem optimizations and execute the recompiled block. However, while this sounds easy, the devil truly is in the details.
For starters, it's not possible to recompile the block inside of the signal handler itself due to the [limitations of what can be done inside of a signal handler](https://www.securecoding.cert.org/confluence/display/c/SIG30-C.+Call+only+asynchronous-safe+functions+within+signal+handlers).
Because of this, the actual recompilation is deferred to a later time (on the next access to the block in fact), and the MMIO access is handled somewhat-inside of the signal handler itself. I say "somewhat-inside" because, for the same reason the block itself can't be recompiled inside of the signal handler, it's not safe to directly invoke the MMIO callbacks inside of the signal handler. Instead of directly invoking the callback, the signal handler adjusts the program counter to land in a thunk that will invoke the callback when the signal handler resumes. This portion of the code is rather involved, but fairly well documented inside of [x64_backend_handle_exception](https://github.com/inolen/redream/blob/master/src/jit/backend/x64/x64_backend.cc#L406).
Because of this, the actual recompilation is deferred to a later time, and the MMIO access is handled somewhat-inside of the signal handler itself. I say "somewhat-inside" because, for the same reason the block itself can't be recompiled inside of the signal handler, it's not safe to directly invoke the MMIO callbacks inside of the signal handler. Instead of directly invoking the callback, the signal handler adjusts the program counter to land in a thunk that will invoke the callback when the thread resumes.
TODO add diagram show how this function works
This is what the control flow looks like when an MMIO segfault occurs:
![MMIO segfault]({{ site.github.url }}/docs/mmio_segfault.png)
From looking at this diagram it should be apparent that this method of servicing the MMIO request is _extremely_ slow. However, this penalty is only paid once, as the block will be recompiled with all fastmem optimizations disabled before the next run.
The trade off of all this effort is that, now the fastmem route needs no padding, providing non-MMIO guest memory access with the absolute minimum amount of overhead.

2
docs/_layouts/doc.html
View File

@ -16,7 +16,7 @@ layout: default
<hr>
<ul>
<li><a {% if page.url.end_with? '/docs/directory-structure' %} class="current"{% endif %} href="{{ site.github.url }}/docs/directory-structure">Directory structure</a></li>
<li><a {% if page.url.end_with? '/docs/memory-design' %} class="current"{% endif %} href="{{ site.github.url }}/docs/memory-design">Memory design</a></li>
<li><a {% if page.url.end_with? '/docs/memory-access' %} class="current"{% endif %} href="{{ site.github.url }}/docs/memory-access">Memory access</a></li>
<li><a {% if page.url.end_with? '/docs/cpu-design' %} class="current"{% endif %} href="{{ site.github.url }}/docs/cpu-design">CPU design</a></li>
</ul>
<hr>

174
docs/css/main.css
View File

@ -1,174 +0,0 @@
body {
margin: 0;
color: #464646;
font-family: 'Open Sans', sans-serif;
}
a {
text-decoration: none;
color: #3a76c3;
}
a:hover {
color: #202020;
}
h1, h2, h3 {
color: #202020;
font-family: 'Roboto' sans-serif;
line-height: 1em;
}
h1 a, h2 a, h3 a {
color: #202020;
}
code {
padding: 3px 4px;
color: #d14;
background-color: #f7f7f9;
border: 1px solid #e1e1e8;
}
pre code {
padding: 0;
color: inherit;
background-color: transparent;
border: 0;
}
.container {
width: 1100px;
margin-left: auto;
margin-right: auto;
}
#header {
background-color: #f8f8f8;
border-bottom: solid 1px #ededed;
margin-bottom: 30px;
}
#header .container {
display: table;
}
#banner {
float: left;
padding: 10px 0 10px 10px;
}
#navbar {
float: right;
color: #202020;
font-family: 'Roboto', sans-serif;
font-weight: bold;
font-size: 1.4em;
text-align: center;
text-transform: uppercase;
padding: 32px 10px 32px 0;
}
#navbar a {
margin: 0 10px;
color: #202020;
}
#navbar a:hover {
color: #3a76c3;
}
#navbar .slack {
vertical-align: middle;
}
#content {
position: relative;
}
/*
* articles
*/
#articles {
list-style: none;
margin: 0;
padding: 0;
}
#articles li {
margin: 0 0 1em;
padding-bottom: 1em;
border-bottom: solid 2px #e6e6e6;
}
#articles li:last-child {
border-bottom: 0;
}
.article header h1,
.article header h2 {
margin-top: 0;
margin-bottom: .4em;
}
.article time {
color: #b9b9b9;
font-family: 'Roboto', sans-serif;
}
.article pre {
font-size: .8em;
}
.article img {
max-width: 100%;
}
.article pre {
background-color: #f8f8f8;
border: 1px solid #ddd;
font-size: 13px;
line-height: 19px;
overflow: auto;
padding: 6px 10px;
border-radius: 3px;
}
/*
* docs
*/
#docs {
position: absolute;
top: 0;
left: 0;
margin: 0;
padding: 6px 10px;
width: 200px;
overflow: hidden;
list-style: none;
background-color: #f8f8f8;
border: 1px solid #ddd;
border-radius: 3px;
}
#docs hr {
border: 1px solid #ddd;
border-bottom: 0;
}
#docs ul {
list-style: none;
margin: 0 0 0 20px;
padding: 0;
}
#docs .current {
color: #202020;
}
.doc {
margin-left: 250px;
}

246
docs/css/main.scss Normal file
View File

@ -0,0 +1,246 @@
---
---
body {
margin: 0;
color: #464646;
font-family: 'Open Sans', sans-serif;
}
a {
text-decoration: none;
color: #3a76c3;
}
a:hover {
color: #202020;
}
h1, h2, h3 {
color: #202020;
font-family: 'Roboto' sans-serif;
line-height: 1em;
}
h1 a, h2 a, h3 a {
color: #202020;
}
code {
padding: 3px 4px;
color: #d14;
background-color: #f7f7f9;
border: 1px solid #e1e1e8;
}
pre code {
padding: 0;
color: inherit;
background-color: transparent;
border: 0;
}
.container {
width: 1100px;
margin-left: auto;
margin-right: auto;
}
#header {
background-color: #f8f8f8;
border-bottom: solid 1px #ededed;
margin-bottom: 30px;
}
#header .container {
display: table;
}
#banner {
float: left;
padding: 10px 0 10px 10px;
}
#navbar {
float: right;
color: #202020;
font-family: 'Roboto', sans-serif;
font-weight: bold;
font-size: 1.4em;
text-align: center;
text-transform: uppercase;
padding: 32px 10px 32px 0;
}
#navbar a {
margin: 0 10px;
color: #202020;
}
#navbar a:hover {
color: #3a76c3;
}
#navbar .slack {
vertical-align: middle;
}
#content {
position: relative;
}
/*
* articles
*/
#articles {
list-style: none;
margin: 0;
padding: 0;
}
#articles li {
margin: 0 0 1em;
padding-bottom: 1em;
border-bottom: solid 2px #e6e6e6;
}
#articles li:last-child {
border-bottom: 0;
}
.article header h1,
.article header h2 {
margin-top: 0;
margin-bottom: .4em;
}
.article time {
color: #b9b9b9;
font-family: 'Roboto', sans-serif;
}
.article pre {
font-size: .8em;
}
.article img {
max-width: 100%;
}
.article pre {
background-color: #f8f8f8;
border: 1px solid #ddd;
font-size: 13px;
line-height: 19px;
overflow: auto;
padding: 6px 10px;
border-radius: 3px;
}
/*
* docs
*/
#docs {
position: absolute;
top: 0;
left: 0;
margin: 0;
padding: 6px 10px;
width: 200px;
overflow: hidden;
list-style: none;
background-color: #f8f8f8;
border: 1px solid #ddd;
border-radius: 3px;
}
#docs hr {
border: 1px solid #ddd;
border-bottom: 0;
}
#docs ul {
list-style: none;
margin: 0 0 0 20px;
padding: 0;
}
#docs .current {
color: #202020;
}
.doc {
margin-left: 250px;
}
/**
* Syntax highlighting styles
*/
.highlight {
background: #fff;
.highlighter-rouge & {
background: #eef;
}
.c { color: #998; font-style: italic } // Comment
.err { color: #a61717; background-color: #e3d2d2 } // Error
.k { font-weight: bold } // Keyword
.o { font-weight: bold } // Operator
.cm { color: #998; font-style: italic } // Comment.Multiline
.cp { color: #999; font-weight: bold } // Comment.Preproc
.c1 { color: #998; font-style: italic } // Comment.Single
.cs { color: #999; font-weight: bold; font-style: italic } // Comment.Special
.gd { color: #000; background-color: #fdd } // Generic.Deleted
.gd .x { color: #000; background-color: #faa } // Generic.Deleted.Specific
.ge { font-style: italic } // Generic.Emph
.gr { color: #a00 } // Generic.Error
.gh { color: #999 } // Generic.Heading
.gi { color: #000; background-color: #dfd } // Generic.Inserted
.gi .x { color: #000; background-color: #afa } // Generic.Inserted.Specific
.go { color: #888 } // Generic.Output
.gp { color: #555 } // Generic.Prompt
.gs { font-weight: bold } // Generic.Strong
.gu { color: #aaa } // Generic.Subheading
.gt { color: #a00 } // Generic.Traceback
.kc { font-weight: bold } // Keyword.Constant
.kd { font-weight: bold } // Keyword.Declaration
.kp { font-weight: bold } // Keyword.Pseudo
.kr { font-weight: bold } // Keyword.Reserved
.kt { color: #458; font-weight: bold } // Keyword.Type
.m { color: #099 } // Literal.Number
.s { color: #d14 } // Literal.String
.na { color: #008080 } // Name.Attribute
.nb { color: #0086B3 } // Name.Builtin
.nc { color: #458; font-weight: bold } // Name.Class
.no { color: #008080 } // Name.Constant
.ni { color: #800080 } // Name.Entity
.ne { color: #900; font-weight: bold } // Name.Exception
.nf { color: #900; font-weight: bold } // Name.Function
.nn { color: #555 } // Name.Namespace
.nt { color: #000080 } // Name.Tag
.nv { color: #008080 } // Name.Variable
.ow { font-weight: bold } // Operator.Word
.w { color: #bbb } // Text.Whitespace
.mf { color: #099 } // Literal.Number.Float
.mh { color: #099 } // Literal.Number.Hex
.mi { color: #099 } // Literal.Number.Integer
.mo { color: #099 } // Literal.Number.Oct
.sb { color: #d14 } // Literal.String.Backtick
.sc { color: #d14 } // Literal.String.Char
.sd { color: #d14 } // Literal.String.Doc
.s2 { color: #d14 } // Literal.String.Double
.se { color: #d14 } // Literal.String.Escape
.sh { color: #d14 } // Literal.String.Heredoc
.si { color: #d14 } // Literal.String.Interpol
.sx { color: #d14 } // Literal.String.Other
.sr { color: #009926 } // Literal.String.Regex
.s1 { color: #d14 } // Literal.String.Single
.ss { color: #990073 } // Literal.String.Symbol
.bp { color: #999 } // Name.Builtin.Pseudo
.vc { color: #008080 } // Name.Variable.Class
.vg { color: #008080 } // Name.Variable.Global
.vi { color: #008080 } // Name.Variable.Instance
.il { color: #099 } // Literal.Number.Integer.Long
}

BIN
docs/docs/mmio_segfault.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 88 KiB