ShuriZma
/
BizHawk
mirror of https://github.com/TASEmulators/BizHawk.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Some more re: allocation limits

James Groom 2024-08-30 23:38:42 +10:00
parent fc4e88564e
commit 63fc552f04
1 changed files with 15 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
C#-and-.NET-docs-supplement.md

@ -5,27 +5,31 @@ Contribute to [the official docs](https://github.com/dotnet/docs) if possible.
## Allocation limits
Under .NET 8:
- The largest 1D byte array is `new byte[Array.MaxLength]`, `Array.MaxLength` being hardcoded to `0x7FFFFFC7` or just under 2 GiB. (Would probably want to alloc in 1 GiB chunks to have nice code, maybe even smaller to keep the GC happy.)
- The largest 1D struct array is `new T[Array.MaxLength]`. So its size is... unbounded since structs don't have a size limit apart from the stack size (and `[InlineArray]`'s 1 MiB cap). (TODO check there isn't an undocumented cap)
Under .NET 8 (all of this is untested, just based on the docs and .NET issue tracker):
- The largest 1D byte array is `new byte[Array.MaxLength]`, `Array.MaxLength` being [hardcoded to `0x7FFF_FFC7`](https://github.com/dotnet/runtime/blob/5535e31a712343a63f5d7d796cd874e563e5ac14/src/libraries/System.Private.CoreLib/src/System/Array.cs#L2135-L2137) or just under 2 GiB. (Would probably want to alloc in 1 GiB chunks to have nice code, maybe even smaller to keep the GC happy.)
- The largest 1D struct array is `new T[Array.MaxLength]`. So its size is... unbounded since structs don't have a size limit apart from the stack size (and `[InlineArray]`'s poorly-documented 1 MiB cap). (TODO check there isn't an undocumented cap on struct size)
- The largest *n*-D byte array is `UNK()` (`LongLength` is `0xUNK` or UNK GiB).
- If a call would allocate beyond those limits (or the process' or machine's limits), an `OutOfMemoryException` will be thrown, which can be caught and handled [but definitely shouldn't be](https://learn.microsoft.com/en-us/dotnet/api/system.outofmemoryexception?view=net-8.0#remarks).
- All of that is on the managed heap. The default/global stack is OS-specific and [cannot be changed](https://github.com/dotnet/runtime/issues/107183): On Windows, it's 1.5 MiB [due to an oversight](https://github.com/dotnet/runtime/issues/96347#issuecomment-1871528297), and on Linux, it's inherited from the OS, [typically 8 MiB](https://github.com/dotnet/runtime/issues/33622#issuecomment-599462300).
- The stack size for threads can be [chosen at runtime](https://learn.microsoft.com/en-us/dotnet/api/system.threading.thread.-ctor?view=net-8.0#system-threading-thread-ctor(system-threading-threadstart-system-int32)), though as with all of this you may have an XY problem and should reconsider.
- The stack size for managed threads can be [specified on init](https://learn.microsoft.com/en-us/dotnet/api/system.threading.thread.-ctor?view=net-8.0#system-threading-thread-ctor(system-threading-threadstart-system-int32)), though as with all of this you may have an XY problem and should reconsider.
- If a call would allocate beyond the stack size (or a call chain grows out of control), a `StackOverflowException` will be thrown, which [cannot be caught](https://learn.microsoft.com/en-us/dotnet/api/system.stackoverflowexception?view=net-8.0#remarks).
Under Mono (x64 unless specified):
- The largest 1D byte array is `new byte[int.MaxValue]` (just under 2 GiB), as there is no way to instantiate an array with a larger size (`Array.CreateInstance(typeof(T), 0x80000000L)` will throw `ArgumentOutOfRangeException: Arrays larger than 2GB are not supported.`, which I suppose is technically incorrect).
Under Mono (x64 unless specified and without `gcAllowVeryLargeObjects`, TODO see if Mono respects that):
- The largest 1D byte array is `new byte[int.MaxValue]` (just under 2 GiB), as there is no way to instantiate an array with a larger size (`Array.CreateInstance(typeof(T), 0x8000_0000L)` will throw `ArgumentOutOfRangeException: Arrays larger than 2GB are not supported.`, which I suppose is technically incorrect).
- Similarly, the largest 1D struct array is `new T[int.MaxValue]`. So its size is... unbounded since structs on the heap don't have a size limit.
- The largest *n*-D byte array is `new byte[2, 0x7FFFFFE6]` (`LongLength` is `0xFFFFFFCC` or just under 4 GiB). Allocating a single byte more gives an OoME. Multidimensional arrays of other structs appear to also be limited to `0xFFFFFFCC` octets.
- If a call would allocate beyond those limits (or the process' or machine's limits), the relevant builtin method will throw an `OutOfMemoryException`, which can be caught and handled but definitely shouldn't be.
- The largest *n*-D byte array is `new byte[2, 0x7FFF_FFE6]` (`LongLength` is `0xFFFF_FFCC` or just under 4 GiB). Allocating a single byte more gives an OoME. Multidimensional arrays of other structs appear to also be limited to `0xFFFF_FFCC` octets.
- If a call would allocate beyond those limits (or the process' or machine's limits), the relevant builtin method will throw an `OutOfMemoryException`, which can be caught and handled [but definitely shouldn't be](https://learn.microsoft.com/en-us/dotnet/api/system.outofmemoryexception?view=netframework-4.8#remarks).
- All of that is on the managed heap. The default/global stack is UNK MiB large and can be changed [externally with `ulimit`](https://github.com/dotnet/runtime/issues/96347#issuecomment-1981511546) (a builtin in most POSIX shells, including BASH), but not with Roslyn. If a `stackalloc` call would allocate beyond those limits (with `stackalloc`)
- The stack size for managed threads can be chosen at runtime, though as with all of this you may have an XY problem and should reconsider. The max. is [1 MiB / 2 MiB (32-bit/64-bit hosts)](https://stackoverflow.com/a/19909421), and there is a [hack to bypass the limit](https://stackoverflow.com/a/48003390) (untested).
- The stack size for managed threads can be [specified on init](https://learn.microsoft.com/en-us/dotnet/api/system.threading.thread.-ctor?view=netframework-4.8#system-threading-thread-ctor(system-threading-threadstart-system-int32)), though as with all of this you may have an XY problem and should reconsider. The max. is [1 MiB / 2 MiB (32-bit/64-bit hosts)](https://stackoverflow.com/a/19909421), and there is a [hack to bypass the limit](https://stackoverflow.com/a/48003390) (untested).
- If a call would allocate beyond the stack size (or a call chain grows out of control), a `StackOverflowException` will be thrown, which [cannot be caught](https://learn.microsoft.com/en-us/dotnet/api/system.stackoverflowexception?view=netframework-4.8#remarks).
Under .NET Framework on Windows (x64 unless specified):
- Untested, but I'm guessing it's probably the same as Mono so long as the [`gcAllowVeryLargeObjects`](https://learn.microsoft.com/en-us/dotnet/framework/configure-apps/file-schema/runtime/gcallowverylargeobjects-element) is enabled in the runtime config, going by those docs and [this blog post](https://www.centerspace.net/large-matrices-and-vectors).
- Or maybe not; see [this SO answer](https://stackoverflow.com/a/34413257) and the comment below it.
- The default/global stack is 1 MiB, specified in a PE header. This can be changed [externally with `EDITBIN`](https://stackoverflow.com/a/54584830), but not with Roslyn nor at runtime in any way.
- The stack size for threads [defaults to 1 MiB / 4 MiB (x86/x64)](https://stackoverflow.com/a/5507910) (TODO surely there's an official source) can be [chosen at runtime](https://learn.microsoft.com/en-us/dotnet/api/system.threading.thread.-ctor?view=netframework-4.8#system-threading-thread-ctor(system-threading-threadstart-system-int32)), though as with all of this you may have an XY problem and should reconsider. The max. is UNK MiB (default AppDomain; 1 MiB if untrusted) and the min. [seems to be OS-dependent](https://learn.microsoft.com/en-us/dotnet/api/system.threading.thread.-ctor?view=netframework-4.8#system-threading-thread-ctor(system-threading-threadstart-system-int32)) but was [~256 KiB on Vista](https://techcommunity.microsoft.com/t5/windows-blog-archive/pushing-the-limits-of-windows-processes-and-threads/ba-p/723824) and presumably hasn't been changed since.
- It should be no surprise that all these limits were inherited from native Win32. Thankfully modern .NET is deviating from that where necessary.
- The stack size for managed threads [defaults to 1 MiB / 4 MiB (x86/x64)](https://stackoverflow.com/a/5507910) (TODO surely there's an official source) can be [specified on init](https://learn.microsoft.com/en-us/dotnet/api/system.threading.thread.-ctor?view=netframework-4.8#system-threading-thread-ctor(system-threading-threadstart-system-int32)), though as with all of this you may have an XY problem and should reconsider. The max. is UNK MiB (default AppDomain; 1 MiB if untrusted) and the min. [seems to be OS-dependent](https://learn.microsoft.com/en-us/dotnet/api/system.threading.thread.-ctor?view=netframework-4.8#system-threading-thread-ctor(system-threading-threadstart-system-int32)) but was [~256 KiB on Vista](https://techcommunity.microsoft.com/t5/windows-blog-archive/pushing-the-limits-of-windows-processes-and-threads/ba-p/723824) and presumably hasn't been changed since.
- If a call would allocate beyond the stack size (or a call chain grows out of control), a `StackOverflowException` will be thrown, which [cannot be caught](https://learn.microsoft.com/en-us/dotnet/api/system.stackoverflowexception?view=netframework-4.8#remarks).
- It should be no surprise that all these limits were inherited from native Win32. Thankfully modern .NET is deviating from that where necessary.
## BCL source