Render scenes upside-down to framebuffer objects (FBO). The strange thing
about rendering to OpenGL FBO is that it follows the bottom-left triangle
rasterization rule with common PC GPUs. At least Intel and AMD. NVIDIA to
be tested. My raster-rule-test github gist demonstrates this.
This commit flips coordinates in y-direction, which effectively turns the
bottom-left rule into top-left rule needed for Xbox compatibility.
This (together with the previous commit) fixes Midtown Madness 3 Seine
water rectangular seam rendering artifacts (and the remaining seams are
present with Xbox hardware too.) May fix similar artifacts in other games.
Xbox triangle rasterization appears to follow the usual top-left rule.
However, since Xemu renders to an OpenGL framebuffer object (FBO) instead
of directly to the default framebuffer, Xemu actually has what could be
called the bottom-left triangle rasterization rule. I'll address that in
another commit.
Also, note that the ProjAdjacentGeometry_0.5625 test in nxdk_pgraph_tests
is very sensitive to floating point rounding errors. For example, the
nxdk_pgraph_tests commit 66b32a0b1feba32a0db7a95d6358e84f7a6246ad changed
the math library which caused the test result to change also on real Xbox
hardware due to floating point rounding error differences in matrix
inverse computation. Apart from the bottom-left rasterization issue, the
differing result between Xbox and the rounding I am proposing here for
Xemu seems to stem from floating point rounding that happens in screen
coordinate calculations before the rounding to 4 bit precision takes place.
Fixing such rounding issues would require carrying all preceding floating
point computations exactly in the same order and with same precision as
Xbox. Note that Xbox Direct3D library seems to add 0.03125 (1/32) to
screen coordinates by default. Likely the idea there was to make floating
point screen coordinates round to the nearest screen coordinates in
4 bit fixed point precision. So the Xbox Direct3D library (and therefore
games) already mitigate against precarious rounding when exactly
half-integer coordinates are used by games. Actually they would use
integer coordinates because it is Direct3D 8, but since nv2a appears to
rasterize at half-integer coordinates like OpenGL, Xbox Direct3D
also adds 0.5 to screen coordinates in addition to 1/32.
Xbox rounds -0.0 to the negative range and 0.0 to the positive range. This
commit also restores RCC instruction clamping to be done on the output of
reciprocal calculation (which current Xemu release does) with fix for the
input=Infinity case.
z_perspective is true implies w-buffering and then the w-coordinate stored
in the depth buffer should also be interpolated in a perspective-correct
way. We do this by calculating w and setting gl_FragDepth in the fragment
shader.
Since enabling polygon offset and setting values using glPolygonOffset
won't have any effect when manually setting gl_FragDepth for w-buffering,
we introduce the depthOffset variable to obtain similar behaviour (but the
glPolygonOffset factor-argument is currently not emulated.) (Note that
glPolygonOffset is OpenGL implementation-dependent and it might be good to
use depthOffset for z-buffering as well, but this is not done here and we
still use OpenGL/Vulkan zbias functionality.)
This also implements depth clipping and clamping in the fragment shader.
If triangles are clipped, the shadows of the small rocks in Halo 2 Beaver
Creek map can have flickering horizontal lines. The shadows are drawn on
the ground in another pass with the same models as for the ground, but for
some reason with depth clamping enabled. The flickering happens if Xemu
clips the ground triangles, but the exact same shadow triangles are depth
clamped, so there are small differences in the coordinates. The shadows
are drawn with depth function GL_EQUAL so there is no tolerance for any
differences. Clipping in the fragment shader solves the problem because
the ground and shadow triangles remain exactly the same regardless of
depth clipping/clamping. For some performance gain, it might be a good
idea to cull triangles by depth in the geometry shader, but this is not
implemented here.
In the programmable vertex shader we always multiply position output by w
because this improves numerical stability in subsequent floating point
computations by modern GPUs. This usually means that the perspective
divide done by the vertex program gets undone.
The magic bounding constants 5.42101e-020 and 1.884467e+019 are replaced
by 5.421011e-20 and 1.8446744e19, i.e. more decimals added. This makes the
32-bit floating point numbers represent exactly 2^(-64) and 2^64 (raw bits
0x1f800000 and 0x5f800000) which seem more likely the correct values
although testing with hardware was not done to this precision.
Testing indicates that the same RCC instruction magic constants are also
applied to both fixed function and programmable vertex shader w-coordinate
output. This bounding replaces the special test for w==0.0 and abs(w)==inf
which used to set vtx_inv_w=1.0 (which did not match Xbox hardware
behaviour.)
Matt Borgerson