gsdx-sw: Nan is not well supported in Vertex

It fixes the bad light on Silent Hill with the SW renderer.

Full story
if Q is NaN, m_vt.m_eq.q becomes wrongly true
/Q will wrongly be optimized in the "Vertex Shader" of the SW

Note: Add an assert for the STQ handler

Code path is quite hot so no need to add extra check for nothing
This commit is contained in:
Gregory Hainaut 2015-07-19 16:24:55 +02:00
parent 7f76883b98
commit 6719fc89a6
3 changed files with 4 additions and 0 deletions

Binary file not shown.

Binary file not shown.

View File

@ -503,6 +503,7 @@ void GSState::GIFPackedRegHandlerSTQ(const GIFPackedReg* RESTRICT r)
q = q.blend8(GSVector4i::cast(GSVector4::m_one), q == GSVector4i::zero()); // character shadow in Vexx, q = 0 (sTex also 0 on the first 16 vertices), setting it to 1.0f to avoid div by zero later
*(int*)&m_q = GSVector4i::store(q);
ASSERT(!std::isnan(m_q)); // See GIFRegHandlerRGBAQ
#ifdef Offset_ST
GIFRegTEX0 TEX0 = m_context->TEX0;
@ -708,6 +709,9 @@ void GSState::GIFRegHandlerRGBAQ(const GIFReg* RESTRICT r)
rgbaq = rgbaq.upl32(rgbaq.blend8(GSVector4i::cast(GSVector4::m_one), rgbaq == GSVector4i::zero()).yyyy()); // see GIFPackedRegHandlerSTQ
m_v.RGBAQ = rgbaq;
if (std::isnan(m_v.RGBAQ.Q)) {
m_v.RGBAQ.Q = std::numeric_limits<float>::max();
}
}
void GSState::GIFRegHandlerST(const GIFReg* RESTRICT r)