GSDX: Fixed my inability to remember or look up (I swear that page was hiding) coordinate systems in r5279 and assumed that it was a rounding error instead. The symptom was that palette indices above around 250 were being rounded up to the next palette entry causing visible glitches (only in D3D10 for some reason). Changed the code to keep the fractional part after multiplication by 256 around 0.5 and the steps around 1.0. Should be very safe against small errors.

git-svn-id: http://pcsx2.googlecode.com/svn/trunk@5280 96395faa-99c1-11dd-bbfe-3dabce05a288
This commit is contained in:
sudonim1@gmail.com 2012-06-09 08:03:27 +00:00
parent 1df5b7ef7a
commit 4f79c4d1cb
1 changed files with 2 additions and 2 deletions

View File

@ -109,7 +109,7 @@ float4 sample_c(float2 uv)
float4 sample_p(float u) float4 sample_p(float u)
{ {
return Palette.Sample(PaletteSampler, u * 255./256); return Palette.Sample(PaletteSampler, u * 255./256 + 0.5/256);
} }
float4 sample_rt(float2 uv) float4 sample_rt(float2 uv)
@ -199,7 +199,7 @@ float4 sample_c(float2 uv)
float4 sample_p(float u) float4 sample_p(float u)
{ {
return tex2D(Palette, u * 255./256 + 1./512); return tex2D(Palette, u * 255./256 + 0.5/256);
} }
float4 sample_rt(float2 uv) float4 sample_rt(float2 uv)