Bo3b's School For Shaderhackers
  32 / 87    
[quote="DarkStarSword"]shadertool does have an autofix for reflections, but I'm not sure if it's specific to Life Is Strange or might apply to other UE3 games as well - it fixes shaders that use "DNEReflectionTexture" and you can try it with --auto-fix-unreal-dne-reflection.[/quote] I had a look at the reflection shader but there is no DNEReflectionTexture. As the lake is only visible for 1-2 min in the background I will simply disable the reflection.
DarkStarSword said:shadertool does have an autofix for reflections, but I'm not sure if it's specific to Life Is Strange or might apply to other UE3 games as well - it fixes shaders that use "DNEReflectionTexture" and you can try it with --auto-fix-unreal-dne-reflection.

I had a look at the reflection shader but there is no DNEReflectionTexture. As the lake is only visible for 1-2 min in the background I will simply disable the reflection.

My original display name is 3d4dd - for some reason Nvidia changed it..?!

Posted 03/27/2015 03:42 PM   
Here's two reflections I fixed in Betrayer (this was the first game I fixed, so most of this was still trial and error) - maybe one might be similar to the reflection you have found? The ocean shader already had three stereo corrections from UE3 (all wrong) - I disabled them all and did it once after the view-projection matrix was used instead: https://github.com/DarkStarSword/3d-fixes/blob/master/Betrayer/ShaderOverride/PixelShaders/8D1914C3.txt The lake shader didn't have any UE3 stereo corrections (I suspect this was a custom shader for Betrayer), but the pattern was the same - find where it uses the view-projection matrix then apply the stereo correction: https://github.com/DarkStarSword/3d-fixes/commit/efd7bc65962a497c04a219dbd4f727113e7ab7a8
Here's two reflections I fixed in Betrayer (this was the first game I fixed, so most of this was still trial and error) - maybe one might be similar to the reflection you have found?

The ocean shader already had three stereo corrections from UE3 (all wrong) - I disabled them all and did it once after the view-projection matrix was used instead:

https://github.com/DarkStarSword/3d-fixes/blob/master/Betrayer/ShaderOverride/PixelShaders/8D1914C3.txt


The lake shader didn't have any UE3 stereo corrections (I suspect this was a custom shader for Betrayer), but the pattern was the same - find where it uses the view-projection matrix then apply the stereo correction:

https://github.com/DarkStarSword/3d-fixes/commit/efd7bc65962a497c04a219dbd4f727113e7ab7a8

2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD

Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword

Posted 03/28/2015 12:08 AM   
Thank You for the links! I already had a look at Your Betrayer fix but I couldn't find obvious similarities... This is the code for Woolfe (PS 6E393C69): [code]// disabled reflections lake // Generated by Microsoft (R) HLSL Shader Compiler 9.29.952.3111 // // Parameters: // // float4 AmbientColorAndSkyFactor; // float4 DiffuseOverrideParameter; // float4 LightMapScale[2]; // sampler2D LightMapTextures[2]; // float4 MinZ_MaxZRatio; // sampler2D PixelTexture2D_0; // sampler2D PixelTexture2D_1; // sampler2D SceneColorTexture; // float4 ScreenPositionScaleBias; // float4 SpecularOverrideParameter; // float4 UniformPixelVector_0; // float4 UniformPixelVector_1; // float4 UniformPixelVector_2; // float4 UniformPixelVector_3; // float4 UniformPixelVector_4; // float4 UniformPixelVector_5; // float4 UniformPixelVector_6; // // // Registers: // // Name Reg Size // ------------------------- ----- ---- // UniformPixelVector_0 c0 1 // ScreenPositionScaleBias c1 1 // MinZ_MaxZRatio c2 1 // DiffuseOverrideParameter c4 1 // SpecularOverrideParameter c5 1 // LightMapScale c8 2 // UniformPixelVector_1 c10 1 // UniformPixelVector_2 c11 1 // UniformPixelVector_3 c12 1 // UniformPixelVector_4 c13 1 // UniformPixelVector_5 c14 1 // UniformPixelVector_6 c15 1 // AmbientColorAndSkyFactor c16 1 // LightMapTextures s0 2 // SceneColorTexture s2 1 // PixelTexture2D_0 s3 1 // PixelTexture2D_1 s4 1 // ps_3_0 def c3, 48, 64, -0.5, 0 def c6, 2, -1, 0.699999988, 0 def c7, 0.0105000008, 0.600000024, -9.99999997e-007, 0.300000012 def c17, -0, -0.0199999996, -0.0186586399, 0 def c18, 2.5, 0.100000001, 501, 0.010001 def c19, -0.707106769, -0.408248305, 0.577350259, 0.707106769 def c20, 0.816496611, 0.577350259, 0, 0 dcl_texcoord4 v0 dcl_texcoord6 v1.xyz dcl_texcoord5 v2.xyw dcl_texcoord v3.xy dcl_texcoord1 v4.xy dcl_2d s0 dcl_2d s1 dcl_2d s2 dcl_2d s3 dcl_2d s4 rcp r0.x, v2.w mul r0.xy, r0.x, v2 mad r0.xy, r0, c1, c1.wzzw mov r0.zw, c3.w texldl r1, r0, s2 mov r1.y, c6.y add r0.z, -r1.y, -c2.y add r0.z, -r0.z, r1.w rcp r0.z, r0.z mad r0.z, -c2.x, r0.z, -v2.w mul_sat r0.z, r0.z, c18.w dp3 r0.w, v1, v1 rsq r0.w, r0.w mad r1.x, v1.z, -r0.w, -c6.y mul r2.xyz, r0.w, v1 cmp r0.w, r2.z, r1_abs.x, -c6.y log r1.x, r0.w add r0.w, r0.w, c7.z mul r1.xz, r1.x, c18.xyyw exp r1.z, r1.z exp r1.x, r1.x cmp r1.x, r0.w, r1.x, c3.w mul_pp r0.z, r0.z, r1.z cmp_pp oC0.w, r0.w, r0.z, c3.w mad r3, v4.xyxy, c3.xxyy, c3.z mov r0.w, c3.w dp2add r4.x, c10, r3, r0.w dp2add r4.y, c11, r3, r0.w add r1.zw, r4.xyxy, c12.xyxy add r1.zw, r1, -c3.z texld r4, r1.zwzw, s3 mad r4.xyz, r4, c6.x, c6.y dp2add r3.x, c13, r3.zwzw, r0.w dp2add r3.y, c14, r3.zwzw, r0.w add r0.zw, r3.xyxy, c15.xyxy add r0.zw, r0, -c3.z texld r3, r0.zwzw, s3 mad r3.xyz, r3, c6.x, c6.y lrp r5.xyz, c6.z, r3, r4 mad_sat r0.xy, r5, c7.x, r0 lrp_pp r3.xyz, c6_abs.z, c6_abs.wwyw, r5 nrm_pp r4.xyz, r3 texld r0, r0, s4 mul r0.xyz, r0, c7.y mad r0.xyz, r1.x, r0, c17 mul r0.w, r1.x, c7.w mad r0.xyz, r0.w, r0, -c17 add r1.xyz, -r1.y, -c0 mul_pp r0.xyz, r0, r1 mad_pp r0.xyz, r0, c4.w, c4 dp3 r0.w, r4, r2 mul r1.xyz, r0.w, r4 mad r1.xyz, r1, c6.x, -r2 dp2add_sat_pp r0.w, r1.yzzw, c20, c20.z log_pp r2.x, r0.w dp3_sat_pp r0.w, r1, c19 dp3_sat_pp r1.x, r1.yzxw, c19.yzww log_pp r2.z, r1.x log_pp r2.y, r0.w mul_pp r1.xyz, r2, c18.z exp_pp r2.x, r1.x exp_pp r2.y, r1.y exp_pp r2.z, r1.z texld r1, v3, s1 mul_pp r1.xyz, r1, c9 dp3_pp r0.w, r1, r2 add_pp r2.xyz, c5, c5.w texld r3, v3, s0 mul_pp r3.xyz, r3, c8 mul r2.xyz, r2, r3 mul r2.xyz, r0.w, r2 dp2add_sat_pp r5.x, r4.yzzw, c20, c20.z dp3_sat_pp r5.y, r4, c19 dp3_sat_pp r5.z, r4.yzxw, c19.yzww mul_pp r4.xyz, r5, r5 dp3_pp r0.w, r1, r4 mul_pp r1.xyz, r0.w, r3 mad_pp r1.xyz, r0, r1, r2 add_pp r1.xyz, r1, c0 mad_pp r0.xyz, r0, c16, r1 mad_pp oC0.xyz, r0, v0.w, v0 // approximately 90 instruction slots used (7 texture, 83 arithmetic)[/code] Setting r0 to 0 after line 108 disables the reflections - which would be OK for me as the lake is only visible in one screen for 1-2 min. So don't spend too much time on it ;)
Thank You for the links! I already had a look at Your Betrayer fix but I couldn't find obvious similarities...
This is the code for Woolfe (PS 6E393C69):
// disabled reflections lake
// Generated by Microsoft (R) HLSL Shader Compiler 9.29.952.3111
//
// Parameters:
//
// float4 AmbientColorAndSkyFactor;
// float4 DiffuseOverrideParameter;
// float4 LightMapScale[2];
// sampler2D LightMapTextures[2];
// float4 MinZ_MaxZRatio;
// sampler2D PixelTexture2D_0;
// sampler2D PixelTexture2D_1;
// sampler2D SceneColorTexture;
// float4 ScreenPositionScaleBias;
// float4 SpecularOverrideParameter;
// float4 UniformPixelVector_0;
// float4 UniformPixelVector_1;
// float4 UniformPixelVector_2;
// float4 UniformPixelVector_3;
// float4 UniformPixelVector_4;
// float4 UniformPixelVector_5;
// float4 UniformPixelVector_6;
//
//
// Registers:
//
// Name Reg Size
// ------------------------- ----- ----
// UniformPixelVector_0 c0 1
// ScreenPositionScaleBias c1 1
// MinZ_MaxZRatio c2 1
// DiffuseOverrideParameter c4 1
// SpecularOverrideParameter c5 1
// LightMapScale c8 2
// UniformPixelVector_1 c10 1
// UniformPixelVector_2 c11 1
// UniformPixelVector_3 c12 1
// UniformPixelVector_4 c13 1
// UniformPixelVector_5 c14 1
// UniformPixelVector_6 c15 1
// AmbientColorAndSkyFactor c16 1
// LightMapTextures s0 2
// SceneColorTexture s2 1
// PixelTexture2D_0 s3 1
// PixelTexture2D_1 s4 1
//

ps_3_0
def c3, 48, 64, -0.5, 0
def c6, 2, -1, 0.699999988, 0
def c7, 0.0105000008, 0.600000024, -9.99999997e-007, 0.300000012
def c17, -0, -0.0199999996, -0.0186586399, 0
def c18, 2.5, 0.100000001, 501, 0.010001
def c19, -0.707106769, -0.408248305, 0.577350259, 0.707106769
def c20, 0.816496611, 0.577350259, 0, 0
dcl_texcoord4 v0
dcl_texcoord6 v1.xyz
dcl_texcoord5 v2.xyw
dcl_texcoord v3.xy
dcl_texcoord1 v4.xy
dcl_2d s0
dcl_2d s1
dcl_2d s2
dcl_2d s3
dcl_2d s4
rcp r0.x, v2.w
mul r0.xy, r0.x, v2
mad r0.xy, r0, c1, c1.wzzw
mov r0.zw, c3.w
texldl r1, r0, s2
mov r1.y, c6.y
add r0.z, -r1.y, -c2.y
add r0.z, -r0.z, r1.w
rcp r0.z, r0.z
mad r0.z, -c2.x, r0.z, -v2.w
mul_sat r0.z, r0.z, c18.w
dp3 r0.w, v1, v1
rsq r0.w, r0.w
mad r1.x, v1.z, -r0.w, -c6.y
mul r2.xyz, r0.w, v1
cmp r0.w, r2.z, r1_abs.x, -c6.y
log r1.x, r0.w
add r0.w, r0.w, c7.z
mul r1.xz, r1.x, c18.xyyw
exp r1.z, r1.z
exp r1.x, r1.x
cmp r1.x, r0.w, r1.x, c3.w
mul_pp r0.z, r0.z, r1.z
cmp_pp oC0.w, r0.w, r0.z, c3.w
mad r3, v4.xyxy, c3.xxyy, c3.z
mov r0.w, c3.w
dp2add r4.x, c10, r3, r0.w
dp2add r4.y, c11, r3, r0.w
add r1.zw, r4.xyxy, c12.xyxy
add r1.zw, r1, -c3.z
texld r4, r1.zwzw, s3
mad r4.xyz, r4, c6.x, c6.y
dp2add r3.x, c13, r3.zwzw, r0.w
dp2add r3.y, c14, r3.zwzw, r0.w
add r0.zw, r3.xyxy, c15.xyxy
add r0.zw, r0, -c3.z
texld r3, r0.zwzw, s3
mad r3.xyz, r3, c6.x, c6.y
lrp r5.xyz, c6.z, r3, r4
mad_sat r0.xy, r5, c7.x, r0
lrp_pp r3.xyz, c6_abs.z, c6_abs.wwyw, r5
nrm_pp r4.xyz, r3
texld r0, r0, s4
mul r0.xyz, r0, c7.y
mad r0.xyz, r1.x, r0, c17
mul r0.w, r1.x, c7.w
mad r0.xyz, r0.w, r0, -c17
add r1.xyz, -r1.y, -c0
mul_pp r0.xyz, r0, r1
mad_pp r0.xyz, r0, c4.w, c4
dp3 r0.w, r4, r2
mul r1.xyz, r0.w, r4
mad r1.xyz, r1, c6.x, -r2
dp2add_sat_pp r0.w, r1.yzzw, c20, c20.z
log_pp r2.x, r0.w
dp3_sat_pp r0.w, r1, c19
dp3_sat_pp r1.x, r1.yzxw, c19.yzww
log_pp r2.z, r1.x
log_pp r2.y, r0.w
mul_pp r1.xyz, r2, c18.z
exp_pp r2.x, r1.x
exp_pp r2.y, r1.y
exp_pp r2.z, r1.z
texld r1, v3, s1
mul_pp r1.xyz, r1, c9
dp3_pp r0.w, r1, r2
add_pp r2.xyz, c5, c5.w
texld r3, v3, s0
mul_pp r3.xyz, r3, c8
mul r2.xyz, r2, r3
mul r2.xyz, r0.w, r2
dp2add_sat_pp r5.x, r4.yzzw, c20, c20.z
dp3_sat_pp r5.y, r4, c19
dp3_sat_pp r5.z, r4.yzxw, c19.yzww
mul_pp r4.xyz, r5, r5
dp3_pp r0.w, r1, r4
mul_pp r1.xyz, r0.w, r3
mad_pp r1.xyz, r0, r1, r2
add_pp r1.xyz, r1, c0
mad_pp r0.xyz, r0, c16, r1
mad_pp oC0.xyz, r0, v0.w, v0
// approximately 90 instruction slots used (7 texture, 83 arithmetic)

Setting r0 to 0 after line 108 disables the reflections - which would be OK for me as the lake is only visible in one screen for 1-2 min. So don't spend too much time on it ;)

My original display name is 3d4dd - for some reason Nvidia changed it..?!

Posted 03/28/2015 09:11 AM   
Nothing obvious in that shader (no transformations)... What does the vertex shader look like?
Nothing obvious in that shader (no transformations)... What does the vertex shader look like?

2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD

Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword

Posted 03/28/2015 07:32 PM   
The VS 85E60A00 is quite short: [code]// Generated by Microsoft (R) HLSL Shader Compiler 9.29.952.3111 vs_3_0 dcl_position v0 dcl_texcoord v1 dcl_texcoord o0.xy dcl_position o1 mov o0.xy, v1 mov o1, v0 // approximately 2 instruction slots used[/code]
The VS 85E60A00 is quite short:
// Generated by Microsoft (R) HLSL Shader Compiler 9.29.952.3111
vs_3_0
dcl_position v0
dcl_texcoord v1
dcl_texcoord o0.xy
dcl_position o1
mov o0.xy, v1
mov o1, v0

// approximately 2 instruction slots used

My original display name is 3d4dd - for some reason Nvidia changed it..?!

Posted 03/29/2015 08:10 PM   
Yeah, that's too simple... I think these are likely to be copying the reflection from an off-screen buffer, and not drawing the reflection themselves. How exactly are these reflections broken? There might be other shaders involved, or it might be that the off-screen buffer is not being stereoised. You mentioned disabling this line removed the reflections:[code] texld r0, r0, s4[/code] You can extract a copy of that texture with Helix mod by adding a section like this to your DX9Settings.ini: [code] [PS6E393C69] GetSampler1FromReg=4 [/code] Then press F12 in the game while the reflection is visible (if you get a crash kill the Steam overlay, or change SaveTextureLogKey to a different button). That should save the texture out to Tex1.dds, which you may be able to open in various tools depending on the texture format (e.g. GIMP with the gimp-dds plugin can handle some fomats). Then, check the LOG.txt and find: 1. The line corresponding with when you pressed F12 (I forget exactly what the keyword is to search for), which will tell you some of the properties of the texture (alternatively you can get some of this from the header in the DDS file with https://raw.githubusercontent.com/DarkStarSword/3d-fixes/master/ddsinfo.py ) 2. All lines starting with D3DUSAGE_RENDERTARGET/D3DUSAGE_DEPTHSTENCIL that have the same width & height of the texture you dumped (there should be at least one more aside from the texture you dumped, which will have a different format & usage and be the depth buffer used with that texture). Now, create individual [SFn] sections (don't forget to reference these from SurfaceCreationModeList) that match each unique surface found in the log file. Refer to these instructions - [url]http://wiki.bo3b.net/index.php?title=HelixMod_Feature_List#.5BSFn.5D[/url] In each section set DefMode=1 to force it to be stereo and see what happens.
Yeah, that's too simple... I think these are likely to be copying the reflection from an off-screen buffer, and not drawing the reflection themselves. How exactly are these reflections broken?

There might be other shaders involved, or it might be that the off-screen buffer is not being stereoised.

You mentioned disabling this line removed the reflections:
texld r0, r0, s4

You can extract a copy of that texture with Helix mod by adding a section like this to your DX9Settings.ini:
[PS6E393C69]
GetSampler1FromReg=4

Then press F12 in the game while the reflection is visible (if you get a crash kill the Steam overlay, or change SaveTextureLogKey to a different button). That should save the texture out to Tex1.dds, which you may be able to open in various tools depending on the texture format (e.g. GIMP with the gimp-dds plugin can handle some fomats).

Then, check the LOG.txt and find:
1. The line corresponding with when you pressed F12 (I forget exactly what the keyword is to search for), which will tell you some of the properties of the texture (alternatively you can get some of this from the header in the DDS file with https://raw.githubusercontent.com/DarkStarSword/3d-fixes/master/ddsinfo.py )
2. All lines starting with D3DUSAGE_RENDERTARGET/D3DUSAGE_DEPTHSTENCIL that have the same width & height of the texture you dumped (there should be at least one more aside from the texture you dumped, which will have a different format & usage and be the depth buffer used with that texture).

Now, create individual [SFn] sections (don't forget to reference these from SurfaceCreationModeList) that match each unique surface found in the log file. Refer to these instructions - http://wiki.bo3b.net/index.php?title=HelixMod_Feature_List#.5BSFn.5D

In each section set DefMode=1 to force it to be stereo and see what happens.

2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD

Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword

Posted 03/30/2015 01:02 AM   
Unfortunately LOG.txt and DRWALOG.txt only contain "Start logging..." and TEXTURESLOG.txt only repeats "BeginScene EndScene" several times. And I don't get Tex1.dds. I have tried F12 and other keys for SaveTextureLogKey without success.
Unfortunately LOG.txt and DRWALOG.txt only contain "Start logging..." and TEXTURESLOG.txt only repeats "BeginScene EndScene" several times. And I don't get Tex1.dds. I have tried F12 and other keys for SaveTextureLogKey without success.

My original display name is 3d4dd - for some reason Nvidia changed it..?!

Posted 04/01/2015 06:29 PM   
Is there another LOG.txt in the game's root? I've found it a bit inconsistent with UE3 games, but it should be somewhere. I use GetCurDirAtLoad = true in UE3 games so that most things use the Win32 directory, but the LOG.txt still ends up in the root. If you can get the LOG.txt you could use trial + error to find the correct surfaces to stereoise (assuming this is the problem, which isn't certain). DRAWLOG.txt has always been empty for me, but I did work out that in order to make TEXTURESLOG.txt work you need at least these lines (replacing nnnnnnnn with the hash of the vertex shader in both sections): [code] [VSnnnnnnnn] CheckTexCRC = true VBOffsetList = 0; UseDefinedOnly = false [VBnnnnnnnn.0] [/code] I haven't tried that with pixel shaders (since the vertex buffer syntax is required, but only makes sense for vertex shaders), but you should be able to use the corresponding vertex shader for it. AFAIK it also only lists the most recent texture set by the game whenever the shader is used, but that pixel shader takes 5 textures so it's going to be luck if it lists the right one. If it is the right one than that might at least give you the width & height to match.
Is there another LOG.txt in the game's root? I've found it a bit inconsistent with UE3 games, but it should be somewhere. I use GetCurDirAtLoad = true in UE3 games so that most things use the Win32 directory, but the LOG.txt still ends up in the root. If you can get the LOG.txt you could use trial + error to find the correct surfaces to stereoise (assuming this is the problem, which isn't certain).

DRAWLOG.txt has always been empty for me, but I did work out that in order to make TEXTURESLOG.txt work you need at least these lines (replacing nnnnnnnn with the hash of the vertex shader in both sections):

[VSnnnnnnnn]
CheckTexCRC = true
VBOffsetList = 0;
UseDefinedOnly = false

[VBnnnnnnnn.0]


I haven't tried that with pixel shaders (since the vertex buffer syntax is required, but only makes sense for vertex shaders), but you should be able to use the corresponding vertex shader for it. AFAIK it also only lists the most recent texture set by the game whenever the shader is used, but that pixel shader takes 5 textures so it's going to be luck if it lists the right one. If it is the right one than that might at least give you the width & height to match.

2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD

Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword

Posted 04/02/2015 12:13 AM   
Thanks! There is indeed the correct LOG.txt in the game's root. But I couldn't find a dds file. So I will have to use the other methods to identify width & height You have suggested. But after spending some time with my DoA 5 hotfix now I want to play that game for some hours ;)
Thanks! There is indeed the correct LOG.txt in the game's root. But I couldn't find a dds file. So I will have to use the other methods to identify width & height You have suggested. But after spending some time with my DoA 5 hotfix now I want to play that game for some hours ;)

My original display name is 3d4dd - for some reason Nvidia changed it..?!

Posted 04/02/2015 06:14 PM   
Back after a long break. Made some progress on Besiege. I've managed to disable the AO effect for the house & the first block on Map1. I commented out everything in the shader B70329F9 & that worked. Feels great to make my first fix :). Broken [IMG]http://i292.photobucket.com/albums/mm23/eroc_remag/Besiege03_99_broken_zpswkayzhnz.jpg[/IMG] Fixed [IMG]http://i292.photobucket.com/albums/mm23/eroc_remag/Besiege02_99_fixed_zpsw7vihe13.jpg[/IMG] Also managed to fix the effect on the conifers. The tree in the background is still broken. Looks like I'll have to move the effect into the right place to fix it.
Back after a long break. Made some progress on Besiege. I've managed to disable the AO effect for the house & the first block on Map1. I commented out everything in the shader B70329F9 & that worked. Feels great to make my first fix :).

Broken
Image

Fixed
Image

Also managed to fix the effect on the conifers. The tree in the background is still broken. Looks like I'll have to move the effect into the right place to fix it.

Posted 04/09/2015 11:07 AM   
Looking good! It's pretty fun when you get something that works, isn't it? I think I might need to add that remove-the-code as the more reliable way to kill a given shader. The zeroed output is not always reliable.
Looking good! It's pretty fun when you get something that works, isn't it? I think I might need to add that remove-the-code as the more reliable way to kill a given shader. The zeroed output is not always reliable.

Acer H5360 (1280x720@120Hz) - ASUS VG248QE with GSync mod - 3D Vision 1&2 - Driver 372.54
GTX 970 - i5-4670K@4.2GHz - 12GB RAM - Win7x64+evilKB2670838 - 4 Disk X25 RAID
SAGER NP9870-S - GTX 980 - i7-6700K - Win10 Pro 1607
Latest 3Dmigoto Release
Bo3b's School for ShaderHackers

Posted 04/10/2015 08:58 PM   
[quote="bo3b"] I think I might need to add that remove-the-code as the more reliable way to kill a given shader. The zeroed output is not always reliable.[/quote] Yes I think that would be really useful for beginners. With this game I found that zeroed output gives entirely different output from the one you'd normally expect. Disabling the shader code entirely is a useful trick to know. I had no idea that it could be done until DarkStarSword mentioned it.
bo3b said: I think I might need to add that remove-the-code as the more reliable way to kill a given shader. The zeroed output is not always reliable.


Yes I think that would be really useful for beginners. With this game I found that zeroed output gives entirely different output from the one you'd normally expect. Disabling the shader code entirely is a useful trick to know. I had no idea that it could be done until DarkStarSword mentioned it.

Posted 04/13/2015 06:27 AM   
I've managed to disable the effect on all objects finally! I couldn't find any other shader that affects only the tree in the background so I went back to the original shader that I mentioned in comment#381. Just on a whim I switched oC0, r0 to oC0, r1 and it worked! The only side effect of the 'fix' is that the shadows on the ground are disabled. Here's the code: [code] ps_3_0 // Converted from ps_2_0 with DarkStarSword's shadertool.py dcl_2d s0 dcl_2d s1 dcl_texcoord v0.xy dcl_texcoord1 v1 dcl_texcoord2 v2.xyz texldp r0, v1, s1 //same effect as setting oC0 to 1111 texld r1, v0, s0 //makes the whole scene dark. same as setting oC0 to 0000 mul r1, r1, c0 log_pp r0.x, r0.x log_pp r0.z, r0.z log_pp r0.y, r0.y add_pp r0.xyz, -r0, v2 mul_pp r0.xyz, r1, r0 mov_pp r0.w, r1 mov_pp oC0, r1 //setting oC0 to r1 fixes the broken AO effect on conifers & background tree [/code] Broken: [IMG]http://i292.photobucket.com/albums/mm23/eroc_remag/Besiege/Besiege02_99new_broken_zpssgqujgm0.jpg[/IMG] Fixed: [IMG]http://i292.photobucket.com/albums/mm23/eroc_remag/Besiege/Besiege03_99new_fixed_zps7cb0vrtv.jpg[/IMG] Now this truly was a shot in the dark. I don't know why it worked & how. Bob & others, can you guys help me understand this or is it really not important to understand the fix as long as it does what it's supposed to do.
I've managed to disable the effect on all objects finally! I couldn't find any other shader that affects only the tree in the background so I went back to the original shader that I mentioned in comment#381. Just on a whim I switched oC0, r0 to oC0, r1 and it worked! The only side effect of the 'fix' is that the shadows on the ground are disabled.

Here's the code:
ps_3_0 // Converted from ps_2_0 with DarkStarSword's shadertool.py

dcl_2d s0
dcl_2d s1
dcl_texcoord v0.xy
dcl_texcoord1 v1
dcl_texcoord2 v2.xyz
texldp r0, v1, s1 //same effect as setting oC0 to 1111
texld r1, v0, s0 //makes the whole scene dark. same as setting oC0 to 0000
mul r1, r1, c0
log_pp r0.x, r0.x
log_pp r0.z, r0.z
log_pp r0.y, r0.y
add_pp r0.xyz, -r0, v2
mul_pp r0.xyz, r1, r0
mov_pp r0.w, r1
mov_pp oC0, r1 //setting oC0 to r1 fixes the broken AO effect on conifers & background tree


Broken:
Image

Fixed:
Image

Now this truly was a shot in the dark. I don't know why it worked & how. Bob & others, can you guys help me understand this or is it really not important to understand the fix as long as it does what it's supposed to do.

Posted 04/13/2015 08:54 AM   
@eroc_remag: Congratulations on that success! To answer your question as to whether it is important to understand or not- there are two broad levels of fixing. First is that a lot of the times, we are just poking the black box and don't really understand what is going on. There is a lot of variance in games, and the shaders are often inscrutably complex to where we cannot understand what they are doing simply from the ASM. This is exacerbated when they don't include headers, which seems to be happening more now. So at the first level, any fix is better than broken, so trial and error is not a bad way to try to see if we can improve a game to make it playable. We can rarely do the highest quality fixes this way though, as it's more luck than anything. Always nice to be lucky, regardless. The second level of fixes are ones where we can actually pin down what it's doing in the shaders, and apply the fix at the right spot in the right shader. This is a rare skill, and I'm not going to kid anyone here, it's unlikely that most of us will be able to get to this level. The only people who have been able to do this on a regular basis are Helix, Mike_ar69, and DarkStarSword. These are de novo fixes for engines, where there is no example to work from. Based on what we know of rendering techniques, it is sometimes possible to find these sorts of fixes and work out what the shader is doing at least well enough to find the right spot for fixes. There is a fair amount of trial and error here as well, trying different experiments to work out what the shaders are doing. There is perhaps a third level, like this shader, where you can make an educated guess on what to change based on the outputs and the code you see. In this case for example, the output is normally getting the last parameter the .w from r1, which for this pixel shader is actually the alpha channel 'a' of .rgba. And the other r0 bits would be from a multiply of r1 and r0, which will be skipped if you use r1, making the output just whatever r1 starts with. This is the inscrutable part of the shader- what is c0? Are their headers that you've stripped? Always supply us with everything in a shader, it's ALL important, but generally the comments and the headers are far more valuable than the code itself. For some reason coders tend to think comments are useless and should be skipped. Are there any names for sampler s0 or s1? Sometimes the naming can really break open what is happening. Outside of further info, like suggestive variable names, or having seen a code sequence before, I can't personally see anything too significant here. It's not a usual transform operation using dot products, and because of the log ops, my best guess is some sort of transparency effect being applied. If that's true, then the alpha channel being set to r1 instead of r0 would mean it's getting the transparency from the r1 sampler, and mostly skipping the addition of r0 sampler. Long story short, my best guess is that this is just changing the alpha channel of the output and making the broken effect invisible. Based on those pictures though, it looks to me like this is a halo-style problem, where you would normally fix that in the associated vertex shader, by making the location of the texture match the location of the output. If you have the vertex shader we can look at that as well. You can also sometimes fix that problem in the pixel shader by moving the .x location of an input texture. You could try applying the prime directive to the v2.x and maybe the v1, before the textureld ops. That can conceivably fix the location of the texture and move it to line up.
@eroc_remag: Congratulations on that success!

To answer your question as to whether it is important to understand or not- there are two broad levels of fixing.

First is that a lot of the times, we are just poking the black box and don't really understand what is going on. There is a lot of variance in games, and the shaders are often inscrutably complex to where we cannot understand what they are doing simply from the ASM. This is exacerbated when they don't include headers, which seems to be happening more now. So at the first level, any fix is better than broken, so trial and error is not a bad way to try to see if we can improve a game to make it playable. We can rarely do the highest quality fixes this way though, as it's more luck than anything. Always nice to be lucky, regardless.

The second level of fixes are ones where we can actually pin down what it's doing in the shaders, and apply the fix at the right spot in the right shader. This is a rare skill, and I'm not going to kid anyone here, it's unlikely that most of us will be able to get to this level. The only people who have been able to do this on a regular basis are Helix, Mike_ar69, and DarkStarSword. These are de novo fixes for engines, where there is no example to work from. Based on what we know of rendering techniques, it is sometimes possible to find these sorts of fixes and work out what the shader is doing at least well enough to find the right spot for fixes. There is a fair amount of trial and error here as well, trying different experiments to work out what the shaders are doing.


There is perhaps a third level, like this shader, where you can make an educated guess on what to change based on the outputs and the code you see.

In this case for example, the output is normally getting the last parameter the .w from r1, which for this pixel shader is actually the alpha channel 'a' of .rgba. And the other r0 bits would be from a multiply of r1 and r0, which will be skipped if you use r1, making the output just whatever r1 starts with.

This is the inscrutable part of the shader- what is c0? Are their headers that you've stripped? Always supply us with everything in a shader, it's ALL important, but generally the comments and the headers are far more valuable than the code itself. For some reason coders tend to think comments are useless and should be skipped. Are there any names for sampler s0 or s1? Sometimes the naming can really break open what is happening.

Outside of further info, like suggestive variable names, or having seen a code sequence before, I can't personally see anything too significant here. It's not a usual transform operation using dot products, and because of the log ops, my best guess is some sort of transparency effect being applied. If that's true, then the alpha channel being set to r1 instead of r0 would mean it's getting the transparency from the r1 sampler, and mostly skipping the addition of r0 sampler.

Long story short, my best guess is that this is just changing the alpha channel of the output and making the broken effect invisible.


Based on those pictures though, it looks to me like this is a halo-style problem, where you would normally fix that in the associated vertex shader, by making the location of the texture match the location of the output. If you have the vertex shader we can look at that as well.

You can also sometimes fix that problem in the pixel shader by moving the .x location of an input texture. You could try applying the prime directive to the v2.x and maybe the v1, before the textureld ops. That can conceivably fix the location of the texture and move it to line up.

Acer H5360 (1280x720@120Hz) - ASUS VG248QE with GSync mod - 3D Vision 1&2 - Driver 372.54
GTX 970 - i5-4670K@4.2GHz - 12GB RAM - Win7x64+evilKB2670838 - 4 Disk X25 RAID
SAGER NP9870-S - GTX 980 - i7-6700K - Win10 Pro 1607
Latest 3Dmigoto Release
Bo3b's School for ShaderHackers

Posted 04/13/2015 08:16 PM   
@Bo3b - According to MSDN c0 is a constant float register. https://msdn.microsoft.com/en-us/library/windows/desktop/bb219857(v=vs.85).aspx. It is not declared in the shader so must be a global. I tried applying the prime directive to v0, v1 & v2 but that doesn't help. Here's the complete 4C7711F0 shader verbatim. [code]//AO-like effect rendering at 2x separation on trees on map1 on Ipsilon. //If I disable this shader in debug mode, then the issue is fixed. //But if I set oC0 to 1111 or 0000 it is not fixed. //commenting out everything in this shader breaks the tree trunk while disabling the effect //setting oC0 to r1 disables the broken AO effect on conifers & background tree ps_3_0 // Converted from ps_2_0 with DarkStarSword's shadertool.py //def c49, Const1, Const2, Const3, Const4 comes in from dx9settings.ini file def c199, 0, 1, 0.0625, 1 dcl_2d s0 dcl_2d s1 dcl_2d s14 //Helix's sampler dcl_texcoord v0.xy dcl_texcoord1 v1 dcl_texcoord2 v2.xyz // //// Fetch the Separation & Convergence using the Helix NVapi trick. c199.z does that. //Applying prime directive on v0.x texld r3, c199.z, s14 //use Helix sampler s14. Now r3.x = Separation, r3.y = Convergence add r3.w, v0.w, -r3.y // (W - Convergence) store in r3.w mul r3.z, r3.x, r3.w // Separation* (W-Convergence) store in r3.z add v0.x, v0.x, r3.z // Xold + Separation*(W-Convergence) //Applying prime directive on v1.x texld r4, c199.z, s14 //use Helix sampler s14. Now r4.x = Separation, r4.y = Convergence add r4.w, v1.w, -r4.y // (W - Convergence) store in r4.w mul r4.z, r4.x, r4.w // Separation* (W-Convergence) store in r4.z add v1.x, v1.x, r4.z // Xold + Separation*(W-Convergence) //Applying prime directive on v2.x texld r5, c199.z, s14 //use Helix sampler s14. Now r5.x = Separation, r5.y = Convergence add r5.w, v2.w, -r5.y // (W - Convergence) store in r5.w mul r5.z, r5.x, r5.w // Separation* (W-Convergence) store in r5.z add v2.x, v2.x, r5.z // Xold + Separation*(W-Convergence) texldp r0, v1, s1 //same effect as setting oC0 to 1111 texld r1, v0, s0 //makes the whole scene dark. same as setting oC0 to 0000 mul r1, r1, c0 log_pp r0.x, r0.x log_pp r0.z, r0.z log_pp r0.y, r0.y add_pp r0.xyz, -r0, v2 mul_pp r0.xyz, r1, r0 mov_pp r0.w, r1 mov_pp oC0, r0 //mov_pp oC0, r1 //setting oC0 to r1 fixes the broken AO effect on conifers & background tree // mov r30, c200.wwww // texkill r30 //using texkill to try & disable the effect of the shader. doesn't work. disables the tree trunks. // //disable the effect by setting output color register oC0 to 1111. // mov r30.x, c49.x // if_eq r30.x, c200.x // mov oC0.xyzw, c200.wwwy //0 doesn't work. 1 fixes effect but breaks tree trunk // endif // approximately 10 instruction slots used (2 texture, 8 arithmetic) [/code]
@Bo3b - According to MSDN c0 is a constant float register. https://msdn.microsoft.com/en-us/library/windows/desktop/bb219857(v=vs.85).aspx. It is not declared in the shader so must be a global.

I tried applying the prime directive to v0, v1 & v2 but that doesn't help. Here's the complete 4C7711F0 shader verbatim.

//AO-like effect rendering at 2x separation on trees on map1 on Ipsilon. 
//If I disable this shader in debug mode, then the issue is fixed.
//But if I set oC0 to 1111 or 0000 it is not fixed.
//commenting out everything in this shader breaks the tree trunk while disabling the effect
//setting oC0 to r1 disables the broken AO effect on conifers & background tree

ps_3_0 // Converted from ps_2_0 with DarkStarSword's shadertool.py

//def c49, Const1, Const2, Const3, Const4 comes in from dx9settings.ini file
def c199, 0, 1, 0.0625, 1
dcl_2d s0
dcl_2d s1

dcl_2d s14 //Helix's sampler

dcl_texcoord v0.xy
dcl_texcoord1 v1
dcl_texcoord2 v2.xyz

// //// Fetch the Separation & Convergence using the Helix NVapi trick. c199.z does that.
//Applying prime directive on v0.x
texld r3, c199.z, s14 //use Helix sampler s14. Now r3.x = Separation, r3.y = Convergence
add r3.w, v0.w, -r3.y // (W - Convergence) store in r3.w
mul r3.z, r3.x, r3.w // Separation* (W-Convergence) store in r3.z
add v0.x, v0.x, r3.z // Xold + Separation*(W-Convergence)

//Applying prime directive on v1.x
texld r4, c199.z, s14 //use Helix sampler s14. Now r4.x = Separation, r4.y = Convergence
add r4.w, v1.w, -r4.y // (W - Convergence) store in r4.w
mul r4.z, r4.x, r4.w // Separation* (W-Convergence) store in r4.z
add v1.x, v1.x, r4.z // Xold + Separation*(W-Convergence)

//Applying prime directive on v2.x
texld r5, c199.z, s14 //use Helix sampler s14. Now r5.x = Separation, r5.y = Convergence
add r5.w, v2.w, -r5.y // (W - Convergence) store in r5.w
mul r5.z, r5.x, r5.w // Separation* (W-Convergence) store in r5.z
add v2.x, v2.x, r5.z // Xold + Separation*(W-Convergence)

texldp r0, v1, s1 //same effect as setting oC0 to 1111
texld r1, v0, s0 //makes the whole scene dark. same as setting oC0 to 0000
mul r1, r1, c0
log_pp r0.x, r0.x
log_pp r0.z, r0.z
log_pp r0.y, r0.y
add_pp r0.xyz, -r0, v2
mul_pp r0.xyz, r1, r0
mov_pp r0.w, r1
mov_pp oC0, r0

//mov_pp oC0, r1 //setting oC0 to r1 fixes the broken AO effect on conifers & background tree


// mov r30, c200.wwww
// texkill r30 //using texkill to try & disable the effect of the shader. doesn't work. disables the tree trunks.

// //disable the effect by setting output color register oC0 to 1111.
// mov r30.x, c49.x
// if_eq r30.x, c200.x
// mov oC0.xyzw, c200.wwwy //0 doesn't work. 1 fixes effect but breaks tree trunk
// endif
// approximately 10 instruction slots used (2 texture, 8 arithmetic)

Posted 04/17/2015 11:31 AM   
  32 / 87    
Scroll To Top