Bo3b's School For Shaderhackers
  35 / 87    
@eroc_remag: Great job on finding the problem! This is sometimes the problem we face with HelixMod, no question. Some inscrutable failure with no hints as to what went wrong, everything looks right. Partly this is HelixMod with some funny techniques, but also partly it's the nature of the games doing things in different and oftentimes bizarre ways that Helix doesn't expect. So, the upshot is that we can expect this to happen periodically and it's always frustrating. The key to scenarios like this where no one can see anything wrong is to do just what you did, and poke at it some, try different things, see if you can collect any further info. Try simplifying the scenarios like you did, down to single shaders, no changes, remove pieces that should have no effect, but might, like the .ini file. Now that you've narrowed it to the .ini file you can do the same thing in it, where you can remove chunks of it to narrow down what piece causes the problem. We usually use a binary-search approach for this sort of thing, where we kill half the pieces and see if it works, then the next 1/4, then next 1/8 until you find the offender. Not always that clear, but that gets you to the root problem the quickest. In the course, I talk about searching through the full dump folder to see if any of the constants (like DefPSConst1) are used in any other shader. This is the sort of thing that prompts those ideas. There can be funny interactions between pieces that in theory should have no effect.
@eroc_remag: Great job on finding the problem! This is sometimes the problem we face with HelixMod, no question. Some inscrutable failure with no hints as to what went wrong, everything looks right.

Partly this is HelixMod with some funny techniques, but also partly it's the nature of the games doing things in different and oftentimes bizarre ways that Helix doesn't expect. So, the upshot is that we can expect this to happen periodically and it's always frustrating.


The key to scenarios like this where no one can see anything wrong is to do just what you did, and poke at it some, try different things, see if you can collect any further info.

Try simplifying the scenarios like you did, down to single shaders, no changes, remove pieces that should have no effect, but might, like the .ini file.

Now that you've narrowed it to the .ini file you can do the same thing in it, where you can remove chunks of it to narrow down what piece causes the problem. We usually use a binary-search approach for this sort of thing, where we kill half the pieces and see if it works, then the next 1/4, then next 1/8 until you find the offender. Not always that clear, but that gets you to the root problem the quickest.


In the course, I talk about searching through the full dump folder to see if any of the constants (like DefPSConst1) are used in any other shader. This is the sort of thing that prompts those ideas. There can be funny interactions between pieces that in theory should have no effect.

Acer H5360 (1280x720@120Hz) - ASUS VG248QE with GSync mod - 3D Vision 1&2 - Driver 372.54
GTX 970 - i5-4670K@4.2GHz - 12GB RAM - Win7x64+evilKB2670838 - 4 Disk X25 RAID
SAGER NP9870-S - GTX 980 - i7-6700K - Win10 Pro 1607
Latest 3Dmigoto Release
Bo3b's School for ShaderHackers

Posted 05/15/2015 10:21 AM   
[quote="eroc_remag"] And here's the shader code. The only change in this shader is conversion from PS2 to PS3 using DarkStar's script. The rest of the code is untouched. [code] //PS for shadows? ps_3_0 // Converted from ps_2_0 with DarkStarSword's shadertool.py dcl_2d s0 dcl_2d s1 def c27, 0, 1, 255, 0.00392156979 dcl_texcoord v0.xy dcl_texcoord1 v1.xyz texld r0, v0, s0 mad r0.x, r0, c20, c20.y rcp r0.x, r0.x mul r2.xyz, r0.x, v1 mov r2.w, c27.y dp4 r3.z, r2, c18 dp4 r3.x, r2, c16 dp4 r3.y, r2, c17 dp4 r3.w, r2, c19 add r1.xyz, r3, -c21 add r5.xyz, r3, -c24 dp3 r1.x, r1, r1 add r4.xyz, r3, -c22 dp3 r1.y, r4, r4 add r4.xyz, r3, -c23 dp3 r1.z, r4, r4 dp3 r1.w, r5, r5 add r1, r1, -c25 cmp r1, r1, c27.x, c27.y mov r4.x, r1.y mov r4.y, r1.z mov r4.z, r1.w add_sat r5.xyz, r4, -r1 dp4 r2.z, r3, c6 dp4 r2.x, r3, c4 dp4 r2.y, r3, c5 mul r4.xyz, r5.x, r2 dp4 r2.z, r3, c2 dp4 r2.y, r3, c1 dp4 r2.x, r3, c0 mad r2.xyz, r1.x, r2, r4 dp4 r1.z, r3, c10 dp4 r1.y, r3, c9 dp4 r1.x, r3, c8 mad r1.xyz, r5.y, r1, r2 dp4 r2.y, r3, c13 dp4 r2.x, r3, c12 dp4 r2.z, r3, c14 mad r1.xyz, r2, r5.z, r1 add r2.x, -r0, c27.y mov r0.x, c26 mul r2.xy, r2.x, c27.yzxw frc r2.xy, r2 add r0.x, c27.y, -r0 mov r0.y, c27 texld r1, r1, s1 mad r0.x, r1, r0, c26 mov r1.y, r2 mad r1.x, -r2.y, c27.w, r2 mov r0.w, r1.y mov r0.z, r1.x mov_pp oC0, r0 // approximately 52 instruction slots used (2 texture, 50 arithmetic) [/code] [/quote] Yes. I believe that's the shadow shader you are looking for. If you haven't already, try this and let me know what you get: [code]// ps_3_0 def c220, 0.6, 0, 0.0625, 0 dcl_2d s13 dcl_2d s0 dcl_2d s1 def c27, 0, 1, 255, 0.00392156979 dcl_texcoord v0.xy dcl_texcoord1 v1.xyz texld r0, v0, s0 mad r0.x, r0, c20, c20.y rcp r0.x, r0.x mul r2.xyz, r0.x, v1 mov r2.w, c27.y texldl r24, c220.z, s13 add r24.y, r2.z, -r24.y mul r24.x, r24.x, r24.y mad r2.x, r24.x, -c220.x, r2.x dp4 r3.z, r2, c18 dp4 r3.x, r2, c16 dp4 r3.y, r2, c17 dp4 r3.w, r2, c19 add r1.xyz, r3, -c21 add r5.xyz, r3, -c24 dp3 r1.x, r1, r1 add r4.xyz, r3, -c22 dp3 r1.y, r4, r4 add r4.xyz, r3, -c23 dp3 r1.z, r4, r4 dp3 r1.w, r5, r5 add r1, r1, -c25 cmp r1, r1, c27.x, c27.y mov r4.x, r1.y mov r4.y, r1.z mov r4.z, r1.w add_sat r5.xyz, r4, -r1 dp4 r2.z, r3, c6 dp4 r2.x, r3, c4 dp4 r2.y, r3, c5 mul r4.xyz, r5.x, r2 dp4 r2.z, r3, c2 dp4 r2.y, r3, c1 dp4 r2.x, r3, c0 mad r2.xyz, r1.x, r2, r4 dp4 r1.z, r3, c10 dp4 r1.y, r3, c9 dp4 r1.x, r3, c8 mad r1.xyz, r5.y, r1, r2 dp4 r2.y, r3, c13 dp4 r2.x, r3, c12 dp4 r2.z, r3, c14 mad r1.xyz, r2, r5.z, r1 add r2.x, -r0, c27.y mov r0.x, c26 mul r2.xy, r2.x, c27.yzxw frc r2.xy, r2 add r0.x, c27.y, -r0 mov r0.y, c27 texld r1, r1, s1 mad r0.x, r1, r0, c26 mov r1.y, r2 mad r1.x, -r2.y, c27.w, r2 mov r0.w, r1.y mov r0.z, r1.x mov_pp oC0, r0 [/code] Adjust c220.x up or down until the shadows line up with the ground (the higher the number, the lower the shadow will be).
eroc_remag said:
And here's the shader code. The only change in this shader is conversion from PS2 to PS3 using DarkStar's script. The rest of the code is untouched.

//PS for shadows?

ps_3_0 // Converted from ps_2_0 with DarkStarSword's shadertool.py
dcl_2d s0
dcl_2d s1
def c27, 0, 1, 255, 0.00392156979
dcl_texcoord v0.xy
dcl_texcoord1 v1.xyz
texld r0, v0, s0
mad r0.x, r0, c20, c20.y
rcp r0.x, r0.x
mul r2.xyz, r0.x, v1
mov r2.w, c27.y
dp4 r3.z, r2, c18
dp4 r3.x, r2, c16
dp4 r3.y, r2, c17
dp4 r3.w, r2, c19
add r1.xyz, r3, -c21
add r5.xyz, r3, -c24
dp3 r1.x, r1, r1
add r4.xyz, r3, -c22
dp3 r1.y, r4, r4
add r4.xyz, r3, -c23
dp3 r1.z, r4, r4
dp3 r1.w, r5, r5
add r1, r1, -c25
cmp r1, r1, c27.x, c27.y
mov r4.x, r1.y
mov r4.y, r1.z
mov r4.z, r1.w
add_sat r5.xyz, r4, -r1
dp4 r2.z, r3, c6
dp4 r2.x, r3, c4
dp4 r2.y, r3, c5
mul r4.xyz, r5.x, r2
dp4 r2.z, r3, c2
dp4 r2.y, r3, c1
dp4 r2.x, r3, c0
mad r2.xyz, r1.x, r2, r4
dp4 r1.z, r3, c10
dp4 r1.y, r3, c9
dp4 r1.x, r3, c8
mad r1.xyz, r5.y, r1, r2
dp4 r2.y, r3, c13
dp4 r2.x, r3, c12
dp4 r2.z, r3, c14
mad r1.xyz, r2, r5.z, r1
add r2.x, -r0, c27.y
mov r0.x, c26
mul r2.xy, r2.x, c27.yzxw
frc r2.xy, r2
add r0.x, c27.y, -r0
mov r0.y, c27
texld r1, r1, s1
mad r0.x, r1, r0, c26
mov r1.y, r2
mad r1.x, -r2.y, c27.w, r2
mov r0.w, r1.y
mov r0.z, r1.x
mov_pp oC0, r0

// approximately 52 instruction slots used (2 texture, 50 arithmetic)



Yes. I believe that's the shadow shader you are looking for. If you haven't already, try this and let me know what you get:

//
ps_3_0
def c220, 0.6, 0, 0.0625, 0
dcl_2d s13
dcl_2d s0
dcl_2d s1
def c27, 0, 1, 255, 0.00392156979
dcl_texcoord v0.xy
dcl_texcoord1 v1.xyz
texld r0, v0, s0
mad r0.x, r0, c20, c20.y
rcp r0.x, r0.x
mul r2.xyz, r0.x, v1
mov r2.w, c27.y

texldl r24, c220.z, s13
add r24.y, r2.z, -r24.y
mul r24.x, r24.x, r24.y
mad r2.x, r24.x, -c220.x, r2.x

dp4 r3.z, r2, c18
dp4 r3.x, r2, c16
dp4 r3.y, r2, c17
dp4 r3.w, r2, c19
add r1.xyz, r3, -c21
add r5.xyz, r3, -c24
dp3 r1.x, r1, r1
add r4.xyz, r3, -c22
dp3 r1.y, r4, r4
add r4.xyz, r3, -c23
dp3 r1.z, r4, r4
dp3 r1.w, r5, r5
add r1, r1, -c25
cmp r1, r1, c27.x, c27.y
mov r4.x, r1.y
mov r4.y, r1.z
mov r4.z, r1.w
add_sat r5.xyz, r4, -r1
dp4 r2.z, r3, c6
dp4 r2.x, r3, c4
dp4 r2.y, r3, c5
mul r4.xyz, r5.x, r2
dp4 r2.z, r3, c2
dp4 r2.y, r3, c1
dp4 r2.x, r3, c0
mad r2.xyz, r1.x, r2, r4
dp4 r1.z, r3, c10
dp4 r1.y, r3, c9
dp4 r1.x, r3, c8
mad r1.xyz, r5.y, r1, r2
dp4 r2.y, r3, c13
dp4 r2.x, r3, c12
dp4 r2.z, r3, c14
mad r1.xyz, r2, r5.z, r1
add r2.x, -r0, c27.y
mov r0.x, c26
mul r2.xy, r2.x, c27.yzxw
frc r2.xy, r2
add r0.x, c27.y, -r0
mov r0.y, c27
texld r1, r1, s1
mad r0.x, r1, r0, c26
mov r1.y, r2
mad r1.x, -r2.y, c27.w, r2
mov r0.w, r1.y
mov r0.z, r1.x
mov_pp oC0, r0


Adjust c220.x up or down until the shadows line up with the ground (the higher the number, the lower the shadow will be).

Dual boot Win 7 x64 & Win 10 (1809) | Geforce Drivers 417.35

Posted 05/15/2015 11:03 PM   
[quote="DarkStarSword"] It might be easier to try to force the render target for the portraits to mono instead. It's a bit tricky as there's no easy way to tell which render targets you will need to change, but it can be done. Read the first paragraph of this reference and see if you can make sense of it: [url]http://wiki.bo3b.net/index.php?title=HelixMod_Feature_List#.5BSFn.5D[/url][/quote] from the documentation: [quote]The information for these surfaces comes from the LOG.txt file - look for lines starting with D3DUSAGE_RENDERTARGET/D3DUSAGE_DEPTHSTENCIL. The value of "n" is an integer from 0 - 9. [/quote] There are no such entries in my log. Is that because I have to use an older version of the debug dll? (The one actually called "debug". The newer ones don't work with the game).
DarkStarSword said:
It might be easier to try to force the render target for the portraits to mono instead. It's a bit tricky as there's no easy way to tell which render targets you will need to change, but it can be done. Read the first paragraph of this reference and see if you can make sense of it:
http://wiki.bo3b.net/index.php?title=HelixMod_Feature_List#.5BSFn.5D


from the documentation:

The information for these surfaces comes from the LOG.txt file - look for lines starting with D3DUSAGE_RENDERTARGET/D3DUSAGE_DEPTHSTENCIL. The value of "n" is an integer from 0 - 9.


There are no such entries in my log. Is that because I have to use an older version of the debug dll? (The one actually called "debug". The newer ones don't work with the game).

Posted 05/16/2015 09:55 AM   
[quote="Muizer"]There are no such entries in my log. Is that because I have to use an older version of the debug dll? (The one actually called "debug". The newer ones don't work with the game).[/quote]It could be - I'm not sure what version that was introduced in. Are there any lines starting with CreateRenderTarget (use [RTn] instead of [SFn]) or CreateDepthStencilSurface (use [DSn])? What's the issue with the latest helix mod and that game (also - remind me which game you were looking at)? If it's a crash on launch issue you might be able to use the injector version (32bit only AFAIK), or try killing the Steam overlay by removing GameOverlayRenderer.dll
Muizer said:There are no such entries in my log. Is that because I have to use an older version of the debug dll? (The one actually called "debug". The newer ones don't work with the game).
It could be - I'm not sure what version that was introduced in. Are there any lines starting with CreateRenderTarget (use [RTn] instead of [SFn]) or CreateDepthStencilSurface (use [DSn])?

What's the issue with the latest helix mod and that game (also - remind me which game you were looking at)? If it's a crash on launch issue you might be able to use the injector version (32bit only AFAIK), or try killing the Steam overlay by removing GameOverlayRenderer.dll

2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD

Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword

Posted 05/16/2015 03:18 PM   
[quote="DarkStarSword"]It could be - I'm not sure what version that was introduced in. Are there any lines starting with CreateRenderTarget (use [RTn] instead of [SFn]) or CreateDepthStencilSurface (use [DSn])?[/quote] Neither. [quote="DarkStarSword"]What's the issue with the latest helix mod and that game (also - remind me which game you were looking at)? If it's a crash on launch issue you might be able to use the injector version (32bit only AFAIK), or try killing the Steam overlay by removing GameOverlayRenderer.dll[/quote] Rome II: Total War. Later HM just don't do anything at all. Game starts as if it's not there. I have the game overlay disabled from within Steam. Can't delete that file though. Steam puts it back when I start the game, even in offline mode, and when the game is running it's apparently in use. :<
DarkStarSword said:It could be - I'm not sure what version that was introduced in. Are there any lines starting with CreateRenderTarget (use [RTn] instead of [SFn]) or CreateDepthStencilSurface (use [DSn])?


Neither.

DarkStarSword said:What's the issue with the latest helix mod and that game (also - remind me which game you were looking at)? If it's a crash on launch issue you might be able to use the injector version (32bit only AFAIK), or try killing the Steam overlay by removing GameOverlayRenderer.dll


Rome II: Total War. Later HM just don't do anything at all. Game starts as if it's not there. I have the game overlay disabled from within Steam. Can't delete that file though. Steam puts it back when I start the game, even in offline mode, and when the game is running it's apparently in use. :<

Posted 05/16/2015 06:38 PM   
One thing I worked out while fixing Euro Truck Simulator 2 is that if Helix Mod isn't engaging at all, you may need to add UseExtInterfaceOnly=true to the DX9Settings.ini. What this setting actually does is intercept the Direct3DCreate9Ex() call, whereas normally Helix Mod only intercepts the Direct3DCreate() call. It's also always worth trying different values for OverrideMethod (0, 1, 2). I'm not entirely sure what these do under the hood, but they can help sometimes (also - in general I find the default OverrideMethod=0 crashes a lot more than 1 or 2).
One thing I worked out while fixing Euro Truck Simulator 2 is that if Helix Mod isn't engaging at all, you may need to add UseExtInterfaceOnly=true to the DX9Settings.ini. What this setting actually does is intercept the Direct3DCreate9Ex() call, whereas normally Helix Mod only intercepts the Direct3DCreate() call.

It's also always worth trying different values for OverrideMethod (0, 1, 2). I'm not entirely sure what these do under the hood, but they can help sometimes (also - in general I find the default OverrideMethod=0 crashes a lot more than 1 or 2).

2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD

Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword

Posted 05/17/2015 05:15 AM   
[quote="DarkStarSword"]One thing I worked out while fixing Euro Truck Simulator 2 is that if Helix Mod isn't engaging at all, you may need to add UseExtInterfaceOnly=true to the DX9Settings.ini. [/quote] Yep, that worked. On to the next issue ;) from the documentation: [/quote] look for lines starting with D3DUSAGE_RENDERTARGET/D3DUSAGE_DEPTHSTENCIL. The value of "n" is an integer from 0 - 9.[/quote] I get entries like so [code]D3DUSAGE_RENDERTARGET/D3DUSAGE_DEPTHSTENCIL Height 1312 Width 2048 Usage 1 Format 21 Pool 0 Levels 1 texptr 0 nIndex 0, Mode 0 [/code] So...........where's "n"?
DarkStarSword said:One thing I worked out while fixing Euro Truck Simulator 2 is that if Helix Mod isn't engaging at all, you may need to add UseExtInterfaceOnly=true to the DX9Settings.ini.


Yep, that worked. On to the next issue ;)

from the documentation:

look for lines starting with D3DUSAGE_RENDERTARGET/D3DUSAGE_DEPTHSTENCIL. The value of "n" is an integer from 0 - 9.

I get entries like so
D3DUSAGE_RENDERTARGET/D3DUSAGE_DEPTHSTENCIL Height 1312 Width 2048 Usage 1 Format 21 Pool 0 Levels 1 texptr 0 nIndex 0, Mode 0


So...........where's "n"?

Posted 05/17/2015 11:50 AM   
Just replace n with whatever number you like, and add it to the SurfaceCreationModeList - it works in the same way as the [KEYn] sections and PresetsKeysList list (including the mandatory semicolon in the list). Edit: Here's some practical examples: https://github.com/DarkStarSword/3d-fixes/commit/ee715abf29f1867d79522de73474b131905f34eb https://github.com/DarkStarSword/3d-fixes/commit/984e22ed5bde1d6b18f3fe729ab124d65c84e63c And a related case where I had to use a [DSn] section & DepthStencilSurfaceModeList to stereoise a Z buffer to fix reflections in Euro Truck Simulator 2: https://github.com/DarkStarSword/3d-fixes/commit/7471b897bee218192444197af5bd2aedf50130c7
Just replace n with whatever number you like, and add it to the SurfaceCreationModeList - it works in the same way as the [KEYn] sections and PresetsKeysList list (including the mandatory semicolon in the list).

Edit: Here's some practical examples:
https://github.com/DarkStarSword/3d-fixes/commit/ee715abf29f1867d79522de73474b131905f34eb
https://github.com/DarkStarSword/3d-fixes/commit/984e22ed5bde1d6b18f3fe729ab124d65c84e63c

And a related case where I had to use a [DSn] section & DepthStencilSurfaceModeList to stereoise a Z buffer to fix reflections in Euro Truck Simulator 2:
https://github.com/DarkStarSword/3d-fixes/commit/7471b897bee218192444197af5bd2aedf50130c7

2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD

Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword

Posted 05/17/2015 02:07 PM   
Thanks! I'll give it a shot.
Thanks! I'll give it a shot.

Posted 05/17/2015 02:34 PM   
@Bo3b - This was the offending bit in my DX9Settings.ini file: DefPSSampler = 0 I changed this to 13 & now I don't run into that problem. @4everAwake - Thanks for the tip. BTW how did you know for sure that this is the shader for shadows? Is there a specific sequence of code that gave you the hint? I worked my way back from r0 to r1 to r3 to r2. Applying stereo correction to r2 renders the shadows in stereo but they are not at the right depth. I haven't tried your exact code because I want to try & solve this bit myself (with help from you guys of course :) )
@Bo3b - This was the offending bit in my DX9Settings.ini file:

DefPSSampler = 0

I changed this to 13 & now I don't run into that problem.

@4everAwake - Thanks for the tip. BTW how did you know for sure that this is the shader for shadows? Is there a specific sequence of code that gave you the hint?

I worked my way back from r0 to r1 to r3 to r2. Applying stereo correction to r2 renders the shadows in stereo but they are not at the right depth. I haven't tried your exact code because I want to try & solve this bit myself (with help from you guys of course :) )

Posted 05/21/2015 10:51 AM   
I got the shadows to render in stereo using this code, but like I said they do not have the correct separation & as a result appear to be at wrong depth. [code] //PS for shadows? ps_3_0 // Converted from ps_2_0 with DarkStarSword's shadertool.py def c199, 0.6, 0, 0.0625, 0.6 dcl_2d s13 //Helix's sampler dcl_2d s0 dcl_2d s1 def c27, 0, 1, 255, 0.00392156979 dcl_texcoord v0.xy dcl_texcoord1 v1.xyz texld r0, v0, s0 mad r0.x, r0, c20, c20.y rcp r0.x, r0.x mul r2.xyz, r0.x, v1 mov r2.w, c27.y //Applying stereo fix to r2 texldl r31, c199.z, s13 //use Helix sampler s0. Now r31.x = Separation, r31.y = Convergence add r31.w, r2.w, -r31.y // (W - Convergence) store in r31.w mul r31.z, r31.x, r31.w // Separation* (W-Convergence) store in r31.z add r2.x, r2.x, r31.z // Xold + Separation*(W-Convergence) dp4 r3.z, r2, c18 dp4 r3.x, r2, c16 dp4 r3.y, r2, c17 dp4 r3.w, r2, c19 add r1.xyz, r3, -c21 add r5.xyz, r3, -c24 dp3 r1.x, r1, r1 add r4.xyz, r3, -c22 dp3 r1.y, r4, r4 add r4.xyz, r3, -c23 dp3 r1.z, r4, r4 dp3 r1.w, r5, r5 add r1, r1, -c25 cmp r1, r1, c27.x, c27.y mov r4.x, r1.y mov r4.y, r1.z mov r4.z, r1.w add_sat r5.xyz, r4, -r1 dp4 r2.z, r3, c6 dp4 r2.x, r3, c4 dp4 r2.y, r3, c5 mul r4.xyz, r5.x, r2 dp4 r2.z, r3, c2 dp4 r2.y, r3, c1 dp4 r2.x, r3, c0 mad r2.xyz, r1.x, r2, r4 dp4 r1.z, r3, c10 dp4 r1.y, r3, c9 dp4 r1.x, r3, c8 mad r1.xyz, r5.y, r1, r2 dp4 r2.y, r3, c13 dp4 r2.x, r3, c12 dp4 r2.z, r3, c14 mad r1.xyz, r2, r5.z, r1 add r2.x, -r0, c27.y mov r0.x, c26 mul r2.xy, r2.x, c27.yzxw frc r2.xy, r2 add r0.x, c27.y, -r0 mov r0.y, c27 texld r1, r1, s1 mad r0.x, r1, r0, c26 mov r1.y, r2 mad r1.x, -r2.y, c27.w, r2 mov r0.w, r1.y mov r0.z, r1.x mov_pp oC0, r0 //draws the shadow // approximately 52 instruction slots used (2 texture, 50 arithmetic) [/code] I fiddled around by multiplying r31.z with a factor to move the shadows to right depth like this but it didn't work. Any ideas what's wrong. [code] //Applying stereo fix to r2 texldl r31, c199.z, s13 //use Helix sampler s0. Now r31.x = Separation, r31.y = Convergence add r31.w, r2.w, -r31.y // (W - Convergence) store in r31.w mul r31.z, r31.x, r31.w // Separation* (W-Convergence) store in r31.z mul r31.z, r31.z, -c199.w add r2.x, r2.x, r31.z // Xold + Separation*(W-Convergence) [/code] EDIT: If I make c199.w sufficiently large (e.g. =20), then the shadow for each eye does separates out, but still not at the right depth. I could fix the shadows for this particular map by experimenting, but I'm afraid the shadows could be broken in some other maps. Shouldn't the texldl command fetch the separation correctly or am I missing something wrt shadows?
I got the shadows to render in stereo using this code, but like I said they do not have the correct separation & as a result appear to be at wrong depth.

//PS for shadows?

ps_3_0 // Converted from ps_2_0 with DarkStarSword's shadertool.py

def c199, 0.6, 0, 0.0625, 0.6
dcl_2d s13 //Helix's sampler

dcl_2d s0
dcl_2d s1
def c27, 0, 1, 255, 0.00392156979
dcl_texcoord v0.xy
dcl_texcoord1 v1.xyz
texld r0, v0, s0
mad r0.x, r0, c20, c20.y
rcp r0.x, r0.x
mul r2.xyz, r0.x, v1
mov r2.w, c27.y


//Applying stereo fix to r2
texldl r31, c199.z, s13 //use Helix sampler s0. Now r31.x = Separation, r31.y = Convergence
add r31.w, r2.w, -r31.y // (W - Convergence) store in r31.w
mul r31.z, r31.x, r31.w // Separation* (W-Convergence) store in r31.z
add r2.x, r2.x, r31.z // Xold + Separation*(W-Convergence)


dp4 r3.z, r2, c18
dp4 r3.x, r2, c16
dp4 r3.y, r2, c17
dp4 r3.w, r2, c19
add r1.xyz, r3, -c21
add r5.xyz, r3, -c24
dp3 r1.x, r1, r1
add r4.xyz, r3, -c22
dp3 r1.y, r4, r4
add r4.xyz, r3, -c23
dp3 r1.z, r4, r4
dp3 r1.w, r5, r5
add r1, r1, -c25
cmp r1, r1, c27.x, c27.y
mov r4.x, r1.y
mov r4.y, r1.z
mov r4.z, r1.w
add_sat r5.xyz, r4, -r1
dp4 r2.z, r3, c6
dp4 r2.x, r3, c4
dp4 r2.y, r3, c5
mul r4.xyz, r5.x, r2
dp4 r2.z, r3, c2
dp4 r2.y, r3, c1
dp4 r2.x, r3, c0
mad r2.xyz, r1.x, r2, r4
dp4 r1.z, r3, c10
dp4 r1.y, r3, c9
dp4 r1.x, r3, c8
mad r1.xyz, r5.y, r1, r2
dp4 r2.y, r3, c13
dp4 r2.x, r3, c12
dp4 r2.z, r3, c14
mad r1.xyz, r2, r5.z, r1
add r2.x, -r0, c27.y
mov r0.x, c26
mul r2.xy, r2.x, c27.yzxw
frc r2.xy, r2
add r0.x, c27.y, -r0
mov r0.y, c27
texld r1, r1, s1
mad r0.x, r1, r0, c26
mov r1.y, r2
mad r1.x, -r2.y, c27.w, r2

mov r0.w, r1.y
mov r0.z, r1.x
mov_pp oC0, r0 //draws the shadow

// approximately 52 instruction slots used (2 texture, 50 arithmetic)


I fiddled around by multiplying r31.z with a factor to move the shadows to right depth like this but it didn't work. Any ideas what's wrong.
//Applying stereo fix to r2
texldl r31, c199.z, s13 //use Helix sampler s0. Now r31.x = Separation, r31.y = Convergence
add r31.w, r2.w, -r31.y // (W - Convergence) store in r31.w
mul r31.z, r31.x, r31.w // Separation* (W-Convergence) store in r31.z
mul r31.z, r31.z, -c199.w
add r2.x, r2.x, r31.z // Xold + Separation*(W-Convergence)


EDIT: If I make c199.w sufficiently large (e.g. =20), then the shadow for each eye does separates out, but still not at the right depth. I could fix the shadows for this particular map by experimenting, but I'm afraid the shadows could be broken in some other maps.

Shouldn't the texldl command fetch the separation correctly or am I missing something wrt shadows?

Posted 05/22/2015 12:30 PM   
[quote="eroc_remag"]I fiddled around by multiplying r31.z with a factor to move the shadows to right depth like this but it didn't work. Any ideas what's wrong. [code] //Applying stereo fix to r2 texldl r31, c199.z, s13 //use Helix sampler s0. Now r31.x = Separation, r31.y = Convergence add r31.w, r2.w, -r31.y // (W - Convergence) store in r31.w mul r31.z, r31.x, r31.w // Separation* (W-Convergence) store in r31.z mul r31.z, r31.z, -c199.w add r2.x, r2.x, r31.z // Xold + Separation*(W-Convergence) [/code] EDIT: If I make c199.w sufficiently large (e.g. =20), then the shadow for each eye does separates out, but still not at the right depth. I could fix the shadows for this particular map by experimenting, but I'm afraid the shadows could be broken in some other maps. Shouldn't the texldl command fetch the separation correctly or am I missing something wrt shadows?[/quote] In this particular shader, depth information is in r2.z, not r2.w.. I don't know why, though. [code]//Applying stereo fix to r2 texldl r31, c199.z, s13 add r31.w, r2.z, -r31.y // r2.w changed to r2.z mul r31.z, r31.x, r31.w mul r31.z, r31.z, -c199.w add r2.x, r2.x, r31.z [/code]
eroc_remag said:I fiddled around by multiplying r31.z with a factor to move the shadows to right depth like this but it didn't work. Any ideas what's wrong.
//Applying stereo fix to r2
texldl r31, c199.z, s13 //use Helix sampler s0. Now r31.x = Separation, r31.y = Convergence
add r31.w, r2.w, -r31.y // (W - Convergence) store in r31.w
mul r31.z, r31.x, r31.w // Separation* (W-Convergence) store in r31.z
mul r31.z, r31.z, -c199.w
add r2.x, r2.x, r31.z // Xold + Separation*(W-Convergence)


EDIT: If I make c199.w sufficiently large (e.g. =20), then the shadow for each eye does separates out, but still not at the right depth. I could fix the shadows for this particular map by experimenting, but I'm afraid the shadows could be broken in some other maps.

Shouldn't the texldl command fetch the separation correctly or am I missing something wrt shadows?


In this particular shader, depth information is in r2.z, not r2.w.. I don't know why, though.

//Applying stereo fix to r2
texldl r31, c199.z, s13
add r31.w, r2.z, -r31.y // r2.w changed to r2.z
mul r31.z, r31.x, r31.w
mul r31.z, r31.z, -c199.w
add r2.x, r2.x, r31.z

Dual boot Win 7 x64 & Win 10 (1809) | Geforce Drivers 417.35

Posted 05/23/2015 12:44 AM   
I just want to say you guys are amazing, and I really appreciate the work everyone is putting into the community. THANK YOU.
I just want to say you guys are amazing, and I really appreciate the work everyone is putting into the community. THANK YOU.

Posted 05/23/2015 01:26 AM   
Hello, I am stucked at this point: [b]Lesson 1 - using HelixMod[/b] [i]Go to the ..\Dumps\SingleShaders\PixelShader\ folder, and look for the file we created. Open that file with the editor, and marvel at the Klingon-esque ASM. [/i] The file has not been created? It is not visible in the PixelShader folder? EDIT: Problem solved, the dumps folder and the log.txt have been created under the Binaries folder. I moved it into Win32 folder and now the folders contain the different shader files.
Hello,

I am stucked at this point:

Lesson 1 - using HelixMod
Go to the ..\Dumps\SingleShaders\PixelShader\ folder, and look for the file we created.
Open that file with the editor, and marvel at the Klingon-esque ASM.


The file has not been created? It is not visible in the PixelShader folder?

EDIT: Problem solved, the dumps folder and the log.txt have been created under the Binaries folder. I moved it into Win32 folder and now the folders contain the different shader files.

Intel Core i7-3820, 4 X 3,60 GHz overclocked to 4,50 GHz ; EVGA Titan X 12VRAM ; 16 GB Corsair Vengeance DDR-1600 (4x 4 GB) ; Asus VG278H 27-inch incl. 3D vision 2 glasses, integrated transmitter ; Xbox One Elite wireless controller ; Windows 10HTC VIVE 2,5 m2 roomscale3D VISION GAMERS - VISIT ME ON STEAM and feel free to add me: http://steamcommunity.com/profiles/76561198064106555 YOUTUBE: https://www.youtube.com/channel/UC1UE5TPoF0HX0HVpF_E4uPQ STEAM CURATOR: https://store.steampowered.com/curator/33611530-Streaming-Deluxe/ Image

Posted 05/27/2015 08:40 PM   
Something went wrong with dumping, nothing that can't be helped with a Skype session. If interested PM me.
Something went wrong with dumping, nothing that can't be helped with a Skype session. If interested PM me.

Thanks to everybody using my assembler it warms my heart.
To have a critical piece of code that everyone can enjoy!
What more can you ask for?

donations: ulfjalmbrant@hotmail.com

Posted 05/27/2015 09:07 PM   
  35 / 87    
Scroll To Top