ok, that makes sense :)
To be clear the code following the fix that is not out-dented is from the original shader, and I just left it there and commented it to provide some context so you can find the correct spot for the fix in other shaders.
To be clear the code following the fix that is not out-dented is from the original shader, and I just left it there and commented it to provide some context so you can find the correct spot for the fix in other shaders.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
[quote="DarkStarSword"]Glad you were able to get it working :)
That's a pretty interesting note about which constants worked and which didn't - I wonder if the limit only applies to constants passed to the shader from the game/Helix mod and not constants defined locally in the shader?[/quote]
I've run into a strange problem. If I disable PS 4C7711F0, the ghosting round the trees is fixed. But if I set it to output 0000 or 1111 it doesn't work. Why is that so? I'll try manipulating the output location for this shader.
http://api.photos.3dvisionlive.com/imagestore/54f819abe7e564f95f0000c5/thumb.square/172/
http://api.photos.3dvisionlive.com/imagestore/54f81a6fe7e5649f630000df/thumb.square/172/
DarkStarSword said:Glad you were able to get it working :)
That's a pretty interesting note about which constants worked and which didn't - I wonder if the limit only applies to constants passed to the shader from the game/Helix mod and not constants defined locally in the shader?
I've run into a strange problem. If I disable PS 4C7711F0, the ghosting round the trees is fixed. But if I set it to output 0000 or 1111 it doesn't work. Why is that so? I'll try manipulating the output location for this shader.
Found a possible VS candidate. If I disable this shader, the effect is removed from the conifers, but not the tree right at the back. So looks like the effect for the tree is done by a different shader.
[quote="eroc_remag"]I've run into a strange problem. If I disable PS 4C7711F0, the ghosting round the trees is fixed. But if I set it to output 0000 or 1111 it doesn't work. Why is that so? I'll try manipulating the output location for this shader.
[/quote]
I've also found it's worth trying 1110 and 0001 to disable effects in the pixel shader - it depends largely on how their result is being used by the game. There are two other ways you can try to disable a pixel shader if you can't find any output that works:
- Comment out everything in the shader after the ps_3_0 line
- Use the texkill instruction, which cancels the current pixel render if you pass it a temporary register with negative value in any of the first three components, e.g.
[code]
def c220, 0, 1, 0.0625, -1
...
// texkill does not work with constant registers, so set r30 to -1 and use it:
mov r30, c220.wwww
texkill r30
[/code]
The texkill instruction is documented here: https://msdn.microsoft.com/en-us/library/windows/desktop/bb206216(v=vs.85).aspx
[quote="eroc_remag"]I'm not sure where to start on this shader though. I tried setting o0 to 0000 but nothing happens when I hit F10.
[code]
...
mad o0.zw, v3.xyxy, c16.xyxy, c16
mad o0.xy, v3, c15, c15.zwzw
mov o0.xyzw, c200.wwww
[/code][/quote]When you disable a vertex shader, you need to check which output is the position:
[code]dcl_position o4[/code]
In this case it's o4 and not o0 (In this case it's my script that numbered them this way after upgrading it to shader model 3 - I originally made it do that so the output texcoord numbers matched the register numbers, but I'm thinking of changing it to always use o0 for the position).
Looking at your screenshots I think the leaves of the tree at the back might have a second halo that you will need to deal with separately, but I'm not certain. The screenshot with the blue background definitely looks like a halo type issue on the fog in the background.
What game (or editor?) is this? I notice the shader you posted doesn't have headers which can sometimes provide clues
eroc_remag said:I've run into a strange problem. If I disable PS 4C7711F0, the ghosting round the trees is fixed. But if I set it to output 0000 or 1111 it doesn't work. Why is that so? I'll try manipulating the output location for this shader.
I've also found it's worth trying 1110 and 0001 to disable effects in the pixel shader - it depends largely on how their result is being used by the game. There are two other ways you can try to disable a pixel shader if you can't find any output that works:
- Comment out everything in the shader after the ps_3_0 line
- Use the texkill instruction, which cancels the current pixel render if you pass it a temporary register with negative value in any of the first three components, e.g.
def c220, 0, 1, 0.0625, -1
...
// texkill does not work with constant registers, so set r30 to -1 and use it:
mov r30, c220.wwww
texkill r30
When you disable a vertex shader, you need to check which output is the position:
dcl_position o4
In this case it's o4 and not o0 (In this case it's my script that numbered them this way after upgrading it to shader model 3 - I originally made it do that so the output texcoord numbers matched the register numbers, but I'm thinking of changing it to always use o0 for the position).
Looking at your screenshots I think the leaves of the tree at the back might have a second halo that you will need to deal with separately, but I'm not certain. The screenshot with the blue background definitely looks like a halo type issue on the fog in the background.
What game (or editor?) is this? I notice the shader you posted doesn't have headers which can sometimes provide clues
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
[quote="DarkStarSword"]ok, that makes sense :)
To be clear the code following the fix that is not out-dented is from the original shader, and I just left it there and commented it to provide some context so you can find the correct spot for the fix in other shaders.[/quote]
Ok, well I've been an ass. The penny finally dropped though. I went looking for a shader that had the "overlay" line in the header and discovered there was only one that had any impact on the campaign map. (i.e. the shaders that disabled separate effects had no such line in them).
I'd already jotted that down as "turns the whole land surface white" and..........wait for it.
1) It matches the structure of the Mike's fix you quoted and (it gets worse)
2) I'd already repaired the section relating to shadows. Just completely overlooked the border and river bit. (insert facepalm smiley of your choice here).
Anyway, in the end all I had to do was copy mike's fix and change an "r4" to "r5".
Ok, I'll go hide in a corner now :D
I'll go over your post again though, because it's one thing to know what the fix is, another how/why it works.
In terms of getting this game playable this was an important fix.
To be clear the code following the fix that is not out-dented is from the original shader, and I just left it there and commented it to provide some context so you can find the correct spot for the fix in other shaders.
Ok, well I've been an ass. The penny finally dropped though. I went looking for a shader that had the "overlay" line in the header and discovered there was only one that had any impact on the campaign map. (i.e. the shaders that disabled separate effects had no such line in them).
I'd already jotted that down as "turns the whole land surface white" and..........wait for it.
1) It matches the structure of the Mike's fix you quoted and (it gets worse)
2) I'd already repaired the section relating to shadows. Just completely overlooked the border and river bit. (insert facepalm smiley of your choice here).
Anyway, in the end all I had to do was copy mike's fix and change an "r4" to "r5".
Ok, I'll go hide in a corner now :D
I'll go over your post again though, because it's one thing to know what the fix is, another how/why it works.
In terms of getting this game playable this was an important fix.
Looks like I need help again.
This time I'm having a hard time finding the shader.
It's once again a 2D texture projected on the ground, but this one wasn't fixed yet. The projection isn't a simple overlay this time (I think). There's a shader I can disable that will remove the texture, but when I do, a "box" of sorts appears around it.
It's visible, because it's a shade lighter than the surroundings. However, once you know it's there, it becomes apparent that (with the shader active) the texture I refer to is only ever visible to the extent it falls within the outline of this box. Zooming in,out and rotating the image shows the texture is being cut off outside the bounds. Perhaps it wouldn't be if the stereo correction was ok.
Last time I had to look for a shader which referred to an "overlay projection". Does the above give any clue as to what kind of projecting I should be looking for this time?
P.S. I should add mike's fix does include this shader, but says it fixes a lighting issue. I tried the fix, but it doesn't appear to have any impact at all.
This time I'm having a hard time finding the shader.
It's once again a 2D texture projected on the ground, but this one wasn't fixed yet. The projection isn't a simple overlay this time (I think). There's a shader I can disable that will remove the texture, but when I do, a "box" of sorts appears around it.
It's visible, because it's a shade lighter than the surroundings. However, once you know it's there, it becomes apparent that (with the shader active) the texture I refer to is only ever visible to the extent it falls within the outline of this box. Zooming in,out and rotating the image shows the texture is being cut off outside the bounds. Perhaps it wouldn't be if the stereo correction was ok.
Last time I had to look for a shader which referred to an "overlay projection". Does the above give any clue as to what kind of projecting I should be looking for this time?
P.S. I should add mike's fix does include this shader, but says it fixes a lighting issue. I tried the fix, but it doesn't appear to have any impact at all.
[quote="Muizer"]It's once again a 2D texture projected on the ground, but this one wasn't fixed yet. The projection isn't a simple overlay this time (I think). There's a shader I can disable that will remove the texture, but when I do, a "box" of sorts appears around it.
It's visible, because it's a shade lighter than the surroundings. However, once you know it's there, it becomes apparent that (with the shader active) the texture I refer to is only ever visible to the extent it falls within the outline of this box. Zooming in,out and rotating the image shows the texture is being cut off outside the bounds. Perhaps it wouldn't be if the stereo correction was ok.[/quote]
Your description of the box makes me suspect that it is some kind of post-processing effect that is being limited to part of the screen, either due to scissor clipping (common and easy to disable) or the vertices only covering part of the screen (moderate to impossible to fix).
First, see what happend if you add this option to the [General] section in DX9Settings.ini:
[code]SkipSetScissorRect = true[/code]
One trick I do in these cases to help me understand what is happening, is to set the pixel shader to output full white (or if there are several overlapping effects I sometimes use a semi-transparent colour) so I can clearly see where the bounds of the effect are (the box you described) - that can help me to work out if the bounds are positioned to correctly cover the whole effect in each eye, or if I might need to adjust the position of the vertex shader to eliminate clipping.
Be aware that if you adjust the output position of the vertex shader you may make the effect worse unless you also adjust any relevant texcoords, so it's probably better to try to get it to render correctly first, then try to fix the clipping afterwards.
Post-processing effects usually need to be fixed in the pixel shaders (but not always, and sometimes they need more complicated fixes in both shaders). Look for any matrix transformations and texture samplers and see what changes if you add a rough adjustment before or after the transform/sample. I always recommend trying to adjust any texcoord outputs of the vertex shader as well - even if they aren't right the first step is to try to learn as much as you can about what parts of the shaders are relevant to the effect.
I can't really guess at a pattern to look for without knowing more details about the effect - would you mind posting screenshots of the broken effect, the box you see when disabling it and any vertex and pixel shaders you suspect may be related.
Muizer said:It's once again a 2D texture projected on the ground, but this one wasn't fixed yet. The projection isn't a simple overlay this time (I think). There's a shader I can disable that will remove the texture, but when I do, a "box" of sorts appears around it.
It's visible, because it's a shade lighter than the surroundings. However, once you know it's there, it becomes apparent that (with the shader active) the texture I refer to is only ever visible to the extent it falls within the outline of this box. Zooming in,out and rotating the image shows the texture is being cut off outside the bounds. Perhaps it wouldn't be if the stereo correction was ok.
Your description of the box makes me suspect that it is some kind of post-processing effect that is being limited to part of the screen, either due to scissor clipping (common and easy to disable) or the vertices only covering part of the screen (moderate to impossible to fix).
First, see what happend if you add this option to the [General] section in DX9Settings.ini:
SkipSetScissorRect = true
One trick I do in these cases to help me understand what is happening, is to set the pixel shader to output full white (or if there are several overlapping effects I sometimes use a semi-transparent colour) so I can clearly see where the bounds of the effect are (the box you described) - that can help me to work out if the bounds are positioned to correctly cover the whole effect in each eye, or if I might need to adjust the position of the vertex shader to eliminate clipping.
Be aware that if you adjust the output position of the vertex shader you may make the effect worse unless you also adjust any relevant texcoords, so it's probably better to try to get it to render correctly first, then try to fix the clipping afterwards.
Post-processing effects usually need to be fixed in the pixel shaders (but not always, and sometimes they need more complicated fixes in both shaders). Look for any matrix transformations and texture samplers and see what changes if you add a rough adjustment before or after the transform/sample. I always recommend trying to adjust any texcoord outputs of the vertex shader as well - even if they aren't right the first step is to try to learn as much as you can about what parts of the shaders are relevant to the effect.
I can't really guess at a pattern to look for without knowing more details about the effect - would you mind posting screenshots of the broken effect, the box you see when disabling it and any vertex and pixel shaders you suspect may be related.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
// approximately 167 instruction slots used (10 texture, 157 arithmetic)
To be clear, the effect I'd like to fix is the ground texture under the vilage. It has the left image displayed on the right and vice versa, relative to the camera (not the world).
// approximately 167 instruction slots used (10 texture, 157 arithmetic)
Says it only concerns a lighting issue.
When I disable the following vertex shader the ground surface disappears, but there's no other visbile effect. There's no fix in Mike's package for this shader:
Interestingly, when I disable both the vertex and the pixel shader, this is what happens:
EDIT: I haven't looked into the role of texcoords at all so far. The main question I have atm is if these shaders are the right place to look for a fix at all, or whether it too has an "overlay" somewhere else where I need to be, as was the case for borders and rivers.
ok, forget what I said about scissor clipping - it's clearly not that from the screenshots (scissor clipping occurs as a 2D rectangle at screen depth, while that box is clearly 3D and correctly positioned).
I noticed in Mike's fix he is using matrices in c200-c203 and c204-c207, which are not in the headers. He is using a section like this to take the inv_view_projection matrix from c0-c3 and invert it so it becomes a regular view-projection matrix in c200-c203, then inverting it yet again into c204-c207 (the later will be approximately the same as the original matrix, but may be more numerically stable):
[code]
[PSFCC0E468]
UseMatrix = true
GetMatrixFromReg = 0
MatrixReg = 200
InverseMatrix = true
DoubleInverseMatrix = true
[/code]
Make sure you have that in your DX9Settings.ini file (replace the hash with whichever shader you are looking at).
Given the box you see when it's disabled I think you might be in the right place, so it probably is worth experimenting with these shaders to see if it is possible to move or disable the effect... but it's hard to say for certain, and I don't see any obvious clues in the shaders.
My gut feeling is that this won't be another overlay like before, but I can't really be certain about that.
[quote] Interestingly, when I disable both the vertex and the pixel shader, this is what happens:[/quote]Yep, if this is a light/shadow shader that would make sense - disabling it can kill the light source so the map is displayed as though the whole thing is in shadow.
ok, forget what I said about scissor clipping - it's clearly not that from the screenshots (scissor clipping occurs as a 2D rectangle at screen depth, while that box is clearly 3D and correctly positioned).
I noticed in Mike's fix he is using matrices in c200-c203 and c204-c207, which are not in the headers. He is using a section like this to take the inv_view_projection matrix from c0-c3 and invert it so it becomes a regular view-projection matrix in c200-c203, then inverting it yet again into c204-c207 (the later will be approximately the same as the original matrix, but may be more numerically stable):
Make sure you have that in your DX9Settings.ini file (replace the hash with whichever shader you are looking at).
Given the box you see when it's disabled I think you might be in the right place, so it probably is worth experimenting with these shaders to see if it is possible to move or disable the effect... but it's hard to say for certain, and I don't see any obvious clues in the shaders.
My gut feeling is that this won't be another overlay like before, but I can't really be certain about that.
Interestingly, when I disable both the vertex and the pixel shader, this is what happens:
Yep, if this is a light/shadow shader that would make sense - disabling it can kill the light source so the map is displayed as though the whole thing is in shadow.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
That worked! So what this entry in the ini instructs is to get the matrix from registry 0 (inv_view_projection), put the inverse and double inverse matrices into registries c200 and onwards?
A couple of things not quite clear yet.
- the use of "rcp". Is this something to do with how matrix calculations work?
- the fact that the fix here substracts from x rather than adds to it.
That worked! So what this entry in the ini instructs is to get the matrix from registry 0 (inv_view_projection), put the inverse and double inverse matrices into registries c200 and onwards?
A couple of things not quite clear yet.
- the use of "rcp". Is this something to do with how matrix calculations work?
- the fact that the fix here substracts from x rather than adds to it.
[quote="Muizer"]That worked! So what this entry in the ini instructs is to get the matrix from registry 0 (inv_view_projection), put the inverse and double inverse matrices into registries c200 and onwards?[/quote]Exactly ;-)
[quote]A couple of things not quite clear yet.
- the use of "rcp". Is this something to do with how matrix calculations work?
[/quote]rcp is reciprocal (1/value) and it's the way to do divisions in shader assembly.
Looking at the fix Mike is using it to normalise a four dimensional homogeneous coordinate into the equivalent three dimensional cartesian coordinate before applying the fix, then scaling it back to the original afterwards. I'm not entirely clear on the maths as to why this is necessary for this particular fix either (and I just made a mental note to try this in future), but I would note that the original shader does the same thing further down if you look for where it uses r0.x.
Projection matrices cause a coordinate to not be normalised (you may have heard the term "perspective divide", which is the normalisation process that turns the result of a projection into screen coordinates), so perhaps the same can happen with an inverse projection? As I said I'm not 100% on the maths here, so I'm not certain.
[quote]- the fact that the fix here substracts from x rather than adds to it.[/quote]This is pretty common when correcting a world-space coordinate. My understanding is that the original coordinate will have already been adjusted for stereo by the driver, and when the game un-projects that coordinate to find the original world coordinates the correction needs to be subtracted so that it gets the real original coordinate, and not some skewed version.
Muizer said:That worked! So what this entry in the ini instructs is to get the matrix from registry 0 (inv_view_projection), put the inverse and double inverse matrices into registries c200 and onwards?
Exactly ;-)
A couple of things not quite clear yet.
- the use of "rcp". Is this something to do with how matrix calculations work?
rcp is reciprocal (1/value) and it's the way to do divisions in shader assembly.
Looking at the fix Mike is using it to normalise a four dimensional homogeneous coordinate into the equivalent three dimensional cartesian coordinate before applying the fix, then scaling it back to the original afterwards. I'm not entirely clear on the maths as to why this is necessary for this particular fix either (and I just made a mental note to try this in future), but I would note that the original shader does the same thing further down if you look for where it uses r0.x.
Projection matrices cause a coordinate to not be normalised (you may have heard the term "perspective divide", which is the normalisation process that turns the result of a projection into screen coordinates), so perhaps the same can happen with an inverse projection? As I said I'm not 100% on the maths here, so I'm not certain.
- the fact that the fix here substracts from x rather than adds to it.
This is pretty common when correcting a world-space coordinate. My understanding is that the original coordinate will have already been adjusted for stereo by the driver, and when the game un-projects that coordinate to find the original world coordinates the correction needs to be subtracted so that it gets the real original coordinate, and not some skewed version.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
[quote="DarkStarSword"]
I've also found it's worth trying 1110 and 0001 to disable effects in the pixel shader - it depends largely on how their result is being used by the game. There are two other ways you can try to disable a pixel shader if you can't find any output that works:
- Comment out everything in the shader after the ps_3_0 line
- Use the texkill instruction, which cancels the current pixel render if you pass it a temporary register with negative value in any of the first three components, e.g.
[/quote]
1110 & 0001 did not fix the effect. I tried commenting out all the lines after ps_3_0 line but that disables the tree trunks. The same thing happens when I use texkill command.
DarkStarSword said:
I've also found it's worth trying 1110 and 0001 to disable effects in the pixel shader - it depends largely on how their result is being used by the game. There are two other ways you can try to disable a pixel shader if you can't find any output that works:
- Comment out everything in the shader after the ps_3_0 line
- Use the texkill instruction, which cancels the current pixel render if you pass it a temporary register with negative value in any of the first three components, e.g.
1110 & 0001 did not fix the effect. I tried commenting out all the lines after ps_3_0 line but that disables the tree trunks. The same thing happens when I use texkill command.
I'm just going through lesson 1. There's a bit of an oddity thatI don't quite understand.
The lesson refers to LOG.txt and a Dumps folder in The Ball\Binaries\Win32.
However, even after running The Ball with helixmod a few times, cycling through shaders and all that, I didn't have those in those locations. I had both LOG.TXT and the Dumps folder in The Ball\ (ie. the root folder). The Dumps folder had the AllShaders and SingleShaders folder inside, but they were both empty.
It seems that the problem might be because I was running the Ball by using the Windows 7 "recent" right-click menu: I would right-click the Steam icon in the taskbar, and then select The Ball from the list of recently opened games.
On a hunch, I tried double-clicking TheBall.exe directly from within The Ball\Binaries\Win32, and immediately LOG.txt and Dumps were created in The Ball\Binaries\Win32 as they were supposed to. And this time, the AllShaders and Singleshaders folders actually had files in them.
Has anyone had this happen to them as well?
I'm just going through lesson 1. There's a bit of an oddity thatI don't quite understand.
The lesson refers to LOG.txt and a Dumps folder in The Ball\Binaries\Win32.
However, even after running The Ball with helixmod a few times, cycling through shaders and all that, I didn't have those in those locations. I had both LOG.TXT and the Dumps folder in The Ball\ (ie. the root folder). The Dumps folder had the AllShaders and SingleShaders folder inside, but they were both empty.
It seems that the problem might be because I was running the Ball by using the Windows 7 "recent" right-click menu: I would right-click the Steam icon in the taskbar, and then select The Ball from the list of recently opened games.
On a hunch, I tried double-clicking TheBall.exe directly from within The Ball\Binaries\Win32, and immediately LOG.txt and Dumps were created in The Ball\Binaries\Win32 as they were supposed to. And this time, the AllShaders and Singleshaders folders actually had files in them.
To be clear the code following the fix that is not out-dented is from the original shader, and I just left it there and commented it to provide some context so you can find the correct spot for the fix in other shaders.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
I've run into a strange problem. If I disable PS 4C7711F0, the ghosting round the trees is fixed. But if I set it to output 0000 or 1111 it doesn't work. Why is that so? I'll try manipulating the output location for this shader.
http://api.photos.3dvisionlive.com/imagestore/54f819abe7e564f95f0000c5/thumb.square/172/
http://api.photos.3dvisionlive.com/imagestore/54f81a6fe7e5649f630000df/thumb.square/172/
http://api.photos.3dvisionlive.com/imagestore/54f82362e7e564796e0001ce/thumb.square/172/
I'm not sure where to start on this shader though. I tried setting o0 to 0000 but nothing happens when I hit F10.
I've also found it's worth trying 1110 and 0001 to disable effects in the pixel shader - it depends largely on how their result is being used by the game. There are two other ways you can try to disable a pixel shader if you can't find any output that works:
- Comment out everything in the shader after the ps_3_0 line
- Use the texkill instruction, which cancels the current pixel render if you pass it a temporary register with negative value in any of the first three components, e.g.
The texkill instruction is documented here: https://msdn.microsoft.com/en-us/library/windows/desktop/bb206216(v=vs.85).aspx
When you disable a vertex shader, you need to check which output is the position:
In this case it's o4 and not o0 (In this case it's my script that numbered them this way after upgrading it to shader model 3 - I originally made it do that so the output texcoord numbers matched the register numbers, but I'm thinking of changing it to always use o0 for the position).
Looking at your screenshots I think the leaves of the tree at the back might have a second halo that you will need to deal with separately, but I'm not certain. The screenshot with the blue background definitely looks like a halo type issue on the fog in the background.
What game (or editor?) is this? I notice the shader you posted doesn't have headers which can sometimes provide clues
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
Ok, well I've been an ass. The penny finally dropped though. I went looking for a shader that had the "overlay" line in the header and discovered there was only one that had any impact on the campaign map. (i.e. the shaders that disabled separate effects had no such line in them).
I'd already jotted that down as "turns the whole land surface white" and..........wait for it.
1) It matches the structure of the Mike's fix you quoted and (it gets worse)
2) I'd already repaired the section relating to shadows. Just completely overlooked the border and river bit. (insert facepalm smiley of your choice here).
Anyway, in the end all I had to do was copy mike's fix and change an "r4" to "r5".
Ok, I'll go hide in a corner now :D
I'll go over your post again though, because it's one thing to know what the fix is, another how/why it works.
In terms of getting this game playable this was an important fix.
This time I'm having a hard time finding the shader.
It's once again a 2D texture projected on the ground, but this one wasn't fixed yet. The projection isn't a simple overlay this time (I think). There's a shader I can disable that will remove the texture, but when I do, a "box" of sorts appears around it.
It's visible, because it's a shade lighter than the surroundings. However, once you know it's there, it becomes apparent that (with the shader active) the texture I refer to is only ever visible to the extent it falls within the outline of this box. Zooming in,out and rotating the image shows the texture is being cut off outside the bounds. Perhaps it wouldn't be if the stereo correction was ok.
Last time I had to look for a shader which referred to an "overlay projection". Does the above give any clue as to what kind of projecting I should be looking for this time?
P.S. I should add mike's fix does include this shader, but says it fixes a lighting issue. I tried the fix, but it doesn't appear to have any impact at all.
Your description of the box makes me suspect that it is some kind of post-processing effect that is being limited to part of the screen, either due to scissor clipping (common and easy to disable) or the vertices only covering part of the screen (moderate to impossible to fix).
First, see what happend if you add this option to the [General] section in DX9Settings.ini:
One trick I do in these cases to help me understand what is happening, is to set the pixel shader to output full white (or if there are several overlapping effects I sometimes use a semi-transparent colour) so I can clearly see where the bounds of the effect are (the box you described) - that can help me to work out if the bounds are positioned to correctly cover the whole effect in each eye, or if I might need to adjust the position of the vertex shader to eliminate clipping.
Be aware that if you adjust the output position of the vertex shader you may make the effect worse unless you also adjust any relevant texcoords, so it's probably better to try to get it to render correctly first, then try to fix the clipping afterwards.
Post-processing effects usually need to be fixed in the pixel shaders (but not always, and sometimes they need more complicated fixes in both shaders). Look for any matrix transformations and texture samplers and see what changes if you add a rough adjustment before or after the transform/sample. I always recommend trying to adjust any texcoord outputs of the vertex shader as well - even if they aren't right the first step is to try to learn as much as you can about what parts of the shaders are relevant to the effect.
I can't really guess at a pattern to look for without knowing more details about the effect - would you mind posting screenshots of the broken effect, the box you see when disabling it and any vertex and pixel shaders you suspect may be related.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
With the ps enabled
Shader code
To be clear, the effect I'd like to fix is the ground texture under the vilage. It has the left image displayed on the right and vice versa, relative to the camera (not the world).
Mike's fix for this shader
Says it only concerns a lighting issue.
When I disable the following vertex shader the ground surface disappears, but there's no other visbile effect. There's no fix in Mike's package for this shader:
Interestingly, when I disable both the vertex and the pixel shader, this is what happens:
EDIT: I haven't looked into the role of texcoords at all so far. The main question I have atm is if these shaders are the right place to look for a fix at all, or whether it too has an "overlay" somewhere else where I need to be, as was the case for borders and rivers.
I noticed in Mike's fix he is using matrices in c200-c203 and c204-c207, which are not in the headers. He is using a section like this to take the inv_view_projection matrix from c0-c3 and invert it so it becomes a regular view-projection matrix in c200-c203, then inverting it yet again into c204-c207 (the later will be approximately the same as the original matrix, but may be more numerically stable):
Make sure you have that in your DX9Settings.ini file (replace the hash with whichever shader you are looking at).
Given the box you see when it's disabled I think you might be in the right place, so it probably is worth experimenting with these shaders to see if it is possible to move or disable the effect... but it's hard to say for certain, and I don't see any obvious clues in the shaders.
My gut feeling is that this won't be another overlay like before, but I can't really be certain about that.
Yep, if this is a light/shadow shader that would make sense - disabling it can kill the light source so the map is displayed as though the whole thing is in shadow.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
A couple of things not quite clear yet.
- the use of "rcp". Is this something to do with how matrix calculations work?
- the fact that the fix here substracts from x rather than adds to it.
rcp is reciprocal (1/value) and it's the way to do divisions in shader assembly.
Looking at the fix Mike is using it to normalise a four dimensional homogeneous coordinate into the equivalent three dimensional cartesian coordinate before applying the fix, then scaling it back to the original afterwards. I'm not entirely clear on the maths as to why this is necessary for this particular fix either (and I just made a mental note to try this in future), but I would note that the original shader does the same thing further down if you look for where it uses r0.x.
Projection matrices cause a coordinate to not be normalised (you may have heard the term "perspective divide", which is the normalisation process that turns the result of a projection into screen coordinates), so perhaps the same can happen with an inverse projection? As I said I'm not 100% on the maths here, so I'm not certain.
This is pretty common when correcting a world-space coordinate. My understanding is that the original coordinate will have already been adjusted for stereo by the driver, and when the game un-projects that coordinate to find the original world coordinates the correction needs to be subtracted so that it gets the real original coordinate, and not some skewed version.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
1110 & 0001 did not fix the effect. I tried commenting out all the lines after ps_3_0 line but that disables the tree trunks. The same thing happens when I use texkill command.
The lesson refers to LOG.txt and a Dumps folder in The Ball\Binaries\Win32.
However, even after running The Ball with helixmod a few times, cycling through shaders and all that, I didn't have those in those locations. I had both LOG.TXT and the Dumps folder in The Ball\ (ie. the root folder). The Dumps folder had the AllShaders and SingleShaders folder inside, but they were both empty.
It seems that the problem might be because I was running the Ball by using the Windows 7 "recent" right-click menu: I would right-click the Steam icon in the taskbar, and then select The Ball from the list of recently opened games.
On a hunch, I tried double-clicking TheBall.exe directly from within The Ball\Binaries\Win32, and immediately LOG.txt and Dumps were created in The Ball\Binaries\Win32 as they were supposed to. And this time, the AllShaders and Singleshaders folders actually had files in them.
Has anyone had this happen to them as well?