Well, I'm kind of stuck I'm afraid. The things I want address just don't seem to have the proper geometry for the fix to work with. While land textures were made to drape over the height map, certain other elements like borders and rivers are set to appear correctly only at sea level (and get messed up on land).
I don't suppose it's possible to get these shaders to use the input the land texture shaders use?
Well, I'm kind of stuck I'm afraid. The things I want address just don't seem to have the proper geometry for the fix to work with. While land textures were made to drape over the height map, certain other elements like borders and rivers are set to appear correctly only at sea level (and get messed up on land).
I don't suppose it's possible to get these shaders to use the input the land texture shaders use?
That does happen from time to time unfortunately where the information you need just isn't available :(
You might need to disable the effect if it won't be missed, approximate it if there is no other option or look for ways to use the information you do have to make up for the information you are missing, or like you suggested - in some cases you might be able to copy the necessary information from a different shader.
Can you post the complete unmodified vertex & pixel shaders for the effects and some 3D screenshots showing how they are broken and we might be able to provide some suggestions - I'd really want to see what you are working with before suggesting anything, particularly for the water as that might involve reflections or other advanced transformations.
For copying inputs from one shader to another - this is an advanced topic, but it is possible with some limitations. For instance, you can't copy the vertex inputs, but you can copy constants and matrices from one shader to another, which can sometimes be enough. You may also have problems if there are too many different instances of the shader you are trying to copy an input from as you never know which one the copy will be from (e.g. enemy health bars, or bloom/halos around lights).
I actually did something along these lines just the other day to fix the ripple distortion on the spirit rings in Oddworld, which were being drawn at screen depth:
[img]https://forums.geforce.com/cmd/default/download-comment-attachment/63658/[/img]
For that particular effect, I copied the 4th column of the model-view-projection matrix for the red rings into the ripple post-processing shader and used it to calculate the depth at the origin (0,0,0,1) of the rings to align the ripples to them - here's the main commit that made this change (later commits polished this a bit, but this should be good enough to demonstrate the concept):
https://github.com/DarkStarSword/3d-fixes/commit/bd87a8f14866a05bcbf242d8c878c7cb6c4afc7a
A limitation here is that all the ripples would be aligned to the spirit ring depth, not just the ones from the spirit rings themselves. In this case that didn't matter since that depth would be approximately correct and most of the ripples were from explosions which were fast enough that any slight misalignment would not be noticed.
That does happen from time to time unfortunately where the information you need just isn't available :(
You might need to disable the effect if it won't be missed, approximate it if there is no other option or look for ways to use the information you do have to make up for the information you are missing, or like you suggested - in some cases you might be able to copy the necessary information from a different shader.
Can you post the complete unmodified vertex & pixel shaders for the effects and some 3D screenshots showing how they are broken and we might be able to provide some suggestions - I'd really want to see what you are working with before suggesting anything, particularly for the water as that might involve reflections or other advanced transformations.
For copying inputs from one shader to another - this is an advanced topic, but it is possible with some limitations. For instance, you can't copy the vertex inputs, but you can copy constants and matrices from one shader to another, which can sometimes be enough. You may also have problems if there are too many different instances of the shader you are trying to copy an input from as you never know which one the copy will be from (e.g. enemy health bars, or bloom/halos around lights).
I actually did something along these lines just the other day to fix the ripple distortion on the spirit rings in Oddworld, which were being drawn at screen depth:
For that particular effect, I copied the 4th column of the model-view-projection matrix for the red rings into the ripple post-processing shader and used it to calculate the depth at the origin (0,0,0,1) of the rings to align the ripples to them - here's the main commit that made this change (later commits polished this a bit, but this should be good enough to demonstrate the concept):
https://github.com/DarkStarSword/3d-fixes/commit/bd87a8f14866a05bcbf242d8c878c7cb6c4afc7a
A limitation here is that all the ripples would be aligned to the spirit ring depth, not just the ones from the spirit rings themselves. In this case that didn't matter since that depth would be approximately correct and most of the ripples were from explosions which were fast enough that any slight misalignment would not be noticed.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
[quote="eroc_remag"]Here's my dx9settings.ini file. This too is a copy of a known working version.
[code]
// Constant registers that will arrive in Vertex and Pixel Shaders, as
// c220. You can define any constant register as long as it is not used in any other shader in the game. Check for the desired constant register in the Dumps folder to find out if any other shander uses it. The constants below will be assigned, based on the key preset.
DefVSConst1 = 220
DefPSConst1 = 220
[/code]
[/quote]
This looks ok as well. You might try a lower number for DefPSConst1 (and change the code in the shader to match) - in theory you can use up to 223 for pixel shaders, but it seems that some games somehow reduce the maximum available constant registers so sometimes higher numbers don't work. Since you already know c200 works inside the shader you might try a number below that, like 190.
eroc_remag said:Here's my dx9settings.ini file. This too is a copy of a known working version.
// Constant registers that will arrive in Vertex and Pixel Shaders, as
// c220. You can define any constant register as long as it is not used in any other shader in the game. Check for the desired constant register in the Dumps folder to find out if any other shander uses it. The constants below will be assigned, based on the key preset.
DefVSConst1 = 220
DefPSConst1 = 220
This looks ok as well. You might try a lower number for DefPSConst1 (and change the code in the shader to match) - in theory you can use up to 223 for pixel shaders, but it seems that some games somehow reduce the maximum available constant registers so sometimes higher numbers don't work. Since you already know c200 works inside the shader you might try a number below that, like 190.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Ahh, now I've looked at the shader and ini file together I can see the problem:
[quote="eroc_remag"][code]
...
//disable the effect by setting output color register oC0 to 1111.
mov r29.x, c220.x
if_eq r29.x, c200.x
mov oC0.xyzw, c200.yyyy //0 doesn't work. 1 fixes effect but breaks tree trunk
endif
[/code]
...
[code]
[PRES1]
Const2 = 0x3f800000
[PRES2]
Const2 = 0x00000000
UseByDef=True
[/code]
[/quote]
The key bindings are setting Const2, but you are testing Const1 (c220.x) in the shader - it should be c220.y for Const2.
I tried setting c190 as you suggested, but even that did not work. I then searched for lower values that are not used by the game & found c49 to be the lowest one. When I used c49 the toggle worked! This goes into my little tutorial document then.
I don't understand why c200 works but not c190.
I tried setting c190 as you suggested, but even that did not work. I then searched for lower values that are not used by the game & found c49 to be the lowest one. When I used c49 the toggle worked! This goes into my little tutorial document then.
[quote="DarkStarSword"]Ahh, now I've looked at the shader and ini file together I can see the problem:
The key bindings are setting Const2, but you are testing Const1 (c220.x) in the shader - it should be c220.y for Const2.[/quote]
Thanks, I corrected that. But with everything else being the same, if I change c49 to c190 in .ini & the shader file, the toggle doesn't work.
Glad you were able to get it working :)
That's a pretty interesting note about which constants worked and which didn't - I wonder if the limit only applies to constants passed to the shader from the game/Helix mod and not constants defined locally in the shader?
That's a pretty interesting note about which constants worked and which didn't - I wonder if the limit only applies to constants passed to the shader from the game/Helix mod and not constants defined locally in the shader?
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
OK, this one looks like it is going to be tricky. I have a suspicion that those are not the only two shaders relevant to this effect - the fact that the line follows the terrain correctly in each eye (even if the eyes disagree with each other) tells me that this effect is being processed entirely on the GPU, so it should be fixable - but I can't see anything here that would suggest how the height map is being used.
I have a strong suspicion that the terrain vertex and/or pixel shaders may be the correct place to fix this. What I think is probably happening is that the shaders you have identified is drawing this border made up of bezier curves to some off-screen buffer in 2D, which is then being drawn over the land/ocean in another shader. Can you post the vertex & pixel shaders for the terrain?
At the moment though, that is just a suspicion. There's a few experiments I'd like you to try which might help narrow this down.
First, see what it looks like if you set the pixel shader to output all black or all white by adding this to the end:
mov oC0.xyzw, c12.xxxx // black
or
mov oC0.xyzw, c12.yyyy // white
In particular I'm interested to know if just the border changes colour, the entire screen, the entire terrain, or if something else happens. The intention of this experiment is to determine how this shader is being used - if it is just drawing the border, if it is being overlayed on the terrain, or if it is a post-processing effect operating over the whole screen.
The next thing to try, is adjusting each of the outputs of the vertex shader in turn to see how the effect changes. It's not particularly clear what value to use for the depth, so for now ignore depth and convergence and just adjust the outputs by separation, something like this:
[code]
def c220, 0, 1, 0.0625, 0.5
dcl_2d s0
...
mov r11.xz, r0.xyyw // Replaced o1 with r11
mov o0, r1
mov o3, r1
mov r11.yw, c9.xxzy // Replaced o1 with r11
texldl r31, c220.z, s0
add r11.x, r11.x, r31.x // Add separation to X
mov o1, r11 // Copy adjusted r11 back to o1
[/code]
Do the same thing for each of the three outputs and note / screenshot the result. That should be enough to show whether the adjustment is in the right direction or not, and in particular to see if the effect continues to follow the terrain height after each adjustment. I think it would also be worth trying the same thing to each of the outputs of the terrain vertex shader as well to see if they also alter this effect.
OK, this one looks like it is going to be tricky. I have a suspicion that those are not the only two shaders relevant to this effect - the fact that the line follows the terrain correctly in each eye (even if the eyes disagree with each other) tells me that this effect is being processed entirely on the GPU, so it should be fixable - but I can't see anything here that would suggest how the height map is being used.
I have a strong suspicion that the terrain vertex and/or pixel shaders may be the correct place to fix this. What I think is probably happening is that the shaders you have identified is drawing this border made up of bezier curves to some off-screen buffer in 2D, which is then being drawn over the land/ocean in another shader. Can you post the vertex & pixel shaders for the terrain?
At the moment though, that is just a suspicion. There's a few experiments I'd like you to try which might help narrow this down.
First, see what it looks like if you set the pixel shader to output all black or all white by adding this to the end:
mov oC0.xyzw, c12.xxxx // black
or
mov oC0.xyzw, c12.yyyy // white
In particular I'm interested to know if just the border changes colour, the entire screen, the entire terrain, or if something else happens. The intention of this experiment is to determine how this shader is being used - if it is just drawing the border, if it is being overlayed on the terrain, or if it is a post-processing effect operating over the whole screen.
The next thing to try, is adjusting each of the outputs of the vertex shader in turn to see how the effect changes. It's not particularly clear what value to use for the depth, so for now ignore depth and convergence and just adjust the outputs by separation, something like this:
texldl r31, c220.z, s0
add r11.x, r11.x, r31.x // Add separation to X
mov o1, r11 // Copy adjusted r11 back to o1
Do the same thing for each of the three outputs and note / screenshot the result. That should be enough to show whether the adjustment is in the right direction or not, and in particular to see if the effect continues to follow the terrain height after each adjustment. I think it would also be worth trying the same thing to each of the outputs of the terrain vertex shader as well to see if they also alter this effect.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Hmm, well I tried the following
- set the colour for the output to white in the PS as you suggested.
x this results in the borders appearing white. No other effects
- commented out the various outputs in the vs
x no effect for o3.
x Commenting out o0 or o1 made the borders disappear
- tried to change the geometry of the outputs.
x No result for o1 (which I didn't really expect).
x Geometry can be affected through o0. Adding separation made no appreciable impact though. I was able to shift the whole geometry by just adding a constant instead. As a result, some borders ended up over the sea and, as expected, being displayed in proper stereo at sea level.
[URL=http://smg.photobucket.com/user/Muizer/media/rome203_50_zpsm0r5bqxu.jpg.html][IMG]http://img.photobucket.com/albums/v724/Muizer/rome203_50_zpsm0r5bqxu.jpg[/IMG][/URL]
- set the colour for the output to white in the PS as you suggested.
x this results in the borders appearing white. No other effects
- commented out the various outputs in the vs
x no effect for o3.
x Commenting out o0 or o1 made the borders disappear
- tried to change the geometry of the outputs.
x No result for o1 (which I didn't really expect).
x Geometry can be affected through o0. Adding separation made no appreciable impact though. I was able to shift the whole geometry by just adding a constant instead. As a result, some borders ended up over the sea and, as expected, being displayed in proper stereo at sea level.
[quote="Muizer"]- set the colour for the output to white in the PS as you suggested.
x this results in the borders appearing white. No other effects[/quote]OK, that indicates that the vertex shader is defining the bounds of the border and probably rules out post-processing.
[quote]- tried to change the geometry of the outputs.
x No result for o1 (which I didn't really expect).
x Geometry can be affected through o0. Adding separation made no appreciable impact though. I was able to shift the whole geometry by just adding a constant instead.[/quote]
What order of magnitude did you have to use for the constant? Separation will be less than 1 since it's a percentage of the screen, but if you found you needed to use a value much greater than 1 than it might indicate that you are working in pixels instead of percentages (if so any adjustment in this shader may need to be multiplied by the width of whatever render target it is using - maybe the screen width, maybe a fraction of the screen width, maybe an arbitrary power of 2 or maybe something else).
Are you able to rotate the camera in this game? If so, did moving the effect always move it horizontally with respect to the camera, or was it in a fixed direction with respect to the world?
I still think this is likely to be a 2D effect drawn onto an off-screen buffer that is later copied to the terrain - I noticed that the view-projection matrix is only 3x4, using only X, Z and W components:
[code]
dp3 r1.x, r0, c0.xzww
dp3 r1.y, r0, c1.xzww
dp3 r1.z, r0, c2.xzww
dp3 r1.w, r0, c3.xzww
[/code]
The fact that the vertical axis Y is missing here really suggests that these shaders are not responsible for the height of the border - if that was a 3D transformation it should be using dp4 instructions and use all four xyzw components of the matrix. The terrain shader seems a likely suspect for adding that height since it is already responsible for the height of the terrain and this border could easily be pained on the terrain in the terrain pixel shader.
Can you post the terrain shaders and experiment with disabling and adjusting the outputs of the terrain vertex shader, and disabling texture loads in the terrain pixel shader?
It's also possible that there is another distinct shader at work - were the shaders you posted the only ones that made the border disappear, and disabling the terrain shader the only one that made it flat? What happened to the borders when the terrain pixel shader is set to output a flat colour? What happens if you disable both the terrain and water at the same time?
[quote]As a result, some borders ended up over the sea and, as expected, being displayed in proper stereo at sea level.[/quote]That's not quite what I'm seeing in your screenshots - I'm seeing it drawn at the correct offset when pained on the ocean, and drawn with the wrong offset when pained on the terrain, even at sea level. Notably the ocean shader is likely to be simpler as it does not need the height that the terrain shader uses...
Muizer said:- set the colour for the output to white in the PS as you suggested.
x this results in the borders appearing white. No other effects
OK, that indicates that the vertex shader is defining the bounds of the border and probably rules out post-processing.
- tried to change the geometry of the outputs.
x No result for o1 (which I didn't really expect).
x Geometry can be affected through o0. Adding separation made no appreciable impact though. I was able to shift the whole geometry by just adding a constant instead.
What order of magnitude did you have to use for the constant? Separation will be less than 1 since it's a percentage of the screen, but if you found you needed to use a value much greater than 1 than it might indicate that you are working in pixels instead of percentages (if so any adjustment in this shader may need to be multiplied by the width of whatever render target it is using - maybe the screen width, maybe a fraction of the screen width, maybe an arbitrary power of 2 or maybe something else).
Are you able to rotate the camera in this game? If so, did moving the effect always move it horizontally with respect to the camera, or was it in a fixed direction with respect to the world?
I still think this is likely to be a 2D effect drawn onto an off-screen buffer that is later copied to the terrain - I noticed that the view-projection matrix is only 3x4, using only X, Z and W components:
The fact that the vertical axis Y is missing here really suggests that these shaders are not responsible for the height of the border - if that was a 3D transformation it should be using dp4 instructions and use all four xyzw components of the matrix. The terrain shader seems a likely suspect for adding that height since it is already responsible for the height of the terrain and this border could easily be pained on the terrain in the terrain pixel shader.
Can you post the terrain shaders and experiment with disabling and adjusting the outputs of the terrain vertex shader, and disabling texture loads in the terrain pixel shader?
It's also possible that there is another distinct shader at work - were the shaders you posted the only ones that made the border disappear, and disabling the terrain shader the only one that made it flat? What happened to the borders when the terrain pixel shader is set to output a flat colour? What happens if you disable both the terrain and water at the same time?
As a result, some borders ended up over the sea and, as expected, being displayed in proper stereo at sea level.
That's not quite what I'm seeing in your screenshots - I'm seeing it drawn at the correct offset when pained on the ocean, and drawn with the wrong offset when pained on the terrain, even at sea level. Notably the ocean shader is likely to be simpler as it does not need the height that the terrain shader uses...
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
[quote="DarkStarSword"]
What order of magnitude did you have to use for the constant? Separation will be less than 1 since it's a percentage of the screen, but if you found you needed to use a value much greater than 1 than it might indicate that you are working in pixels instead of percentages (if so any adjustment in this shader may need to be multiplied by the width of whatever render target it is using - maybe the screen width, maybe a fraction of the screen width, maybe an arbitrary power of 2 or maybe something else).[/quote]
In this instance I added 5.
[quote="DarkStarSword"]Are you able to rotate the camera in this game? If so, did moving the effect always move it horizontally with respect to the camera, or was it in a fixed direction with respect to the world?[/quote]
It was moved relative to the camera.
[quote="DarkStarSword"]
I still think this is likely to be a 2D effect drawn onto an off-screen buffer that is later copied to the terrain - I noticed that the view-projection matrix is only 3x4, using only X, Z and W components:
[code]
dp3 r1.x, r0, c0.xzww
dp3 r1.y, r0, c1.xzww
dp3 r1.z, r0, c2.xzww
dp3 r1.w, r0, c3.xzww
[/code]
The fact that the vertical axis Y is missing here really suggests that these shaders are not responsible for the height of the border - if that was a 3D transformation it should be using dp4 instructions and use all four xyzw components of the matrix. The terrain shader seems a likely suspect for adding that height since it is already responsible for the height of the terrain and this border could easily be pained on the terrain in the terrain pixel shader.
[/quote]
Not quite sure how to think of this. Do you mean like a straightforward vertical projection of a 2d map onto the surface?
I think you're probably right there's no z information coming with this shader. It's not mentioned in the header as being an input either.
[quote="DarkStarSword"]Can you post the terrain shaders and experiment with disabling and adjusting the outputs of the terrain vertex shader, and disabling texture loads in the terrain pixel shader?
It's also possible that there is another distinct shader at work - were the shaders you posted the only ones that made the border disappear, and disabling the terrain shader the only one that made it flat? What happened to the borders when the terrain pixel shader is set to output a flat colour? What happens if you disable both the terrain and water at the same time?[/quote]
I'll get back to you on this one. From the header, I noticed there definitely was possible elevation input in the land surface vs.
The identified shaders were the only ones that selectively removed the borders when disabled. There also were shaders that made the entire surface white. Bob mentioned such shaders aren't likely candidates to investigate though.
Oh and......thanks for trying to help me drag up the steep learning curve :D
DarkStarSword said:
What order of magnitude did you have to use for the constant? Separation will be less than 1 since it's a percentage of the screen, but if you found you needed to use a value much greater than 1 than it might indicate that you are working in pixels instead of percentages (if so any adjustment in this shader may need to be multiplied by the width of whatever render target it is using - maybe the screen width, maybe a fraction of the screen width, maybe an arbitrary power of 2 or maybe something else).
In this instance I added 5.
DarkStarSword said:Are you able to rotate the camera in this game? If so, did moving the effect always move it horizontally with respect to the camera, or was it in a fixed direction with respect to the world?
It was moved relative to the camera.
DarkStarSword said:
I still think this is likely to be a 2D effect drawn onto an off-screen buffer that is later copied to the terrain - I noticed that the view-projection matrix is only 3x4, using only X, Z and W components:
The fact that the vertical axis Y is missing here really suggests that these shaders are not responsible for the height of the border - if that was a 3D transformation it should be using dp4 instructions and use all four xyzw components of the matrix. The terrain shader seems a likely suspect for adding that height since it is already responsible for the height of the terrain and this border could easily be pained on the terrain in the terrain pixel shader.
Not quite sure how to think of this. Do you mean like a straightforward vertical projection of a 2d map onto the surface?
I think you're probably right there's no z information coming with this shader. It's not mentioned in the header as being an input either.
DarkStarSword said:Can you post the terrain shaders and experiment with disabling and adjusting the outputs of the terrain vertex shader, and disabling texture loads in the terrain pixel shader?
It's also possible that there is another distinct shader at work - were the shaders you posted the only ones that made the border disappear, and disabling the terrain shader the only one that made it flat? What happened to the borders when the terrain pixel shader is set to output a flat colour? What happens if you disable both the terrain and water at the same time?
I'll get back to you on this one. From the header, I noticed there definitely was possible elevation input in the land surface vs.
The identified shaders were the only ones that selectively removed the borders when disabled. There also were shaders that made the entire surface white. Bob mentioned such shaders aren't likely candidates to investigate though.
Oh and......thanks for trying to help me drag up the steep learning curve :D
[quote="Muizer"]Here goes. The terrain vs
[code]
<snip>
// g_height_map_dimension c5 1
<snip>
// g_height_map_lf_scale c9 1
// g_height_map_hf_scale c10 1
// g_height_map_vertical_offset c11 1
// s_low_res_height_map s0 1
<snip>
[/code][/quote]OK, as expected the terrain vertex shader uses a height map, but unfortunately...
[quote]And I am quite sure this is the accompanying ps. Disabling it returns a white land surface, but WITH relief.
[code]
<snip>
// vtex_page_table_sampler s0 1
// vtex_diffuse_cache_sampler s1 1
// vtex_normal_cache_sampler s2 1
<snip>
[/code]
[/quote]... none of these samplers in the pixel shader sound like they would be related to the border or rivers :-( There might be another pixel shader related to this effect.
[quote]Disabling it returns a white land surface, but WITH relief.[/quote]What about the border? Did you get white terrain with a border, or did the border disappear as well?
Just a quick question so I know where you are coming from - you mentioned you are looking at Mike's Rome II fix - are you using it purely as a learning exercise to understand how it was fixed, or is the fix incomplete or no longer work and you are trying to get it to work?
I've had a look at Mike's fix, and it looks like pixel shaders 47C2E5B6.txt, 86E3BF02.txt and 96A051BF are responsible for fixing this effect (and shadows). The pattern looks the same in each of them (I'd guess they are three slightly different variations of essentially the same effect), so I'll pick 86E3BF02.txt and explain what he is doing. Keep in mind that this is quite an advanced effect to fix and is *way* ahead of where we are in the lessons. Fixing this effect is on the same level of difficulty as fixing shadows, i.e. hard.
[code]
<snip>
// view_projection c16 4
// inv_view_projection c20 4
// g_overlay_view_projection c24 4
[/code]
Here are some matrices passed into the shader that are used to translate between different coordinate systems. We have view_projection which translates a point from the world to the screen, and inv_view_projection which does the opposite, translating a point from the screen to where it is in the world - these will become very important shortly. Plus there is g_overlay_view_projection, which sounds extremely relevant to the problem at hand since it is an overlay that we are trying to fix.
[code]
// s_overlay s8 1
[/code]
And *this* is what I was hoping we would see in the terrain pixel shader - this would be the 2D off-screen texture that the shaders you identified for the border were drawing to, and this shader is responsible for drawing it onto the terrain.
This part is why I chose this particular shader to explain:
[code]
<snip>
rcp r0.x, r0.x
mul r4.xyz, r0.x, r1
mov r4.w, c55.w
//Here....? NOO
dp4 r6.x, r4, c16
dp4 r6.y, r4, c17
dp4 r0.y, r4, c19
mov r7.xyz, c38
mad r7.xyz, r7, -c37.x, r4
add r8, r0.y, -c35
<snip>
[/code]
You might be wondering why that comment is there. The reason is that on the next three lines you can see a pattern that you will quickly learn to recognise as a matrix transformation to move from one coordinate system into another - and these signify prime suspects for places you may need to fix before or after. In this case the matrix is c16-c19, which as we saw in the headers is the view_projection matrix. Clearly Mike tried an adjustment here, but it did not work so he moved on.
I'm going to skip forward to around 90% through the shader - partly because there is a shadow fix in the middle that I'm ignoring for now and partly because I want to show the patterns to look for first:
[code]
<snip>
dp4 r3.x, r2, c24
dp4 r3.y, r2, c25
dp4 r1.w, r2, c27
rcp r1.w, r1.w
mul r2, r1.w, r3.xyyy
mad r2, r2, c50.xyyy, c50.x
texldl r2, r2, s8
<snip>
[/code]
Here we see two things. Firstly, you can see a series of three or four dp4 instructions with consecutively numbered constant registers (usually it will be four dp4 instructions, but sometimes one gets optimised out and you see three like this). Like the above example this shows that a matrix transformation is being used, so we are in a prime spot for a fix. You should also note that the particular matrix being used (c24-c27) is g_overlay_view_projection, which is very suggestive that it may be related to the overlay we are trying to fix because of it's helpful name.
The second thing to note is that the s8 sampler register is being used here, which we noted is s_overlay - again, the name is a significant clue that this code is working on the overlayed borders + rivers.
If I saw these two things I'd throw in a quick adjustment to see if the effect I was interested in moved, and how it moved. That would tell me if I *might* have found something relevant, and might give me a clue as to which coordinate system it's in. If an adjustment moves the effect horizontally with respect to the camera no matter how it is rotated it suggests that the coordinate is in either projection-space or view-space. If the adjustment moves the coordinate in some fixed direction with respect to the world (e.g. moves East/West no matter how the camera is rotated) it suggests that it is in world-space coordinates. Based on Mike's fix it's safe to assume that in this case it turned out to be in world-space coordinates.
Ok, let's look at Mike's fix:
[code]
//This part fixes the roads and rivers in campaign map
//Adjust r4 here and not r2, because r2 is adjusted from r4 before transforming
[/code]Here's a nice helpful comment that Mike left for us, and an explanation of which register he is adjusting - I'll come back to that in a moment.
The rest of the comments here are ones I've added:
[code]
// Copy r4 into r20:
mov r20.xyz, r4.xyz
// Set the homogeneous W coordinate to 1, to match the XYZ Cartesian
// coordinate. Necessary for the following matrix operations to work:
mov r20.w, c220.x
// Run this coordinate through the view_projection matrix (c16-c19) to
// transform it from world coordinates into projection (screen XY + Z buffer +
// W depth/perspective) coordinates:
dp4 r21.x, r20, c16
dp4 r21.y, r20, c17
dp4 r21.z, r20, c18
dp4 r21.w, r20, c19
// *Subtract* the stereo correction formula - this formula only applies in
// projection/screen space, which is why the fix just transformed a coordinate
// into it:
texldl r24, c220.z, s13
add r24.y, r21.w, -r24.y
mul r24.x, r24.x, r24.y
add r21.x, r21.x, -r24.x
// Use the inv_view_projection matrix (c20-c23) to transform the modified
// coordinate back into world-space:
dp4 r20.x, r21, c20
dp4 r20.y, r21, c21
dp4 r20.z, r21, c22
dp4 r20.w, r21, c23
// Replace the original coordinate with the modified one:
mov r4.xyz, r20.xyz
// As per Mike's comment, r2 is derived from r4 here.
// Not entirely clear what it's doing, but Mike clearly found through
// experimentation that the correction had to take place before this
// point:
mad r2.xyz, r4.y, -c62.yxyw, r4
mov r2.w, c55.w
// As noted earlier, the now corrected coordinate is translated to the
// coordinates on the overlay off-screen texture (not the screen
// coordinates) via the g_overlay_view_projection matrix:
dp4 r3.x, r2, c24
dp4 r3.y, r2, c25
dp4 r1.w, r2, c27
// This looks like a perspective divide, which makes sense given that
// this coordinate has just been run through a projection matrix:
rcp r1.w, r1.w
mul r2, r1.w, r3.xyyy
// And this looks like it is scaling the coordinate from a range of -1:1
// from the projection to a range of 0:1 required to sample a texture.
// It's also negating some of the axis:
mad r2, r2, c50.xyyy, c50.x
// And finally it samples the s_overlay texture with the borders + rivers:
texldl r2, r2, s8
[/code]
Here he is taking a world-space coordinate, transforming it into projection space where the stereo correction formula can be used, adjusting the coordinate and then transforming it back into world space.
The other thing to note here is that the stereo correction formula is being subtracted instead of added. This is easy enough to just try both ways to see which one you need. It may be that the world coordinate was already adjusted and this subtraction takes it back to the original world coordinate, or it might be that one of the axis was negated somewhere.
This shader also includes a shadow fix about 35% of the way through the file. The maths for the shadow fix is exactly the same as the maths for the borders fix, so you might like to study it as well.
... none of these samplers in the pixel shader sound like they would be related to the border or rivers :-( There might be another pixel shader related to this effect.
Disabling it returns a white land surface, but WITH relief.
What about the border? Did you get white terrain with a border, or did the border disappear as well?
Just a quick question so I know where you are coming from - you mentioned you are looking at Mike's Rome II fix - are you using it purely as a learning exercise to understand how it was fixed, or is the fix incomplete or no longer work and you are trying to get it to work?
I've had a look at Mike's fix, and it looks like pixel shaders 47C2E5B6.txt, 86E3BF02.txt and 96A051BF are responsible for fixing this effect (and shadows). The pattern looks the same in each of them (I'd guess they are three slightly different variations of essentially the same effect), so I'll pick 86E3BF02.txt and explain what he is doing. Keep in mind that this is quite an advanced effect to fix and is *way* ahead of where we are in the lessons. Fixing this effect is on the same level of difficulty as fixing shadows, i.e. hard.
Here are some matrices passed into the shader that are used to translate between different coordinate systems. We have view_projection which translates a point from the world to the screen, and inv_view_projection which does the opposite, translating a point from the screen to where it is in the world - these will become very important shortly. Plus there is g_overlay_view_projection, which sounds extremely relevant to the problem at hand since it is an overlay that we are trying to fix.
// s_overlay s8 1
And *this* is what I was hoping we would see in the terrain pixel shader - this would be the 2D off-screen texture that the shaders you identified for the border were drawing to, and this shader is responsible for drawing it onto the terrain.
This part is why I chose this particular shader to explain:
You might be wondering why that comment is there. The reason is that on the next three lines you can see a pattern that you will quickly learn to recognise as a matrix transformation to move from one coordinate system into another - and these signify prime suspects for places you may need to fix before or after. In this case the matrix is c16-c19, which as we saw in the headers is the view_projection matrix. Clearly Mike tried an adjustment here, but it did not work so he moved on.
I'm going to skip forward to around 90% through the shader - partly because there is a shadow fix in the middle that I'm ignoring for now and partly because I want to show the patterns to look for first:
Here we see two things. Firstly, you can see a series of three or four dp4 instructions with consecutively numbered constant registers (usually it will be four dp4 instructions, but sometimes one gets optimised out and you see three like this). Like the above example this shows that a matrix transformation is being used, so we are in a prime spot for a fix. You should also note that the particular matrix being used (c24-c27) is g_overlay_view_projection, which is very suggestive that it may be related to the overlay we are trying to fix because of it's helpful name.
The second thing to note is that the s8 sampler register is being used here, which we noted is s_overlay - again, the name is a significant clue that this code is working on the overlayed borders + rivers.
If I saw these two things I'd throw in a quick adjustment to see if the effect I was interested in moved, and how it moved. That would tell me if I *might* have found something relevant, and might give me a clue as to which coordinate system it's in. If an adjustment moves the effect horizontally with respect to the camera no matter how it is rotated it suggests that the coordinate is in either projection-space or view-space. If the adjustment moves the coordinate in some fixed direction with respect to the world (e.g. moves East/West no matter how the camera is rotated) it suggests that it is in world-space coordinates. Based on Mike's fix it's safe to assume that in this case it turned out to be in world-space coordinates.
Ok, let's look at Mike's fix:
//This part fixes the roads and rivers in campaign map
//Adjust r4 here and not r2, because r2 is adjusted from r4 before transforming
Here's a nice helpful comment that Mike left for us, and an explanation of which register he is adjusting - I'll come back to that in a moment.
The rest of the comments here are ones I've added:
// Copy r4 into r20:
mov r20.xyz, r4.xyz
// Set the homogeneous W coordinate to 1, to match the XYZ Cartesian
// coordinate. Necessary for the following matrix operations to work:
mov r20.w, c220.x
// Run this coordinate through the view_projection matrix (c16-c19) to
// transform it from world coordinates into projection (screen XY + Z buffer +
// W depth/perspective) coordinates:
dp4 r21.x, r20, c16
dp4 r21.y, r20, c17
dp4 r21.z, r20, c18
dp4 r21.w, r20, c19
// *Subtract* the stereo correction formula - this formula only applies in
// projection/screen space, which is why the fix just transformed a coordinate
// into it:
texldl r24, c220.z, s13
add r24.y, r21.w, -r24.y
mul r24.x, r24.x, r24.y
add r21.x, r21.x, -r24.x
// Use the inv_view_projection matrix (c20-c23) to transform the modified
// coordinate back into world-space:
dp4 r20.x, r21, c20
dp4 r20.y, r21, c21
dp4 r20.z, r21, c22
dp4 r20.w, r21, c23
// Replace the original coordinate with the modified one:
mov r4.xyz, r20.xyz
// As per Mike's comment, r2 is derived from r4 here.
// Not entirely clear what it's doing, but Mike clearly found through
// experimentation that the correction had to take place before this
// point:
mad r2.xyz, r4.y, -c62.yxyw, r4
mov r2.w, c55.w
// As noted earlier, the now corrected coordinate is translated to the
// coordinates on the overlay off-screen texture (not the screen
// coordinates) via the g_overlay_view_projection matrix:
dp4 r3.x, r2, c24
dp4 r3.y, r2, c25
dp4 r1.w, r2, c27
// This looks like a perspective divide, which makes sense given that
// this coordinate has just been run through a projection matrix:
rcp r1.w, r1.w
mul r2, r1.w, r3.xyyy
// And this looks like it is scaling the coordinate from a range of -1:1
// from the projection to a range of 0:1 required to sample a texture.
// It's also negating some of the axis:
mad r2, r2, c50.xyyy, c50.x
// And finally it samples the s_overlay texture with the borders + rivers:
texldl r2, r2, s8
Here he is taking a world-space coordinate, transforming it into projection space where the stereo correction formula can be used, adjusting the coordinate and then transforming it back into world space.
The other thing to note here is that the stereo correction formula is being subtracted instead of added. This is easy enough to just try both ways to see which one you need. It may be that the world coordinate was already adjusted and this subtraction takes it back to the original world coordinate, or it might be that one of the axis was negated somewhere.
This shader also includes a shadow fix about 35% of the way through the file. The maths for the shadow fix is exactly the same as the maths for the borders fix, so you might like to study it as well.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Interesting! To provide some context, Mike made a fix for this game a year ago. The developers, however, kept releasing patches until late last year and one of them broke the fix.
I wanted to see if I could repair the fix and in the process learn how to fix other games in this series, like the recently released Attila. I used to have a bit of a stake in the TW modding community working on the original Rome TW for many years. Of course I took an interest in Rome II as well, but these days, I can hardly suffer playing in 2D.
I'll have a closer look at the code you've quoted and the comments. I did note the back and forth projecting going on around the stereo fix, but wasn't aware the fix involved subsequent code as well (it not being out-dented).
Interesting! To provide some context, Mike made a fix for this game a year ago. The developers, however, kept releasing patches until late last year and one of them broke the fix.
I wanted to see if I could repair the fix and in the process learn how to fix other games in this series, like the recently released Attila. I used to have a bit of a stake in the TW modding community working on the original Rome TW for many years. Of course I took an interest in Rome II as well, but these days, I can hardly suffer playing in 2D.
I'll have a closer look at the code you've quoted and the comments. I did note the back and forth projecting going on around the stereo fix, but wasn't aware the fix involved subsequent code as well (it not being out-dented).
I don't suppose it's possible to get these shaders to use the input the land texture shaders use?
You might need to disable the effect if it won't be missed, approximate it if there is no other option or look for ways to use the information you do have to make up for the information you are missing, or like you suggested - in some cases you might be able to copy the necessary information from a different shader.
Can you post the complete unmodified vertex & pixel shaders for the effects and some 3D screenshots showing how they are broken and we might be able to provide some suggestions - I'd really want to see what you are working with before suggesting anything, particularly for the water as that might involve reflections or other advanced transformations.
For copying inputs from one shader to another - this is an advanced topic, but it is possible with some limitations. For instance, you can't copy the vertex inputs, but you can copy constants and matrices from one shader to another, which can sometimes be enough. You may also have problems if there are too many different instances of the shader you are trying to copy an input from as you never know which one the copy will be from (e.g. enemy health bars, or bloom/halos around lights).
I actually did something along these lines just the other day to fix the ripple distortion on the spirit rings in Oddworld, which were being drawn at screen depth:
For that particular effect, I copied the 4th column of the model-view-projection matrix for the red rings into the ripple post-processing shader and used it to calculate the depth at the origin (0,0,0,1) of the rings to align the ripples to them - here's the main commit that made this change (later commits polished this a bit, but this should be good enough to demonstrate the concept):
https://github.com/DarkStarSword/3d-fixes/commit/bd87a8f14866a05bcbf242d8c878c7cb6c4afc7a
A limitation here is that all the ripples would be aligned to the spirit ring depth, not just the ones from the spirit rings themselves. In this case that didn't matter since that depth would be approximately correct and most of the ripples were from explosions which were fast enough that any slight misalignment would not be noticed.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
This looks ok as well. You might try a lower number for DefPSConst1 (and change the code in the shader to match) - in theory you can use up to 223 for pixel shaders, but it seems that some games somehow reduce the maximum available constant registers so sometimes higher numbers don't work. Since you already know c200 works inside the shader you might try a number below that, like 190.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
The key bindings are setting Const2, but you are testing Const1 (c220.x) in the shader - it should be c220.y for Const2.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
I don't understand why c200 works but not c190.
Thanks, I corrected that. But with everything else being the same, if I change c49 to c190 in .ini & the shader file, the toggle doesn't work.
That's a pretty interesting note about which constants worked and which didn't - I wonder if the limit only applies to constants passed to the shader from the game/Helix mod and not constants defined locally in the shader?
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
With land surface VS disabled:
VS borders:
PS borders
I have a strong suspicion that the terrain vertex and/or pixel shaders may be the correct place to fix this. What I think is probably happening is that the shaders you have identified is drawing this border made up of bezier curves to some off-screen buffer in 2D, which is then being drawn over the land/ocean in another shader. Can you post the vertex & pixel shaders for the terrain?
At the moment though, that is just a suspicion. There's a few experiments I'd like you to try which might help narrow this down.
First, see what it looks like if you set the pixel shader to output all black or all white by adding this to the end:
mov oC0.xyzw, c12.xxxx // black
or
mov oC0.xyzw, c12.yyyy // white
In particular I'm interested to know if just the border changes colour, the entire screen, the entire terrain, or if something else happens. The intention of this experiment is to determine how this shader is being used - if it is just drawing the border, if it is being overlayed on the terrain, or if it is a post-processing effect operating over the whole screen.
The next thing to try, is adjusting each of the outputs of the vertex shader in turn to see how the effect changes. It's not particularly clear what value to use for the depth, so for now ignore depth and convergence and just adjust the outputs by separation, something like this:
Do the same thing for each of the three outputs and note / screenshot the result. That should be enough to show whether the adjustment is in the right direction or not, and in particular to see if the effect continues to follow the terrain height after each adjustment. I think it would also be worth trying the same thing to each of the outputs of the terrain vertex shader as well to see if they also alter this effect.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
- set the colour for the output to white in the PS as you suggested.
x this results in the borders appearing white. No other effects
- commented out the various outputs in the vs
x no effect for o3.
x Commenting out o0 or o1 made the borders disappear
- tried to change the geometry of the outputs.
x No result for o1 (which I didn't really expect).
x Geometry can be affected through o0. Adding separation made no appreciable impact though. I was able to shift the whole geometry by just adding a constant instead. As a result, some borders ended up over the sea and, as expected, being displayed in proper stereo at sea level.
What order of magnitude did you have to use for the constant? Separation will be less than 1 since it's a percentage of the screen, but if you found you needed to use a value much greater than 1 than it might indicate that you are working in pixels instead of percentages (if so any adjustment in this shader may need to be multiplied by the width of whatever render target it is using - maybe the screen width, maybe a fraction of the screen width, maybe an arbitrary power of 2 or maybe something else).
Are you able to rotate the camera in this game? If so, did moving the effect always move it horizontally with respect to the camera, or was it in a fixed direction with respect to the world?
I still think this is likely to be a 2D effect drawn onto an off-screen buffer that is later copied to the terrain - I noticed that the view-projection matrix is only 3x4, using only X, Z and W components:
The fact that the vertical axis Y is missing here really suggests that these shaders are not responsible for the height of the border - if that was a 3D transformation it should be using dp4 instructions and use all four xyzw components of the matrix. The terrain shader seems a likely suspect for adding that height since it is already responsible for the height of the terrain and this border could easily be pained on the terrain in the terrain pixel shader.
Can you post the terrain shaders and experiment with disabling and adjusting the outputs of the terrain vertex shader, and disabling texture loads in the terrain pixel shader?
It's also possible that there is another distinct shader at work - were the shaders you posted the only ones that made the border disappear, and disabling the terrain shader the only one that made it flat? What happened to the borders when the terrain pixel shader is set to output a flat colour? What happens if you disable both the terrain and water at the same time?
That's not quite what I'm seeing in your screenshots - I'm seeing it drawn at the correct offset when pained on the ocean, and drawn with the wrong offset when pained on the terrain, even at sea level. Notably the ocean shader is likely to be simpler as it does not need the height that the terrain shader uses...
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
In this instance I added 5.
It was moved relative to the camera.
Not quite sure how to think of this. Do you mean like a straightforward vertical projection of a 2d map onto the surface?
I think you're probably right there's no z information coming with this shader. It's not mentioned in the header as being an input either.
I'll get back to you on this one. From the header, I noticed there definitely was possible elevation input in the land surface vs.
The identified shaders were the only ones that selectively removed the borders when disabled. There also were shaders that made the entire surface white. Bob mentioned such shaders aren't likely candidates to investigate though.
Oh and......thanks for trying to help me drag up the steep learning curve :D
And I am quite sure this is the accompanying ps. Disabling it returns a white land surface, but WITH relief.
... none of these samplers in the pixel shader sound like they would be related to the border or rivers :-( There might be another pixel shader related to this effect.
What about the border? Did you get white terrain with a border, or did the border disappear as well?
Just a quick question so I know where you are coming from - you mentioned you are looking at Mike's Rome II fix - are you using it purely as a learning exercise to understand how it was fixed, or is the fix incomplete or no longer work and you are trying to get it to work?
I've had a look at Mike's fix, and it looks like pixel shaders 47C2E5B6.txt, 86E3BF02.txt and 96A051BF are responsible for fixing this effect (and shadows). The pattern looks the same in each of them (I'd guess they are three slightly different variations of essentially the same effect), so I'll pick 86E3BF02.txt and explain what he is doing. Keep in mind that this is quite an advanced effect to fix and is *way* ahead of where we are in the lessons. Fixing this effect is on the same level of difficulty as fixing shadows, i.e. hard.
Here are some matrices passed into the shader that are used to translate between different coordinate systems. We have view_projection which translates a point from the world to the screen, and inv_view_projection which does the opposite, translating a point from the screen to where it is in the world - these will become very important shortly. Plus there is g_overlay_view_projection, which sounds extremely relevant to the problem at hand since it is an overlay that we are trying to fix.
And *this* is what I was hoping we would see in the terrain pixel shader - this would be the 2D off-screen texture that the shaders you identified for the border were drawing to, and this shader is responsible for drawing it onto the terrain.
This part is why I chose this particular shader to explain:
You might be wondering why that comment is there. The reason is that on the next three lines you can see a pattern that you will quickly learn to recognise as a matrix transformation to move from one coordinate system into another - and these signify prime suspects for places you may need to fix before or after. In this case the matrix is c16-c19, which as we saw in the headers is the view_projection matrix. Clearly Mike tried an adjustment here, but it did not work so he moved on.
I'm going to skip forward to around 90% through the shader - partly because there is a shadow fix in the middle that I'm ignoring for now and partly because I want to show the patterns to look for first:
Here we see two things. Firstly, you can see a series of three or four dp4 instructions with consecutively numbered constant registers (usually it will be four dp4 instructions, but sometimes one gets optimised out and you see three like this). Like the above example this shows that a matrix transformation is being used, so we are in a prime spot for a fix. You should also note that the particular matrix being used (c24-c27) is g_overlay_view_projection, which is very suggestive that it may be related to the overlay we are trying to fix because of it's helpful name.
The second thing to note is that the s8 sampler register is being used here, which we noted is s_overlay - again, the name is a significant clue that this code is working on the overlayed borders + rivers.
If I saw these two things I'd throw in a quick adjustment to see if the effect I was interested in moved, and how it moved. That would tell me if I *might* have found something relevant, and might give me a clue as to which coordinate system it's in. If an adjustment moves the effect horizontally with respect to the camera no matter how it is rotated it suggests that the coordinate is in either projection-space or view-space. If the adjustment moves the coordinate in some fixed direction with respect to the world (e.g. moves East/West no matter how the camera is rotated) it suggests that it is in world-space coordinates. Based on Mike's fix it's safe to assume that in this case it turned out to be in world-space coordinates.
Ok, let's look at Mike's fix:
Here's a nice helpful comment that Mike left for us, and an explanation of which register he is adjusting - I'll come back to that in a moment.
The rest of the comments here are ones I've added:
Here he is taking a world-space coordinate, transforming it into projection space where the stereo correction formula can be used, adjusting the coordinate and then transforming it back into world space.
The other thing to note here is that the stereo correction formula is being subtracted instead of added. This is easy enough to just try both ways to see which one you need. It may be that the world coordinate was already adjusted and this subtraction takes it back to the original world coordinate, or it might be that one of the axis was negated somewhere.
This shader also includes a shadow fix about 35% of the way through the file. The maths for the shadow fix is exactly the same as the maths for the borders fix, so you might like to study it as well.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
I wanted to see if I could repair the fix and in the process learn how to fix other games in this series, like the recently released Attila. I used to have a bit of a stake in the TW modding community working on the original Rome TW for many years. Of course I took an interest in Rome II as well, but these days, I can hardly suffer playing in 2D.
I'll have a closer look at the code you've quoted and the comments. I did note the back and forth projecting going on around the stereo fix, but wasn't aware the fix involved subsequent code as well (it not being out-dented).