INSIDE
  6 / 11    
[quote="sgsrules"]Ok I give up for now :/ The only thing that's left to fix is to stereoize the projection matrix for the reflections. I don't know how the reflected projection matrix is being derived. I was hoping that i could use the reflected matrix to derive the offset projection that's needed but I don't know how. Maybe someone here can provide some insight. Here's some code I'm using to calculate the offset projection matrix for the camera: [code] public void GetStereoProjectionMatrix(bool isRightEye, Camera camera) { float fovRadians = Mathf.Deg2Rad * camera.fieldOfView; float w = camera.nearClipPlane * Mathf.Tan(fovRadians / 2); float dist = Separation * .5f; if (!isRightEye) dist *= -1; float top = w; float bottom = -w; float left = -camera.aspect * w - dist * camera.nearClipPlane / FocalDistance; float right = camera.aspect * w - dist * camera.nearClipPlane / FocalDistance; camera.projectionMatrix = PerspectiveOffCenter(left, right, bottom, top, camera.nearClipPlane, camera.farClipPlane); } static Matrix4x4 PerspectiveOffCenter(float left, float right, float bottom, float top, float near, float far) { float x = 2f * near / (right - left); float y = 2f * near / (top - bottom); float a = (right + left) / (right - left); float b = (top + bottom) / (top - bottom); float c = -(far + near) / (far - near); float d = -(2f * far * near) / (far - near); float e = -1f; Matrix4x4 m = new Matrix4x4(); m[0, 0] = x; m[0, 1] = 0; m[0, 2] = a; m[0, 3] = 0; m[1, 0] = 0; m[1, 1] = y; m[1, 2] = b; m[1, 3] = 0; m[2, 0] = 0; m[2, 1] = 0; m[2, 2] = c; m[2, 3] = d; m[3, 0] = 0; m[3, 1] = 0; m[3, 2] = e; m[3, 3] = 0; return m; } [/code] I looked through some of the source for Unity's standard reflection and they're using a oblique matrix to calculate the reflection based of a reflection plane, but I don't have access to the reflection plane. This link seemed relevant: https://www.opengl.org/discussion_boards/showthread.php/178177-Oblique-vs-Standard-projection-matrix[/quote] Hi, Any update on the fix?? The screenshots looked awesome! I'll settle with that. :)
sgsrules said:Ok I give up for now :/

The only thing that's left to fix is to stereoize the projection matrix for the reflections. I don't know how the reflected projection matrix is being derived. I was hoping that i could use the reflected matrix to derive the offset projection that's needed but I don't know how. Maybe someone here can provide some insight. Here's some code I'm using to calculate the offset projection matrix for the camera:
public void GetStereoProjectionMatrix(bool isRightEye, Camera camera)
{
float fovRadians = Mathf.Deg2Rad * camera.fieldOfView;
float w = camera.nearClipPlane * Mathf.Tan(fovRadians / 2);

float dist = Separation * .5f;
if (!isRightEye) dist *= -1;

float top = w;
float bottom = -w;

float left = -camera.aspect * w - dist * camera.nearClipPlane / FocalDistance;
float right = camera.aspect * w - dist * camera.nearClipPlane / FocalDistance;

camera.projectionMatrix = PerspectiveOffCenter(left, right, bottom, top, camera.nearClipPlane, camera.farClipPlane);
}

static Matrix4x4 PerspectiveOffCenter(float left, float right, float bottom, float top, float near, float far)
{
float x = 2f * near / (right - left);
float y = 2f * near / (top - bottom);
float a = (right + left) / (right - left);
float b = (top + bottom) / (top - bottom);
float c = -(far + near) / (far - near);
float d = -(2f * far * near) / (far - near);
float e = -1f;
Matrix4x4 m = new Matrix4x4();
m[0, 0] = x;
m[0, 1] = 0;
m[0, 2] = a;
m[0, 3] = 0;
m[1, 0] = 0;
m[1, 1] = y;
m[1, 2] = b;
m[1, 3] = 0;
m[2, 0] = 0;
m[2, 1] = 0;
m[2, 2] = c;
m[2, 3] = d;
m[3, 0] = 0;
m[3, 1] = 0;
m[3, 2] = e;
m[3, 3] = 0;
return m;
}


I looked through some of the source for Unity's standard reflection and they're using a oblique matrix to calculate the reflection based of a reflection plane, but I don't have access to the reflection plane.

This link seemed relevant:

https://www.opengl.org/discussion_boards/showthread.php/178177-Oblique-vs-Standard-projection-matrix




Hi, Any update on the fix??

The screenshots looked awesome! I'll settle with that. :)

#76
Posted 05/10/2017 06:41 AM   
@karmageddon, Still runing into issues with the reflections. @Bo3b maybe you can give me a hand. After looking at the 3dVision direct code we decided that the projection matrix could be modified via something like the following: [code] projection.31 += separation; projection.41 = -separation * convergence; projection = transposeMatrix(projection); [/code] except that getting the transpose of the onlique matix totally screws things up so i ended up transposing the values that are being modified only [code] projection.13 += separation; projection.14 = -separation * convergence; [/code] I had already tried offsetting the projection.13 which sets the separation but never thought about modifying the projection.14 value, normally this value is 0 for a projection matrix so i'm not sure how its working but it does. So now the convergence value is affecting the reflections but stuff is still lining up properly: [url=http://www.iforce.co.nz/View.aspx?i=e1ojzcrg.v3r.jpg][img]http://iforce.co.nz/i/e1ojzcrg.v3r.jpg[/img][/url] Even under ideal settings if i line up the character with his reflection using the right convergence settings things start to get misaligned the further away things are from the point of convergence. Notice how the reflections of the rocks and trees don't line up properly, this is compounded when viewed in stereoscopic 3d. I captured the fov, near and far clip planes from the camera's projection matrix right before the frame is rendered (without me fudging any values): MainCamera - fov: 50.98734 near: 1 far: 600 ReflectionCamera - fov: 60 near: .3 far: 1000 This seems to be why the two images don't line up. Whenever Unity is calculating the reflection projection it first sets things to the default values (60,.3,1000), recalculates the projection matrix, and then applies the oblique transformation. If i could extract the clipping planes values from the projection matrix I could recalculate the whole matrix from scratch, but i don't know if that's even possible. I'm kind of out of ideas at this point.. well, besides just removing the damn reflections :/
@karmageddon, Still runing into issues with the reflections.

@Bo3b maybe you can give me a hand. After looking at the 3dVision direct code we decided that the projection matrix could be modified via something like the following:

projection.31 += separation;
projection.41 = -separation * convergence;
projection = transposeMatrix(projection);


except that getting the transpose of the onlique matix totally screws things up so i ended up transposing the values that are being modified only
projection.13 += separation;
projection.14 = -separation * convergence;

I had already tried offsetting the projection.13 which sets the separation but never thought about modifying the projection.14 value, normally this value is 0 for a projection matrix so i'm not sure how its working but it does. So now the convergence value is affecting the reflections but stuff is still lining up properly:

Image

Even under ideal settings if i line up the character with his reflection using the right convergence settings things start to get misaligned the further away things are from the point of convergence. Notice how the reflections of the rocks and trees don't line up properly, this is compounded when viewed in stereoscopic 3d.

I captured the fov, near and far clip planes from the camera's projection matrix right before the frame is rendered (without me fudging any values):

MainCamera - fov: 50.98734 near: 1 far: 600
ReflectionCamera - fov: 60 near: .3 far: 1000

This seems to be why the two images don't line up. Whenever Unity is calculating the reflection projection it first sets things to the default values (60,.3,1000), recalculates the projection matrix, and then applies the oblique transformation.

If i could extract the clipping planes values from the projection matrix I could recalculate the whole matrix from scratch, but i don't know if that's even possible.

I'm kind of out of ideas at this point.. well, besides just removing the damn reflections :/

Like my work? You can send a donation via Paypal to sgs.rules@gmail.com

Windows 7 Pro 64x - Nvidia Driver 398.82 - EVGA 980Ti SC - Optoma HD26 with Edid override - 3D Vision 2 - i7-8700K CPU at 5.0Ghz - ASROCK Z370 Ext 4 Motherboard - 32 GB RAM Corsair Vengeance - 512 GB Samsung SSD 850 Pro - Creative Sound Blaster Z

#77
Posted 05/10/2017 09:56 PM   
[quote="sgsrules"]@Bo3b maybe you can give me a hand. After looking at the 3dVision direct code we decided that the projection matrix could be modified via something like the following: [code] projection.31 += separation; projection.41 = -separation * convergence; projection = transposeMatrix(projection); [/code] except that getting the transpose of the onlique matix totally screws things up so i ended up transposing the values that are being modified only [code] projection.13 += separation; projection.14 = -separation * convergence; [/code] I had already tried offsetting the projection.13 which sets the separation but never thought about modifying the projection.14 value, normally this value is 0 for a projection matrix so i'm not sure how its working but it does. So now the convergence value is affecting the reflections but stuff is still lining up properly: <snip>[/quote]Interesting results there. We are forging new ground here, so it's not a surprise it will be a bit sticky. However, I am surprised that doing the offset did not work. I think that typically for reflections, we will do a stereo-correction on the sampled point, so that it picks up the proper sky/sideways pixel from the normal to the reflection surface. In this case, I'd expect that changing the projection matrix would already put it into the right spot, so that the sampling is correct relative to an offset projection. Let's do more experiments. I'll dump out the PS for that reflection and see if I can discern any extra trickiness they might be doing. Always possible they don't use surface normals and use something else. I'll post the shader here. I know the headers are already stripped which will make it fairly inscrutable. Is this by chance a standard reflection shader in Unity? Might be worth trying to narrow it down to see. If you can experiment with the matrices, maybe you can see something. As a good experiment, what if you just make a single eye projection, just different from the original mono projection? So for example, just setup the left eye. We'd expect that to just move the mono projection to the left, and should be almost indistinguishable. And be 2D viewable. If that doesn't work, then something is off with the projection modification. Simplifying from the stereo values, it would be worth trying a simple translation. Like multiply their reflection matrix by a fixed x offset. Like a [proj 4x4]*[px py pz 1] with just setting px. The _41 value, I don't understand. In a standard projection I don't know what that would be, so the conv*sep is a mystery here, outside of the fact that it works in the sample. I've only ever see _41 change from rotations, which doesn't seem likely for this. The nvidia documentation sucks like hell though, so it's also possible, maybe probable that the formula is off. I was using _41 = sep*con, but a safer thing to try would be _41 += sep*con, in case the value is valuable. The documentation shows straight assignment, but += should still work if it's zero. All cases in the sample program, it's zero to start. There is also the complexity of the matrices being left or right handed, and row-major or column-major. The sample code is setup to handle the column-major order destination of HLSL. Since the starting matrix is row-major as XMMATRIX, it needs the transpose to set it correctly for consumption by the HLSL shaders. I don't know if INSIDE uses right-hand matrices, but it's unlikely. But, if Play Dead tends toward OpenGL, or uses OpenGL as their basis, it's a possibility, because OpenGL is Right-handed. It might be worth experimenting with the transpose though to see if it's necessary or not. The matrix you get from their setup might already be transposed, in which case you'd modify _13 and _14. <rant>Microsoft sucks donkey. They can't even keep their matrices consistent from generation to generation, and they use Left-hand matrices instead of right-hand like every other graphics software in the world. It doesn't have to be this complicated! Grumble.</rant>
sgsrules said:@Bo3b maybe you can give me a hand. After looking at the 3dVision direct code we decided that the projection matrix could be modified via something like the following:

projection.31 += separation;
projection.41 = -separation * convergence;
projection = transposeMatrix(projection);


except that getting the transpose of the onlique matix totally screws things up so i ended up transposing the values that are being modified only
projection.13 += separation;
projection.14 = -separation * convergence;

I had already tried offsetting the projection.13 which sets the separation but never thought about modifying the projection.14 value, normally this value is 0 for a projection matrix so i'm not sure how its working but it does. So now the convergence value is affecting the reflections but stuff is still lining up properly:
<snip>
Interesting results there. We are forging new ground here, so it's not a surprise it will be a bit sticky. However, I am surprised that doing the offset did not work.

I think that typically for reflections, we will do a stereo-correction on the sampled point, so that it picks up the proper sky/sideways pixel from the normal to the reflection surface.

In this case, I'd expect that changing the projection matrix would already put it into the right spot, so that the sampling is correct relative to an offset projection.

Let's do more experiments.

I'll dump out the PS for that reflection and see if I can discern any extra trickiness they might be doing. Always possible they don't use surface normals and use something else. I'll post the shader here. I know the headers are already stripped which will make it fairly inscrutable.

Is this by chance a standard reflection shader in Unity? Might be worth trying to narrow it down to see.


If you can experiment with the matrices, maybe you can see something. As a good experiment, what if you just make a single eye projection, just different from the original mono projection? So for example, just setup the left eye. We'd expect that to just move the mono projection to the left, and should be almost indistinguishable. And be 2D viewable.

If that doesn't work, then something is off with the projection modification. Simplifying from the stereo values, it would be worth trying a simple translation. Like multiply their reflection matrix by a fixed x offset. Like a [proj 4x4]*[px py pz 1] with just setting px.


The _41 value, I don't understand. In a standard projection I don't know what that would be, so the conv*sep is a mystery here, outside of the fact that it works in the sample. I've only ever see _41 change from rotations, which doesn't seem likely for this.

The nvidia documentation sucks like hell though, so it's also possible, maybe probable that the formula is off. I was using _41 = sep*con, but a safer thing to try would be _41 += sep*con, in case the value is valuable. The documentation shows straight assignment, but += should still work if it's zero. All cases in the sample program, it's zero to start.


There is also the complexity of the matrices being left or right handed, and row-major or column-major. The sample code is setup to handle the column-major order destination of HLSL. Since the starting matrix is row-major as XMMATRIX, it needs the transpose to set it correctly for consumption by the HLSL shaders.

I don't know if INSIDE uses right-hand matrices, but it's unlikely. But, if Play Dead tends toward OpenGL, or uses OpenGL as their basis, it's a possibility, because OpenGL is Right-handed.

It might be worth experimenting with the transpose though to see if it's necessary or not. The matrix you get from their setup might already be transposed, in which case you'd modify _13 and _14.

<rant>Microsoft sucks donkey. They can't even keep their matrices consistent from generation to generation, and they use Left-hand matrices instead of right-hand like every other graphics software in the world. It doesn't have to be this complicated! Grumble.</rant>

Acer H5360 (1280x720@120Hz) - ASUS VG248QE with GSync mod - 3D Vision 1&2 - Driver 372.54
GTX 970 - i5-4670K@4.2GHz - 12GB RAM - Win7x64+evilKB2670838 - 4 Disk X25 RAID
SAGER NP9870-S - GTX 980 - i7-6700K - Win10 Pro 1607
Latest 3Dmigoto Release
Bo3b's School for ShaderHackers

#78
Posted 05/11/2017 05:51 AM   
[quote="bo3b"]Interesting results there. We are forging new ground here, so it's not a surprise it will be a bit sticky. However, I am surprised that doing the offset did not work. I think that typically for reflections, we will do a stereo-correction on the sampled point, so that it picks up the proper sky/sideways pixel from the normal to the reflection surface. In this case, I'd expect that changing the projection matrix would already put it into the right spot, so that the sampling is correct relative to an offset projection. Let's do more experiments. I'll dump out the PS for that reflection and see if I can discern any extra trickiness they might be doing. Always possible they don't use surface normals and use something else. I'll post the shader here. I know the headers are already stripped which will make it fairly inscrutable. Is this by chance a standard reflection shader in Unity? Might be worth trying to narrow it down to see. [/quote] To be clear, I'm not modifying any shaders. My method simply looks for any cameras during unity's update loop. If it finds one it attaches a script to it that modifies the camera's position, view matrix and projection matrix. Another script blits the contents of the camera after all rendering is done to the appropriate eye texture. Since everything is being rendered twice from different view points everything works perfectly without having to screw around with any shaders, all the lighting, volumetric effects etc are perfect. I've already played through the game completely and the only thing that is messed up are the water reflections. The water reflections are generally comprised of two separate components, One that uses a screen space reflection shader, and another that uses a standard planar reflection. The screen space reflections are fairly faint and have a lot of warping applied to them via surface normals, you can barely make it out in the above screenshot. The screen space reflections are working perfectly and don't need to be fixed. The planar reflections were not working because they are rendered during a separate pass using their own camera. The game has a total of 4 or 5 active cameras. One for the main game, another for menus, and 2 or 3 to handle reflections. The reflection cameras copy the main cameras settings, then flip their position and view direction along the y axis using the reflection plane. The projection matrix is then modified with an oblique clipping plane transformation. The rendered image is then projected on to the reflected surface. if you've ever done simple planar reflections this should be pretty standard stuff. I'm not 100% certain of this but i'm fairly confident that's what's happening based on my experiments and looking the unity source code for water reflections that's part of their standard effects package. That being said INSIDE uses a ton of things that aren't part of the standard unity package so this may be different. [quote] If you can experiment with the matrices, maybe you can see something. As a good experiment, what if you just make a single eye projection, just different from the original mono projection? So for example, just setup the left eye. We'd expect that to just move the mono projection to the left, and should be almost indistinguishable. And be 2D viewable. If that doesn't work, then something is off with the projection modification. Simplifying from the stereo values, it would be worth trying a simple translation. Like multiply their reflection matrix by a fixed x offset. Like a [proj 4x4]*[px py pz 1] with just setting px. The _41 value, I don't understand. In a standard projection I don't know what that would be, so the conv*sep is a mystery here, outside of the fact that it works in the sample. I've only ever see _41 change from rotations, which doesn't seem likely for this. The nvidia documentation sucks like hell though, so it's also possible, maybe probable that the formula is off. I was using _41 = sep*con, but a safer thing to try would be _41 += sep*con, in case the value is valuable. The documentation shows straight assignment, but += should still work if it's zero. All cases in the sample program, it's zero to start. There is also the complexity of the matrices being left or right handed, and row-major or column-major. The sample code is setup to handle the column-major order destination of HLSL. Since the starting matrix is row-major as XMMATRIX, it needs the transpose to set it correctly for consumption by the HLSL shaders. I don't know if INSIDE uses right-hand matrices, but it's unlikely. But, if Play Dead tends toward OpenGL, or uses OpenGL as their basis, it's a possibility, because OpenGL is Right-handed. It might be worth experimenting with the transpose though to see if it's necessary or not. The matrix you get from their setup might already be transposed, in which case you'd modify _13 and _14. <rant>Microsoft sucks donkey. They can't even keep their matrices consistent from generation to generation, and they use Left-hand matrices instead of right-hand like every other graphics software in the world. It doesn't have to be this complicated! Grumble.</rant> [/quote] Unity uses column major matrices and is I believe is left handed. I've also ran into a couple of annoying quirks with the engine. Modifying a matrices components directly doesn't seem to do anything, but replacing the whole matrix does. This probably because there's some sort of callback that updates the uniform variable when a new matrix is assigned to it but they didn't add one when modifying the components directly. i can understand them not doing this for individual components but their setColumn and setRow methods should update the camera matrix.. oh well. Anyhow, I went back and double checked the view matrix. I only applied a view matrix offset and left the projection alone and didn't modify the reflection camera at all to see if the reflections lined up. I ended up with this: (apologies for the ugly anaglyph ;)) [url=http://www.iforce.co.nz/View.aspx?i=11zdsvcv.w2r.jpg][img]http://iforce.co.nz/i/11zdsvcv.w2r.jpg[/img][/url] As you can see in the background rocks, they're being reflected incorrectly. It turned out that the offset in the reflection was inverted (since it's a reflection). The reflection camera was copying the modified values from the main camera but when it flipped it's values it also flipped the eye direction. To fix it i reverted the reflected camera position back to it's original position and used the camera -xaxis to perform the offset. I then ended up with this (keep in my that i'm using a ridiculous separation value for testing ;) ): [url=http://www.iforce.co.nz/View.aspx?i=eo4kwyin.ke0.jpg][img]http://iforce.co.nz/i/eo4kwyin.ke0.jpg[/img][/url] Sweet! So that's fixes the first part. But now when i apply the off center projection to the main camera it breaks the reflection since it's projection isn't modified: [url=http://www.iforce.co.nz/View.aspx?i=5ewupelh.wiz.jpg][img]http://iforce.co.nz/i/5ewupelh.wiz.jpg[/img][/url] But now that the view matrix is at least working i should be able to just offset the values e.g. [code] projection.13 += separation; projection.14 = -separation * convergence; [/code] But it's a no go. I tried all sorts of permutations. The results that weren't total garbage ended up being: [code] float finalSeparation = Separation* .5 * MainCamForReflect.nearClipPlane / FocalDistance; projection.13-= finalSeparation ; projection.14 = FocalDistance * finalSeparation ; [/code] the finalSeparation is the offset value i use to calculate the off center projection matrix left and right values. But the results are still off, I won't even bother posting a screen. Luckily I finally came to a solution... First I yelled "Fuck you you piece of shit projection matrix!" and decided to leave the reflection matrix alone. Instead of using a parallel projection, I used the toe-in method to calculate the stereo pairs for the reflection (everything else still uses a parallel off center projection). I know a toe-in projection is wrong but since this is being used for a reflection that is being distorted a bit I think it's a good enough approximation. Besides, the original projection matrix that is being used for the reflection doesn't even have the same properties as the main camera, so that's already an approximation. Here's the final result: [url=http://www.iforce.co.nz/View.aspx?i=5fo1ghts.0p1.jpg][img]http://iforce.co.nz/i/5fo1ghts.0p1.jpg[/img][/url] I'll test things out later on an actual 3d setup but I'm fairly certain that this will work. Now we just need to incorporate 3d Vision Direct. Depending on how long that takes i might just release the fix as is for now and just warn people to reduce the 3dvision automatic separation value to 1. It just seems silly that the only reason I'm using 3dmigoto is to force Unity to render in fullscreen exclusive mode.
bo3b said:Interesting results there. We are forging new ground here, so it's not a surprise it will be a bit sticky. However, I am surprised that doing the offset did not work.

I think that typically for reflections, we will do a stereo-correction on the sampled point, so that it picks up the proper sky/sideways pixel from the normal to the reflection surface.

In this case, I'd expect that changing the projection matrix would already put it into the right spot, so that the sampling is correct relative to an offset projection.

Let's do more experiments.

I'll dump out the PS for that reflection and see if I can discern any extra trickiness they might be doing. Always possible they don't use surface normals and use something else. I'll post the shader here. I know the headers are already stripped which will make it fairly inscrutable.

Is this by chance a standard reflection shader in Unity? Might be worth trying to narrow it down to see.

To be clear, I'm not modifying any shaders. My method simply looks for any cameras during unity's update loop. If it finds one it attaches a script to it that modifies the camera's position, view matrix and projection matrix. Another script blits the contents of the camera after all rendering is done to the appropriate eye texture. Since everything is being rendered twice from different view points everything works perfectly without having to screw around with any shaders, all the lighting, volumetric effects etc are perfect. I've already played through the game completely and the only thing that is messed up are the water reflections.

The water reflections are generally comprised of two separate components, One that uses a screen space reflection shader, and another that uses a standard planar reflection.

The screen space reflections are fairly faint and have a lot of warping applied to them via surface normals, you can barely make it out in the above screenshot. The screen space reflections are working perfectly and don't need to be fixed.

The planar reflections were not working because they are rendered during a separate pass using their own camera. The game has a total of 4 or 5 active cameras. One for the main game, another for menus, and 2 or 3 to handle reflections. The reflection cameras copy the main cameras settings, then flip their position and view direction along the y axis using the reflection plane. The projection matrix is then modified with an oblique clipping plane transformation. The rendered image is then projected on to the reflected surface. if you've ever done simple planar reflections this should be pretty standard stuff. I'm not 100% certain of this but i'm fairly confident that's what's happening based on my experiments and looking the unity source code for water reflections that's part of their standard effects package. That being said INSIDE uses a ton of things that aren't part of the standard unity package so this may be different.


If you can experiment with the matrices, maybe you can see something. As a good experiment, what if you just make a single eye projection, just different from the original mono projection? So for example, just setup the left eye. We'd expect that to just move the mono projection to the left, and should be almost indistinguishable. And be 2D viewable.

If that doesn't work, then something is off with the projection modification. Simplifying from the stereo values, it would be worth trying a simple translation. Like multiply their reflection matrix by a fixed x offset. Like a [proj 4x4]*[px py pz 1] with just setting px.


The _41 value, I don't understand. In a standard projection I don't know what that would be, so the conv*sep is a mystery here, outside of the fact that it works in the sample. I've only ever see _41 change from rotations, which doesn't seem likely for this.

The nvidia documentation sucks like hell though, so it's also possible, maybe probable that the formula is off. I was using _41 = sep*con, but a safer thing to try would be _41 += sep*con, in case the value is valuable. The documentation shows straight assignment, but += should still work if it's zero. All cases in the sample program, it's zero to start.


There is also the complexity of the matrices being left or right handed, and row-major or column-major. The sample code is setup to handle the column-major order destination of HLSL. Since the starting matrix is row-major as XMMATRIX, it needs the transpose to set it correctly for consumption by the HLSL shaders.

I don't know if INSIDE uses right-hand matrices, but it's unlikely. But, if Play Dead tends toward OpenGL, or uses OpenGL as their basis, it's a possibility, because OpenGL is Right-handed.

It might be worth experimenting with the transpose though to see if it's necessary or not. The matrix you get from their setup might already be transposed, in which case you'd modify _13 and _14.

<rant>Microsoft sucks donkey. They can't even keep their matrices consistent from generation to generation, and they use Left-hand matrices instead of right-hand like every other graphics software in the world. It doesn't have to be this complicated! Grumble.</rant>

Unity uses column major matrices and is I believe is left handed. I've also ran into a couple of annoying quirks with the engine. Modifying a matrices components directly doesn't seem to do anything, but replacing the whole matrix does. This probably because there's some sort of callback that updates the uniform variable when a new matrix is assigned to it but they didn't add one when modifying the components directly. i can understand them not doing this for individual components but their setColumn and setRow methods should update the camera matrix.. oh well.

Anyhow, I went back and double checked the view matrix. I only applied a view matrix offset and left the projection alone and didn't modify the reflection camera at all to see if the reflections lined up. I ended up with this: (apologies for the ugly anaglyph ;))

Image
As you can see in the background rocks, they're being reflected incorrectly. It turned out that the offset in the reflection was inverted (since it's a reflection). The reflection camera was copying the modified values from the main camera but when it flipped it's values it also flipped the eye direction. To fix it i reverted the reflected camera position back to it's original position and used the camera -xaxis to perform the offset. I then ended up with this (keep in my that i'm using a ridiculous separation value for testing ;) ):
Image
Sweet! So that's fixes the first part. But now when i apply the off center projection to the main camera it breaks the reflection since it's projection isn't modified:
Image
But now that the view matrix is at least working i should be able to just offset the values e.g.
projection.13 += separation;
projection.14 = -separation * convergence;

But it's a no go. I tried all sorts of permutations. The results that weren't total garbage ended up being:
float finalSeparation = Separation* .5 * MainCamForReflect.nearClipPlane / FocalDistance; 
projection.13-= finalSeparation ;
projection.14 = FocalDistance * finalSeparation ;

the finalSeparation is the offset value i use to calculate the off center projection matrix left and right values. But the results are still off, I won't even bother posting a screen.

Luckily I finally came to a solution... First I yelled "Fuck you you piece of shit projection matrix!" and decided to leave the reflection matrix alone. Instead of using a parallel projection, I used the toe-in method to calculate the stereo pairs for the reflection (everything else still uses a parallel off center projection). I know a toe-in projection is wrong but since this is being used for a reflection that is being distorted a bit I think it's a good enough approximation. Besides, the original projection matrix that is being used for the reflection doesn't even have the same properties as the main camera, so that's already an approximation. Here's the final result:
Image
I'll test things out later on an actual 3d setup but I'm fairly certain that this will work. Now we just need to incorporate 3d Vision Direct. Depending on how long that takes i might just release the fix as is for now and just warn people to reduce the 3dvision automatic separation value to 1. It just seems silly that the only reason I'm using 3dmigoto is to force Unity to render in fullscreen exclusive mode.

Like my work? You can send a donation via Paypal to sgs.rules@gmail.com

Windows 7 Pro 64x - Nvidia Driver 398.82 - EVGA 980Ti SC - Optoma HD26 with Edid override - 3D Vision 2 - i7-8700K CPU at 5.0Ghz - ASROCK Z370 Ext 4 Motherboard - 32 GB RAM Corsair Vengeance - 512 GB Samsung SSD 850 Pro - Creative Sound Blaster Z

#79
Posted 05/11/2017 08:55 PM   
[quote="sgsrules"]It just seems silly that the only reason I'm using 3dmigoto is to force Unity to render in fullscreen exclusive mode.[/quote] And use the SBS/TaB Mode from 3Dmigoto :))) otherwise i/others that use this mode will have to play 720p :( (i have 3DTV Play) Very nice results!! Waiting to play this game....In my opinion, reflections sometimes are more difficult to fix than shadows/lights.
sgsrules said:It just seems silly that the only reason I'm using 3dmigoto is to force Unity to render in fullscreen exclusive mode.

And use the SBS/TaB Mode from 3Dmigoto :))) otherwise i/others that use this mode will have to play 720p :( (i have 3DTV Play)

Very nice results!! Waiting to play this game....In my opinion, reflections sometimes are more difficult to fix than shadows/lights.

MY WEB

Helix Mod - Making 3D Better

My 3D Screenshot Gallery

Like my fixes? you can donate to Paypal: dhr.donation@gmail.com

#80
Posted 05/11/2017 09:16 PM   
How I've felt while working on this: https://i.imgur.com/C1HaL9W.gifv
How I've felt while working on this:

https://i.imgur.com/C1HaL9W.gifv

Like my work? You can send a donation via Paypal to sgs.rules@gmail.com

Windows 7 Pro 64x - Nvidia Driver 398.82 - EVGA 980Ti SC - Optoma HD26 with Edid override - 3D Vision 2 - i7-8700K CPU at 5.0Ghz - ASROCK Z370 Ext 4 Motherboard - 32 GB RAM Corsair Vengeance - 512 GB Samsung SSD 850 Pro - Creative Sound Blaster Z

#81
Posted 05/11/2017 09:17 PM   
[quote="sgsrules"]Luckily I finally came to a solution... First I yelled "Fuck you you piece of shit projection matrix!" and decided to leave the reflection matrix alone.[/quote] [img]https://media.tenor.co/images/0489fb2f025d80cb993ac1e2712682fa/tenor.gif[/img]
sgsrules said:Luckily I finally came to a solution... First I yelled "Fuck you you piece of shit projection matrix!" and decided to leave the reflection matrix alone.

Image

3D Vision must live! NVIDIA, don't let us down!

#82
Posted 05/11/2017 11:02 PM   
@sgsrules: Hot damn, that's some killer stuff right there! It hadn't occurred to me that with the inverted matrix we ought to be left/right swap as well. I only barely understand reflections, I've never done any 3D shader stuff before starting on 3Dmigoto. For looking at the reflections I knew that you were not modifying shaders, just was thinking that looking at their original code might give some clues to what goes wrong. If toe-in works, I think that's a great compromise, and way better than disabled. [quote="DHR"][quote="sgsrules"]It just seems silly that the only reason I'm using 3dmigoto is to force Unity to render in fullscreen exclusive mode.[/quote]And use the SBS/TaB Mode from 3Dmigoto :))) otherwise i/others that use this mode will have to play 720p :( (i have 3DTV Play) Very nice results!! Waiting to play this game....In my opinion, reflections sometimes are more difficult to fix than shadows/lights.[/quote]The SBS/TAB support is a very good point. It would also give us all the key handling mechanisms for convergence and separation presets. Given that you presumably have it all working with 3Dmigoto, I'd say let's go that route. No need to make more work for us if it's unnecessary, even if it's a little clunky. Thanks for keeping after this one, this is really cool!
@sgsrules: Hot damn, that's some killer stuff right there!

It hadn't occurred to me that with the inverted matrix we ought to be left/right swap as well. I only barely understand reflections, I've never done any 3D shader stuff before starting on 3Dmigoto.

For looking at the reflections I knew that you were not modifying shaders, just was thinking that looking at their original code might give some clues to what goes wrong. If toe-in works, I think that's a great compromise, and way better than disabled.


DHR said:
sgsrules said:It just seems silly that the only reason I'm using 3dmigoto is to force Unity to render in fullscreen exclusive mode.
And use the SBS/TaB Mode from 3Dmigoto :))) otherwise i/others that use this mode will have to play 720p :( (i have 3DTV Play)

Very nice results!! Waiting to play this game....In my opinion, reflections sometimes are more difficult to fix than shadows/lights.
The SBS/TAB support is a very good point. It would also give us all the key handling mechanisms for convergence and separation presets.

Given that you presumably have it all working with 3Dmigoto, I'd say let's go that route. No need to make more work for us if it's unnecessary, even if it's a little clunky.

Thanks for keeping after this one, this is really cool!

Acer H5360 (1280x720@120Hz) - ASUS VG248QE with GSync mod - 3D Vision 1&2 - Driver 372.54
GTX 970 - i5-4670K@4.2GHz - 12GB RAM - Win7x64+evilKB2670838 - 4 Disk X25 RAID
SAGER NP9870-S - GTX 980 - i7-6700K - Win10 Pro 1607
Latest 3Dmigoto Release
Bo3b's School for ShaderHackers

#83
Posted 05/12/2017 01:35 AM   
[quote="sgsrules"]How I've felt while working on this: https://i.imgur.com/C1HaL9W.gifv[/quote] That is excellent! :-) Not as excellent as what you've done/are doing though :-)
sgsrules said:How I've felt while working on this:

https://i.imgur.com/C1HaL9W.gifv


That is excellent! :-) Not as excellent as what you've done/are doing though :-)

Gigabyte RTX2080TI Gaming OC, I7-6700k ~ 4.4Ghz, 3x BenQ XL2420T, BenQ TK800, LG 55EG960V (3D OLED), Samsung 850 EVO SSD, Crucial M4 SSD, 3D vision kit, Xpand x104 glasses, Corsair HX1000i, Win 10 pro 64/Win 7 64https://www.3dmark.com/fs/9529310

#84
Posted 05/12/2017 10:11 AM   
[quote="bo3b"] Given that you presumably have it all working with 3Dmigoto, I'd say let's go that route. No need to make more work for us if it's unnecessary, even if it's a little clunky. [/quote] yes, it's definitely a bit clunky i need to cleanup a lot of the injection code before i can release this, half the time two copies of the game are launched and one crashes lol. I agree though it would be a lot simpler to just use 3dmigoto for now. Is it possible to force the Separation value in automatic mode to zero? the lowest that the it seems to go is 1. Is it also possible to create a profile for the game or change a setting in 3dmigoto so that it always loads up the game with a separation of zero so that people don't have to manually do this?
bo3b said:
Given that you presumably have it all working with 3Dmigoto, I'd say let's go that route. No need to make more work for us if it's unnecessary, even if it's a little clunky.


yes, it's definitely a bit clunky i need to cleanup a lot of the injection code before i can release this, half the time two copies of the game are launched and one crashes lol.

I agree though it would be a lot simpler to just use 3dmigoto for now.

Is it possible to force the Separation value in automatic mode to zero? the lowest that the it seems to go is 1. Is it also possible to create a profile for the game or change a setting in 3dmigoto so that it always loads up the game with a separation of zero so that people don't have to manually do this?

Like my work? You can send a donation via Paypal to sgs.rules@gmail.com

Windows 7 Pro 64x - Nvidia Driver 398.82 - EVGA 980Ti SC - Optoma HD26 with Edid override - 3D Vision 2 - i7-8700K CPU at 5.0Ghz - ASROCK Z370 Ext 4 Motherboard - 32 GB RAM Corsair Vengeance - 512 GB Samsung SSD 850 Pro - Creative Sound Blaster Z

#85
Posted 05/12/2017 06:36 PM   
[quote="sgsrules"] Is it possible to force the Separation value in automatic mode to zero?[/quote] You have asked that multiple times and I then asked "did you try setting StereoTextureEnable to 0x00000000 in its Nvidia profile" :p? Did you try it? At least some time ago it made Dolphin use 2x the resources of 2D instead of 4x (it was using 3D Vision Automatic on top of Dolphin's own 3D Vision). I want to know if it will make the game 2D with your injector or if it will work flawlessly. Apart from the Nvidia profile directly, you can also configure that setting with 3Dmigoto.
sgsrules said:
Is it possible to force the Separation value in automatic mode to zero?


You have asked that multiple times and I then asked "did you try setting StereoTextureEnable to 0x00000000 in its Nvidia profile" :p? Did you try it? At least some time ago it made Dolphin use 2x the resources of 2D instead of 4x (it was using 3D Vision Automatic on top of Dolphin's own 3D Vision).

I want to know if it will make the game 2D with your injector or if it will work flawlessly.

Apart from the Nvidia profile directly, you can also configure that setting with 3Dmigoto.

CPU: Intel Core i7 7700K @ 4.9GHz
Motherboard: Gigabyte Aorus GA-Z270X-Gaming 5
RAM: GSKILL Ripjaws Z 16GB 3866MHz CL18
GPU: MSI GeForce RTX 2080Ti Gaming X Trio
Monitor: Asus PG278QR
Speakers: Logitech Z506
Donations account: masterotakusuko@gmail.com

#86
Posted 05/12/2017 06:47 PM   
[quote="masterotaku"][quote="sgsrules"] Is it possible to force the Separation value in automatic mode to zero?[/quote] You have asked that multiple times and I then asked "did you try setting StereoTextureEnable to 0x00000000 in its Nvidia profile" :p? Did you try it? At least some time ago it made Dolphin use 2x the resources of 2D instead of 4x (it was using 3D Vision Automatic on top of Dolphin's own 3D Vision). I want to know if it will make the game 2D with your injector or if it will work flawlessly.[/quote] [quote="sgsrules"][quote="masterotaku"][quote="sgsrules"] I set separation to zero. lmao! [/quote] Did you try the StereoTextureEnable setting? It should get rid of having to change separation, and GPU usage would be the same as in 2D.[/quote] StereoTextureEnable was already set to zero in the profile. I tried setting it again but it made no difference. [/quote];) [quote="masterotaku"] Apart from the Nvidia profile directly, you can also configure that setting with 3Dmigoto. [/quote] I checked all the stereo related settings in the profile and they're all set to zero except for 0x701EB457(MYSTERY STEREO SETTING) which has a value of 0x00000001. I looked at the 3dmigoto ini as well and unless i missed something I didn't see anything to about setting the default separation apart from applying hotkeys.
masterotaku said:
sgsrules said:
Is it possible to force the Separation value in automatic mode to zero?


You have asked that multiple times and I then asked "did you try setting StereoTextureEnable to 0x00000000 in its Nvidia profile" :p? Did you try it? At least some time ago it made Dolphin use 2x the resources of 2D instead of 4x (it was using 3D Vision Automatic on top of Dolphin's own 3D Vision).

I want to know if it will make the game 2D with your injector or if it will work flawlessly.

sgsrules said:
masterotaku said:
sgsrules said:
I set separation to zero. lmao!

Did you try the StereoTextureEnable setting? It should get rid of having to change separation, and GPU usage would be the same as in 2D.

StereoTextureEnable was already set to zero in the profile. I tried setting it again but it made no difference.
;)

masterotaku said:
Apart from the Nvidia profile directly, you can also configure that setting with 3Dmigoto.

I checked all the stereo related settings in the profile and they're all set to zero except for 0x701EB457(MYSTERY STEREO SETTING) which has a value of 0x00000001.

I looked at the 3dmigoto ini as well and unless i missed something I didn't see anything to about setting the default separation apart from applying hotkeys.

Like my work? You can send a donation via Paypal to sgs.rules@gmail.com

Windows 7 Pro 64x - Nvidia Driver 398.82 - EVGA 980Ti SC - Optoma HD26 with Edid override - 3D Vision 2 - i7-8700K CPU at 5.0Ghz - ASROCK Z370 Ext 4 Motherboard - 32 GB RAM Corsair Vengeance - 512 GB Samsung SSD 850 Pro - Creative Sound Blaster Z

#87
Posted 05/12/2017 07:19 PM   
@sgsrules The easy way is to add a preset like this: So when you enter the game press "P" to set separation to 0 (will be 1, is the lowest), and when you exit the game press "P" again to go back to the user separation. [code][Key_cycle_separation] Key = P type = cycle separation = 0[/code]
@sgsrules
The easy way is to add a preset like this:
So when you enter the game press "P" to set separation to 0 (will be 1, is the lowest), and when you exit the game press "P" again to go back to the user separation.

[Key_cycle_separation]
Key = P
type = cycle
separation = 0

MY WEB

Helix Mod - Making 3D Better

My 3D Screenshot Gallery

Like my fixes? you can donate to Paypal: dhr.donation@gmail.com

#88
Posted 05/12/2017 07:56 PM   
I know i could setup a hotkey for it. i wanted to know if there was a way to have a default sepration value that is game specific saved, like the convergence value is saved per profile.
I know i could setup a hotkey for it. i wanted to know if there was a way to have a default sepration value that is game specific saved, like the convergence value is saved per profile.

Like my work? You can send a donation via Paypal to sgs.rules@gmail.com

Windows 7 Pro 64x - Nvidia Driver 398.82 - EVGA 980Ti SC - Optoma HD26 with Edid override - 3D Vision 2 - i7-8700K CPU at 5.0Ghz - ASROCK Z370 Ext 4 Motherboard - 32 GB RAM Corsair Vengeance - 512 GB Samsung SSD 850 Pro - Creative Sound Blaster Z

#89
Posted 05/12/2017 09:39 PM   
[quote="sgsrules"];) [/quote] Note to myself: don't trust your own memory. That or I didn't see your answer that time :/. Another guess that might be right or wrong. I think that the separation setting isn't saved into Nvidia profiles. Only convergence. In the "d3dx.ini" file: [code] ;------------------------------------------------------------------------------------------------------ ; Shader manipulations without patches + shader filtering. ;------------------------------------------------------------------------------------------------------ ;[ShaderOverride1] ;Hash=69732c4f23cb6c48 ; Custom stereo separation value while rendering objects using this shader. ;Separation=0 ; Custom stereo convergence value while rendering objects using this ; shader (e.g. convergence=0 will move an object to infinity). ;Convergence=0 ; don't draw anything using this shader. ;Handling=skip [/code] I haven't tried that separation option. What would happen if you apply it to a shader that is always present? Does it use 0 separation for the effect, or to the whole game? I think bo3b or someone else mentioned that 3Dmigoto doesn't have situation dependant convergence and separation. But my memory... you know :p.
sgsrules said:;)


Note to myself: don't trust your own memory. That or I didn't see your answer that time :/.

Another guess that might be right or wrong. I think that the separation setting isn't saved into Nvidia profiles. Only convergence.

In the "d3dx.ini" file:

;------------------------------------------------------------------------------------------------------
; Shader manipulations without patches + shader filtering.
;------------------------------------------------------------------------------------------------------
;[ShaderOverride1]
;Hash=69732c4f23cb6c48
; Custom stereo separation value while rendering objects using this shader.
;Separation=0
; Custom stereo convergence value while rendering objects using this
; shader (e.g. convergence=0 will move an object to infinity).
;Convergence=0
; don't draw anything using this shader.
;Handling=skip


I haven't tried that separation option. What would happen if you apply it to a shader that is always present? Does it use 0 separation for the effect, or to the whole game? I think bo3b or someone else mentioned that 3Dmigoto doesn't have situation dependant convergence and separation.

But my memory... you know :p.

CPU: Intel Core i7 7700K @ 4.9GHz
Motherboard: Gigabyte Aorus GA-Z270X-Gaming 5
RAM: GSKILL Ripjaws Z 16GB 3866MHz CL18
GPU: MSI GeForce RTX 2080Ti Gaming X Trio
Monitor: Asus PG278QR
Speakers: Logitech Z506
Donations account: masterotakusuko@gmail.com

#90
Posted 05/12/2017 10:03 PM   
  6 / 11    
Scroll To Top