[quote="helifax"]Hi Bo3b, I was wondering if there is something I can do to decrease the time it takes the wrapper to dump the shaders?
In any frostbyte 3 game it takes 15 minutes to load a game... I am currently only using "export_hlsl=2" option.
Nomally it dumps around 20k shaders when a level loads...
If you can think of anything I can do to decrease this insane time, please let me know!
Cheers![/quote]
I haven't profiled this part of the code, but I don't think there is anything to be done there, at least not easily. The code does an unbelievable amount of string management, and I'm sure it could be improved. As a general rule though, I put most of my effort into fixing Decompiler bugs. When I get back to that code, I'll do a quick profile to see if there is any obvious quick fix, but it'll be awhile before I can get to it.
It is worth taking a look at the log though. No need for full debug, but check for shaders that are causing exceptions in particular. Exceptions are super expensive on Windows apps, so if it's happening a lot that would be bad.
Also, if you don't need the CS/DS/HS it might be worth commenting out that section in the code, as I know those tend to decompile poorly. It's setup for VS2013. If you are setup for doing one-off builds, you could edit the DecompileHLSL.cpp file.
Let me know if there seems to be a lot of errors in the log, or wasted effort that is obvious.
helifax said:Hi Bo3b, I was wondering if there is something I can do to decrease the time it takes the wrapper to dump the shaders?
In any frostbyte 3 game it takes 15 minutes to load a game... I am currently only using "export_hlsl=2" option.
Nomally it dumps around 20k shaders when a level loads...
If you can think of anything I can do to decrease this insane time, please let me know!
Cheers!
I haven't profiled this part of the code, but I don't think there is anything to be done there, at least not easily. The code does an unbelievable amount of string management, and I'm sure it could be improved. As a general rule though, I put most of my effort into fixing Decompiler bugs. When I get back to that code, I'll do a quick profile to see if there is any obvious quick fix, but it'll be awhile before I can get to it.
It is worth taking a look at the log though. No need for full debug, but check for shaders that are causing exceptions in particular. Exceptions are super expensive on Windows apps, so if it's happening a lot that would be bad.
Also, if you don't need the CS/DS/HS it might be worth commenting out that section in the code, as I know those tend to decompile poorly. It's setup for VS2013. If you are setup for doing one-off builds, you could edit the DecompileHLSL.cpp file.
Let me know if there seems to be a lot of errors in the log, or wasted effort that is obvious.
Acer H5360 (1280x720@120Hz) - ASUS VG248QE with GSync mod - 3D Vision 1&2 - Driver 372.54
GTX 970 - i5-4670K@4.2GHz - 12GB RAM - Win7x64+evilKB2670838 - 4 Disk X25 RAID
SAGER NP9870-S - GTX 980 - i7-6700K - Win10 Pro 1607 Latest 3Dmigoto Release Bo3b's School for ShaderHackers
[quote="bo3b"][quote="helifax"]Hi Bo3b, I was wondering if there is something I can do to decrease the time it takes the wrapper to dump the shaders?
In any frostbyte 3 game it takes 15 minutes to load a game... I am currently only using "export_hlsl=2" option.
Nomally it dumps around 20k shaders when a level loads...
If you can think of anything I can do to decrease this insane time, please let me know!
Cheers![/quote]
I haven't profiled this part of the code, but I don't think there is anything to be done there, at least not easily. The code does an unbelievable amount of string management, and I'm sure it could be improved. As a general rule though, I put most of my effort into fixing Decompiler bugs. When I get back to that code, I'll do a quick profile to see if there is any obvious quick fix, but it'll be awhile before I can get to it.
It is worth taking a look at the log though. No need for full debug, but check for shaders that are causing exceptions in particular. Exceptions are super expensive on Windows apps, so if it's happening a lot that would be bad.
Also, if you don't need the CS/DS/HS it might be worth commenting out that section in the code, as I know those tend to decompile poorly. It's setup for VS2013. If you are setup for doing one-off builds, you could edit the DecompileHLSL.cpp file.
Let me know if there seems to be a lot of errors in the log, or wasted effort that is obvious.[/quote]
Thx Bo3b,
I did look in the log and saw a couple of exceptions. After which I realised that the HLSL had decompiling issues and I switched to ASM shaders.
The dumping now is instant (as there is no string parsing involved), so this issue is "solved" in this case;)
I can definitely see where the complication and time consumption comes from though^_^
Big thank you again for all the help!!!
(I am writing some scripts for the FrostByte3 engine - for future use) and having to filter through 100-200k shaders is definitely something that can be done manually;) But switching to ASM solves this issue;) So, I'll stick with it ^_^
helifax said:Hi Bo3b, I was wondering if there is something I can do to decrease the time it takes the wrapper to dump the shaders?
In any frostbyte 3 game it takes 15 minutes to load a game... I am currently only using "export_hlsl=2" option.
Nomally it dumps around 20k shaders when a level loads...
If you can think of anything I can do to decrease this insane time, please let me know!
Cheers!
I haven't profiled this part of the code, but I don't think there is anything to be done there, at least not easily. The code does an unbelievable amount of string management, and I'm sure it could be improved. As a general rule though, I put most of my effort into fixing Decompiler bugs. When I get back to that code, I'll do a quick profile to see if there is any obvious quick fix, but it'll be awhile before I can get to it.
It is worth taking a look at the log though. No need for full debug, but check for shaders that are causing exceptions in particular. Exceptions are super expensive on Windows apps, so if it's happening a lot that would be bad.
Also, if you don't need the CS/DS/HS it might be worth commenting out that section in the code, as I know those tend to decompile poorly. It's setup for VS2013. If you are setup for doing one-off builds, you could edit the DecompileHLSL.cpp file.
Let me know if there seems to be a lot of errors in the log, or wasted effort that is obvious.
Thx Bo3b,
I did look in the log and saw a couple of exceptions. After which I realised that the HLSL had decompiling issues and I switched to ASM shaders.
The dumping now is instant (as there is no string parsing involved), so this issue is "solved" in this case;)
I can definitely see where the complication and time consumption comes from though^_^
Big thank you again for all the help!!!
(I am writing some scripts for the FrostByte3 engine - for future use) and having to filter through 100-200k shaders is definitely something that can be done manually;) But switching to ASM solves this issue;) So, I'll stick with it ^_^
1x Palit RTX 2080Ti Pro Gaming OC(watercooled and overclocked to hell)
3x 3D Vision Ready Asus VG278HE monitors (5760x1080).
Intel i9 9900K (overclocked to 5.3 and watercooled ofc).
Asus Maximus XI Hero Mobo.
16 GB Team Group T-Force Dark Pro DDR4 @ 3600.
Lots of Disks:
- Raid 0 - 256GB Sandisk Extreme SSD.
- Raid 0 - WD Black - 2TB.
- SanDisk SSD PLUS 480 GB.
- Intel 760p 256GB M.2 PCIe NVMe SSD.
Creative Sound Blaster Z.
Windows 10 x64 Pro.
etc
[quote="helifax"]
@mx-2:
- Thanks for that code. Didn't try it yet, but it definitely makes sense;) Big thanks for your reply!!![/quote]
I tested my inverse matrix code and fixed some bugs. If you (or anybody else) likes to try my code in a fix, the updated version can be found on [url=https://github.com/mx-2/3d-fix/blob/master/_tools_/inverseMatrix.asm]github[/url].
helifax said:
@mx-2:
- Thanks for that code. Didn't try it yet, but it definitely makes sense;) Big thanks for your reply!!!
I tested my inverse matrix code and fixed some bugs. If you (or anybody else) likes to try my code in a fix, the updated version can be found on github.
[quote="helifax"]
1) The HLSL for Matrix Inversion works;) but 3DMigoto just decides to make a low beep when I put in the HASH of the Compute shader. So it doesn't work with the game. Doesn't like that HASH for some reason no matter what I do.
Thus, I had to put the matrix inverse in the shader code :)
2)The CS not working IS NOT A DRIVER ISSUE. Actually is the CS shaders that needs fixing! I haven't made FULL fix for them, but I see where things go wrong! Believe it or not, but the driver is actually working as it should;) It only affects the LEFT eye. What we know is that for Left eye we say "(-1) * separation". I expect that the CS doesn't like the NEGATIVE value of the position and just discards it or does weird thing with it!
@DHR:
I managed to hack it to some degree, but is not a proper fix;)
Sadly, I don't know what much about Compute shaders in 3D Vision. I know DSS is the expert as he always helped me before with them. If you want we can try to see what is wrong, but without a proper understand I don't think we can come up with the true formula;)
[/quote]
I see you make a inverse matrix in ASM....the important thing is working (inject or direct code). The CS have also a clipping in the left that need fixing.....with more time i will take a look if the CS need to be fixed in another spot.
helifax said:
1) The HLSL for Matrix Inversion works;) but 3DMigoto just decides to make a low beep when I put in the HASH of the Compute shader. So it doesn't work with the game. Doesn't like that HASH for some reason no matter what I do.
Thus, I had to put the matrix inverse in the shader code :)
2)The CS not working IS NOT A DRIVER ISSUE. Actually is the CS shaders that needs fixing! I haven't made FULL fix for them, but I see where things go wrong! Believe it or not, but the driver is actually working as it should;) It only affects the LEFT eye. What we know is that for Left eye we say "(-1) * separation". I expect that the CS doesn't like the NEGATIVE value of the position and just discards it or does weird thing with it!
@DHR:
I managed to hack it to some degree, but is not a proper fix;)
Sadly, I don't know what much about Compute shaders in 3D Vision. I know DSS is the expert as he always helped me before with them. If you want we can try to see what is wrong, but without a proper understand I don't think we can come up with the true formula;)
I see you make a inverse matrix in ASM....the important thing is working (inject or direct code). The CS have also a clipping in the left that need fixing.....with more time i will take a look if the CS need to be fixed in another spot.
I added the code snippets to the wiki for future reference. Thanks to mx-2 and Helifax for sharing the code.
[url]http://wiki.bo3b.net/index.php?title=Canonical_Stereo_Code[/url]
I included the preferred approach, which is to use DarkStarSwords matrix inversion compute shader. This is preferred because it only runs once per frame, not every vertex, or worse, every pixel.
If you can, move any inversion code out of Pixel Shaders. Running inversion code every pixel is costly, and not generally necessary.
If you are using HelixMod, it has the built in CPU based matrix inversions, which run once per frame, and should be used there. [url]http://wiki.bo3b.net/index.php?title=HelixMod_Feature_List#InverseMatrix.2C_InverseMatrix1[/url]
I included the preferred approach, which is to use DarkStarSwords matrix inversion compute shader. This is preferred because it only runs once per frame, not every vertex, or worse, every pixel.
If you can, move any inversion code out of Pixel Shaders. Running inversion code every pixel is costly, and not generally necessary.
[quote="bo3b"]If you can, move any inversion code out of Pixel Shaders. Running inversion code every pixel is costly, and not generally necessary.[/quote]
Interesting. So lets say I need to do an inversion of a VPM in a pixel shader (fairly common for shadows), would it be worthwhile for me to actually include the matrix.hlsl shader into the VS, perform the inversion there, and then pass the inverted matrix through to the PS through some texcoord outputs? Or does this only apply if we're not using the custom shader, and actually performing the long route via calculating the inversion inside of the shader?
bo3b said:If you can, move any inversion code out of Pixel Shaders. Running inversion code every pixel is costly, and not generally necessary.
Interesting. So lets say I need to do an inversion of a VPM in a pixel shader (fairly common for shadows), would it be worthwhile for me to actually include the matrix.hlsl shader into the VS, perform the inversion there, and then pass the inverted matrix through to the PS through some texcoord outputs? Or does this only apply if we're not using the custom shader, and actually performing the long route via calculating the inversion inside of the shader?
3D Gaming Rig: CPU: i7 7700K @ 4.9Ghz | Mobo: Asus Maximus Hero VIII | RAM: Corsair Dominator 16GB | GPU: 2 x GTX 1080 Ti SLI | 3xSSDs for OS and Apps, 2 x HDD's for 11GB storage | PSU: Seasonic X-1250 M2| Case: Corsair C70 | Cooling: Corsair H115i Hydro cooler | Displays: Asus PG278QR, BenQ XL2420TX & BenQ HT1075 | OS: Windows 10 Pro + Windows 7 dual boot
I've tried to fix the skybox in the original Deus Ex, as well as some just disabling some fake lights.
However, there's only like 5 VS, and 5 PS. So when I toggle though the skybox also selects geometry not in the skybox. (This is with helixmod).
Is this because it's a DX7 or 8 game that's converted to DX9 using GMDX?
Also, what is a good game to learn to fix with 3dmigoto DX11. I just want basically use someones fix as a tool to see how they fixed the game while I refix it.
If that makes sense.
Thanks.
I've tried to fix the skybox in the original Deus Ex, as well as some just disabling some fake lights.
However, there's only like 5 VS, and 5 PS. So when I toggle though the skybox also selects geometry not in the skybox. (This is with helixmod).
Is this because it's a DX7 or 8 game that's converted to DX9 using GMDX?
Also, what is a good game to learn to fix with 3dmigoto DX11. I just want basically use someones fix as a tool to see how they fixed the game while I refix it.
[quote="DJ-RK"][quote="bo3b"]If you can, move any inversion code out of Pixel Shaders. Running inversion code every pixel is costly, and not generally necessary.[/quote]
Interesting. So lets say I need to do an inversion of a VPM in a pixel shader (fairly common for shadows), would it be worthwhile for me to actually include the matrix.hlsl shader into the VS, perform the inversion there, and then pass the inverted matrix through to the PS through some texcoord outputs? Or does this only apply if we're not using the custom shader, and actually performing the long route via calculating the inversion inside of the shader?[/quote]
As always, without actually measuring it, we can't say for sure if it makes a difference. If we are CPU bound for example, then this would not matter at all. But, if we are in a scenario where we expect to be GPU bound, it might matter. Even in that case though, it's going to depend upon what the game is doing as to whether this matters.
However, we know that in at least some scenarios it will impact frame rate, so unless there is a compelling reason, we would want to move this code out of that inner loop of pixels.
So, putting the inversion in the VS and passing it as a texcoord would be better, because that is only run for every vertex, not every pixel. Probably, unless for some bizarre game reason there are more vertices than pixels.
The way to determine whether this matters is to check your performance both with the fix active, and without. If there is no measurable impact on frame-rate, then it doesn't matter. Keep everything as close to the same as possible, just comment out the fixing code, but keep the shader active, or tie it to a hot-key to see an on/off live.
Something close you can try that is super easy would be to use the F9 'show original', which will disable all the fixes while held down. If that showed an impact, it would be worth digging further.
bo3b said:If you can, move any inversion code out of Pixel Shaders. Running inversion code every pixel is costly, and not generally necessary.
Interesting. So lets say I need to do an inversion of a VPM in a pixel shader (fairly common for shadows), would it be worthwhile for me to actually include the matrix.hlsl shader into the VS, perform the inversion there, and then pass the inverted matrix through to the PS through some texcoord outputs? Or does this only apply if we're not using the custom shader, and actually performing the long route via calculating the inversion inside of the shader?
As always, without actually measuring it, we can't say for sure if it makes a difference. If we are CPU bound for example, then this would not matter at all. But, if we are in a scenario where we expect to be GPU bound, it might matter. Even in that case though, it's going to depend upon what the game is doing as to whether this matters.
However, we know that in at least some scenarios it will impact frame rate, so unless there is a compelling reason, we would want to move this code out of that inner loop of pixels.
So, putting the inversion in the VS and passing it as a texcoord would be better, because that is only run for every vertex, not every pixel. Probably, unless for some bizarre game reason there are more vertices than pixels.
The way to determine whether this matters is to check your performance both with the fix active, and without. If there is no measurable impact on frame-rate, then it doesn't matter. Keep everything as close to the same as possible, just comment out the fixing code, but keep the shader active, or tie it to a hot-key to see an on/off live.
Something close you can try that is super easy would be to use the F9 'show original', which will disable all the fixes while held down. If that showed an impact, it would be worth digging further.
Acer H5360 (1280x720@120Hz) - ASUS VG248QE with GSync mod - 3D Vision 1&2 - Driver 372.54
GTX 970 - i5-4670K@4.2GHz - 12GB RAM - Win7x64+evilKB2670838 - 4 Disk X25 RAID
SAGER NP9870-S - GTX 980 - i7-6700K - Win10 Pro 1607 Latest 3Dmigoto Release Bo3b's School for ShaderHackers
[quote="ishiki"]I've tried to fix the skybox in the original Deus Ex, as well as some just disabling some fake lights.
However, there's only like 5 VS, and 5 PS. So when I toggle though the skybox also selects geometry not in the skybox. (This is with helixmod).
Is this because it's a DX7 or 8 game that's converted to DX9 using GMDX?
Also, what is a good game to learn to fix with 3dmigoto DX11. I just want basically use someones fix as a tool to see how they fixed the game while I refix it.
If that makes sense.
Thanks.[/quote]
That's not too surprising for an older game like original Deus Ex. Having a single shader that does multiple jobs happens a lot with older games. I don't think this is related to the conversion using GMDX, but it's possible.
Maybe take a look at the HelixMod feature to make alternate shaders: [url]http://wiki.bo3b.net/index.php?title=HelixMod_Feature_List#Overriding_an_Individual_Instance_of_a_Shader[/url]
There may be a way for you to separate those pieces out to different shaders, so you can fix just the skybox.
For DX11 example games, there are a lot. Take a look at the two big repositories of game fixes:
3Dmigoto: https://github.com/bo3b/3Dmigoto
DarkStarSword: https://github.com/DarkStarSword/3d-fixes
DarkStarSword fixes tend to be better commented, and the GitHub check-in comments can also be very helpful. The flip side is that DarkStarSword is also a genius, so his fixes also tend to do really unbelievably complex things that are hard to understand.
Skimming the list of games there, the two I'd recommend starting with are these two that are complete fixes, but don't include much in the way of advanced features and other weird game complexity.
https://github.com/bo3b/3Dmigoto/tree/master/Alien
https://github.com/bo3b/3Dmigoto/tree/master/Mordor
Incomplete, but more straightforward.
https://github.com/DarkStarSword/3d-fixes/tree/master/ABZU
More complicated, using texture overrides, but fairly clear:
https://github.com/DarkStarSword/3d-fixes/tree/master/Mad%20Max
ishiki said:I've tried to fix the skybox in the original Deus Ex, as well as some just disabling some fake lights.
However, there's only like 5 VS, and 5 PS. So when I toggle though the skybox also selects geometry not in the skybox. (This is with helixmod).
Is this because it's a DX7 or 8 game that's converted to DX9 using GMDX?
Also, what is a good game to learn to fix with 3dmigoto DX11. I just want basically use someones fix as a tool to see how they fixed the game while I refix it.
If that makes sense.
Thanks.
That's not too surprising for an older game like original Deus Ex. Having a single shader that does multiple jobs happens a lot with older games. I don't think this is related to the conversion using GMDX, but it's possible.
DarkStarSword fixes tend to be better commented, and the GitHub check-in comments can also be very helpful. The flip side is that DarkStarSword is also a genius, so his fixes also tend to do really unbelievably complex things that are hard to understand.
Skimming the list of games there, the two I'd recommend starting with are these two that are complete fixes, but don't include much in the way of advanced features and other weird game complexity.
Hi all,
while working on my Risen 3 fix, I noticed that ambient occlusion shadows look slightly different in both eyes when looking from a non perpendicular angle on the shadowed surface. However, they are basically on the correct depth so that modifying the shaders makes it worse. This results in some highlighted or "floating" shadows.
I also saw this issue in the Witcher 3 (with fix) HBAO+ shadows so I believe it is unfixable. Am I correct? If yes, can somebody please explain me the reason for this.
while working on my Risen 3 fix, I noticed that ambient occlusion shadows look slightly different in both eyes when looking from a non perpendicular angle on the shadowed surface. However, they are basically on the correct depth so that modifying the shaders makes it worse. This results in some highlighted or "floating" shadows.
I also saw this issue in the Witcher 3 (with fix) HBAO+ shadows so I believe it is unfixable. Am I correct? If yes, can somebody please explain me the reason for this.
[quote="mx-2"]Hi all,
while working on my Risen 3 fix, I noticed that ambient occlusion shadows look slightly different in both eyes when looking from a non perpendicular angle on the shadowed surface. However, they are basically on the correct depth so that modifying the shaders makes it worse. This results in some highlighted or "floating" shadows.
I also saw this issue in the Witcher 3 (with fix) HBAO+ shadows so I believe it is unfixable. Am I correct? If yes, can somebody please explain me the reason for this.[/quote]
Read this thread: https://forums.geforce.com/default/topic/897529/3d-hbao-normal-map-artefact-fix
Which will lead you here: https://github.com/bo3b/3Dmigoto/commit/6fcb2354435b866d3697b588c982d3f44c0ec552
It seems very hard to do. I don't know if the same concept can be applied to SSAO. It certainly would help my Dark Souls 2 and Skyrim SE fixes.
By the way, I need help here with Skyrim Special Edition, in case someone that can help didn't read that thread:
https://forums.geforce.com/default/topic/969381/3d-vision/the-elder-scrolls-v-skyrim-special-edition/post/5008074/#5008074
while working on my Risen 3 fix, I noticed that ambient occlusion shadows look slightly different in both eyes when looking from a non perpendicular angle on the shadowed surface. However, they are basically on the correct depth so that modifying the shaders makes it worse. This results in some highlighted or "floating" shadows.
I also saw this issue in the Witcher 3 (with fix) HBAO+ shadows so I believe it is unfixable. Am I correct? If yes, can somebody please explain me the reason for this.
I found the correct shader which looks similar to the linked github shader and tried to fix it with DarkStarSwords pattern. Based on the shader headers, I guess that this game uses SSAO not HBAO+.
The first correction works and improves it a bit but then the shader multiplies something with a 3x3 view matrix and the fixes don't work for the other texture samples. Sometimes they need a positive, sometimes a negative fix to improve it a bit but it does not result in a complete fix.
I tried different things like multiplying the fix with the matrix as well but without success. Does anybody has some ideas?
It also seems that the fix needs a distance dependency but I'm not sure with this. This effect may be caused by the errornous correction.
// Below fixes dont work very well:
// Some need positive correction, some need negative correction,
// some need no correction for best (but not completely fixed) results
texld r31, c200.z, s13
mul r28.x, r5.z, c5.y
add r31.w, -r31.y, r28.x
mul r31.w, r31.w, r31.x
mul r31.w, r31.w, c200.w
add r6.x, r6.x, -r31.w
I remember DSS making a python script for HelixMod that allows you to put different screenshots in different modes, like seen here:
http://helixmod.blogspot.co.uk/2016/11/event0.html
Yet, know when I search for it, I can't find it:(( I was thinking, maybe add a link to it in HelixMod at guide section? :)
In the meantime PLEASE can anyone point me the web-page where I can find it?
Thank you!
Yet, know when I search for it, I can't find it:(( I was thinking, maybe add a link to it in HelixMod at guide section? :)
In the meantime PLEASE can anyone point me the web-page where I can find it?
Thank you!
1x Palit RTX 2080Ti Pro Gaming OC(watercooled and overclocked to hell)
3x 3D Vision Ready Asus VG278HE monitors (5760x1080).
Intel i9 9900K (overclocked to 5.3 and watercooled ofc).
Asus Maximus XI Hero Mobo.
16 GB Team Group T-Force Dark Pro DDR4 @ 3600.
Lots of Disks:
- Raid 0 - 256GB Sandisk Extreme SSD.
- Raid 0 - WD Black - 2TB.
- SanDisk SSD PLUS 480 GB.
- Intel 760p 256GB M.2 PCIe NVMe SSD.
Creative Sound Blaster Z.
Windows 10 x64 Pro.
etc
This is the post you are looking for: [url]https://forums.geforce.com/default/topic/766890/3d-vision/bo3bs-school-for-shaderhackers/post/4923723/#4923723[/url]
@mx-2 my main experience with AO shaders has been that there are often a lot of places that need to be adjusted, and it will often look much worse then when you started until they are all adjusted simultaneously when suddenly it all looks perfect. I've fixed SSAO shaders in a couple of Unity games and as I recall it was along the same lines as fixing HBAO+, but there are more variations of SSAO than there are of HBAO+, so the details may vary.
What game is that shader from?
@mx-2 my main experience with AO shaders has been that there are often a lot of places that need to be adjusted, and it will often look much worse then when you started until they are all adjusted simultaneously when suddenly it all looks perfect. I've fixed SSAO shaders in a couple of Unity games and as I recall it was along the same lines as fixing HBAO+, but there are more variations of SSAO than there are of HBAO+, so the details may vary.
What game is that shader from?
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
I haven't profiled this part of the code, but I don't think there is anything to be done there, at least not easily. The code does an unbelievable amount of string management, and I'm sure it could be improved. As a general rule though, I put most of my effort into fixing Decompiler bugs. When I get back to that code, I'll do a quick profile to see if there is any obvious quick fix, but it'll be awhile before I can get to it.
It is worth taking a look at the log though. No need for full debug, but check for shaders that are causing exceptions in particular. Exceptions are super expensive on Windows apps, so if it's happening a lot that would be bad.
Also, if you don't need the CS/DS/HS it might be worth commenting out that section in the code, as I know those tend to decompile poorly. It's setup for VS2013. If you are setup for doing one-off builds, you could edit the DecompileHLSL.cpp file.
Let me know if there seems to be a lot of errors in the log, or wasted effort that is obvious.
Acer H5360 (1280x720@120Hz) - ASUS VG248QE with GSync mod - 3D Vision 1&2 - Driver 372.54
GTX 970 - i5-4670K@4.2GHz - 12GB RAM - Win7x64+evilKB2670838 - 4 Disk X25 RAID
SAGER NP9870-S - GTX 980 - i7-6700K - Win10 Pro 1607
Latest 3Dmigoto Release
Bo3b's School for ShaderHackers
Thx Bo3b,
I did look in the log and saw a couple of exceptions. After which I realised that the HLSL had decompiling issues and I switched to ASM shaders.
The dumping now is instant (as there is no string parsing involved), so this issue is "solved" in this case;)
I can definitely see where the complication and time consumption comes from though^_^
Big thank you again for all the help!!!
(I am writing some scripts for the FrostByte3 engine - for future use) and having to filter through 100-200k shaders is definitely something that can be done manually;) But switching to ASM solves this issue;) So, I'll stick with it ^_^
1x Palit RTX 2080Ti Pro Gaming OC(watercooled and overclocked to hell)
3x 3D Vision Ready Asus VG278HE monitors (5760x1080).
Intel i9 9900K (overclocked to 5.3 and watercooled ofc).
Asus Maximus XI Hero Mobo.
16 GB Team Group T-Force Dark Pro DDR4 @ 3600.
Lots of Disks:
- Raid 0 - 256GB Sandisk Extreme SSD.
- Raid 0 - WD Black - 2TB.
- SanDisk SSD PLUS 480 GB.
- Intel 760p 256GB M.2 PCIe NVMe SSD.
Creative Sound Blaster Z.
Windows 10 x64 Pro.
etc
My website with my fixes and OpenGL to 3D Vision wrapper:
http://3dsurroundgaming.com
(If you like some of the stuff that I've done and want to donate something, you can do it with PayPal at tavyhome@gmail.com)
I tested my inverse matrix code and fixed some bugs. If you (or anybody else) likes to try my code in a fix, the updated version can be found on github.
My 3D fixes with Helixmod for the Risen series on GitHub
Bo3b's School for Shaderhackers - starting point for your first 3D fix
I see you make a inverse matrix in ASM....the important thing is working (inject or direct code). The CS have also a clipping in the left that need fixing.....with more time i will take a look if the CS need to be fixed in another spot.
MY WEB
Helix Mod - Making 3D Better
My 3D Screenshot Gallery
Like my fixes? you can donate to Paypal: dhr.donation@gmail.com
http://wiki.bo3b.net/index.php?title=Canonical_Stereo_Code
I included the preferred approach, which is to use DarkStarSwords matrix inversion compute shader. This is preferred because it only runs once per frame, not every vertex, or worse, every pixel.
If you can, move any inversion code out of Pixel Shaders. Running inversion code every pixel is costly, and not generally necessary.
If you are using HelixMod, it has the built in CPU based matrix inversions, which run once per frame, and should be used there. http://wiki.bo3b.net/index.php?title=HelixMod_Feature_List#InverseMatrix.2C_InverseMatrix1
Acer H5360 (1280x720@120Hz) - ASUS VG248QE with GSync mod - 3D Vision 1&2 - Driver 372.54
GTX 970 - i5-4670K@4.2GHz - 12GB RAM - Win7x64+evilKB2670838 - 4 Disk X25 RAID
SAGER NP9870-S - GTX 980 - i7-6700K - Win10 Pro 1607
Latest 3Dmigoto Release
Bo3b's School for ShaderHackers
Interesting. So lets say I need to do an inversion of a VPM in a pixel shader (fairly common for shadows), would it be worthwhile for me to actually include the matrix.hlsl shader into the VS, perform the inversion there, and then pass the inverted matrix through to the PS through some texcoord outputs? Or does this only apply if we're not using the custom shader, and actually performing the long route via calculating the inversion inside of the shader?
3D Gaming Rig: CPU: i7 7700K @ 4.9Ghz | Mobo: Asus Maximus Hero VIII | RAM: Corsair Dominator 16GB | GPU: 2 x GTX 1080 Ti SLI | 3xSSDs for OS and Apps, 2 x HDD's for 11GB storage | PSU: Seasonic X-1250 M2| Case: Corsair C70 | Cooling: Corsair H115i Hydro cooler | Displays: Asus PG278QR, BenQ XL2420TX & BenQ HT1075 | OS: Windows 10 Pro + Windows 7 dual boot
Like my fixes? Dontations can be made to: www.paypal.me/DShanz or rshannonca@gmail.com
Like electronic music? Check out: www.soundcloud.com/dj-ryan-king
However, there's only like 5 VS, and 5 PS. So when I toggle though the skybox also selects geometry not in the skybox. (This is with helixmod).
Is this because it's a DX7 or 8 game that's converted to DX9 using GMDX?
Also, what is a good game to learn to fix with 3dmigoto DX11. I just want basically use someones fix as a tool to see how they fixed the game while I refix it.
If that makes sense.
Thanks.
I'm ishiki, forum screwed up my name.
9900K @5.0 GHZ, 16GBDDR4@4233MHZ, 2080 Ti
As always, without actually measuring it, we can't say for sure if it makes a difference. If we are CPU bound for example, then this would not matter at all. But, if we are in a scenario where we expect to be GPU bound, it might matter. Even in that case though, it's going to depend upon what the game is doing as to whether this matters.
However, we know that in at least some scenarios it will impact frame rate, so unless there is a compelling reason, we would want to move this code out of that inner loop of pixels.
So, putting the inversion in the VS and passing it as a texcoord would be better, because that is only run for every vertex, not every pixel. Probably, unless for some bizarre game reason there are more vertices than pixels.
The way to determine whether this matters is to check your performance both with the fix active, and without. If there is no measurable impact on frame-rate, then it doesn't matter. Keep everything as close to the same as possible, just comment out the fixing code, but keep the shader active, or tie it to a hot-key to see an on/off live.
Something close you can try that is super easy would be to use the F9 'show original', which will disable all the fixes while held down. If that showed an impact, it would be worth digging further.
Acer H5360 (1280x720@120Hz) - ASUS VG248QE with GSync mod - 3D Vision 1&2 - Driver 372.54
GTX 970 - i5-4670K@4.2GHz - 12GB RAM - Win7x64+evilKB2670838 - 4 Disk X25 RAID
SAGER NP9870-S - GTX 980 - i7-6700K - Win10 Pro 1607
Latest 3Dmigoto Release
Bo3b's School for ShaderHackers
That's not too surprising for an older game like original Deus Ex. Having a single shader that does multiple jobs happens a lot with older games. I don't think this is related to the conversion using GMDX, but it's possible.
Maybe take a look at the HelixMod feature to make alternate shaders: http://wiki.bo3b.net/index.php?title=HelixMod_Feature_List#Overriding_an_Individual_Instance_of_a_Shader
There may be a way for you to separate those pieces out to different shaders, so you can fix just the skybox.
For DX11 example games, there are a lot. Take a look at the two big repositories of game fixes:
3Dmigoto: https://github.com/bo3b/3Dmigoto
DarkStarSword: https://github.com/DarkStarSword/3d-fixes
DarkStarSword fixes tend to be better commented, and the GitHub check-in comments can also be very helpful. The flip side is that DarkStarSword is also a genius, so his fixes also tend to do really unbelievably complex things that are hard to understand.
Skimming the list of games there, the two I'd recommend starting with are these two that are complete fixes, but don't include much in the way of advanced features and other weird game complexity.
https://github.com/bo3b/3Dmigoto/tree/master/Alien
https://github.com/bo3b/3Dmigoto/tree/master/Mordor
Incomplete, but more straightforward.
https://github.com/DarkStarSword/3d-fixes/tree/master/ABZU
More complicated, using texture overrides, but fairly clear:
https://github.com/DarkStarSword/3d-fixes/tree/master/Mad%20Max
Acer H5360 (1280x720@120Hz) - ASUS VG248QE with GSync mod - 3D Vision 1&2 - Driver 372.54
GTX 970 - i5-4670K@4.2GHz - 12GB RAM - Win7x64+evilKB2670838 - 4 Disk X25 RAID
SAGER NP9870-S - GTX 980 - i7-6700K - Win10 Pro 1607
Latest 3Dmigoto Release
Bo3b's School for ShaderHackers
while working on my Risen 3 fix, I noticed that ambient occlusion shadows look slightly different in both eyes when looking from a non perpendicular angle on the shadowed surface. However, they are basically on the correct depth so that modifying the shaders makes it worse. This results in some highlighted or "floating" shadows.
I also saw this issue in the Witcher 3 (with fix) HBAO+ shadows so I believe it is unfixable. Am I correct? If yes, can somebody please explain me the reason for this.
My 3D fixes with Helixmod for the Risen series on GitHub
Bo3b's School for Shaderhackers - starting point for your first 3D fix
Read this thread: https://forums.geforce.com/default/topic/897529/3d-hbao-normal-map-artefact-fix
Which will lead you here: https://github.com/bo3b/3Dmigoto/commit/6fcb2354435b866d3697b588c982d3f44c0ec552
It seems very hard to do. I don't know if the same concept can be applied to SSAO. It certainly would help my Dark Souls 2 and Skyrim SE fixes.
By the way, I need help here with Skyrim Special Edition, in case someone that can help didn't read that thread:
https://forums.geforce.com/default/topic/969381/3d-vision/the-elder-scrolls-v-skyrim-special-edition/post/5008074/#5008074
CPU: Intel Core i7 7700K @ 4.9GHz
Motherboard: Gigabyte Aorus GA-Z270X-Gaming 5
RAM: GSKILL Ripjaws Z 16GB 3866MHz CL18
GPU: MSI GeForce RTX 2080Ti Gaming X Trio
Monitor: Asus PG278QR
Speakers: Logitech Z506
Donations account: masterotakusuko@gmail.com
Thank you for the link, masterotaku.
I found the correct shader which looks similar to the linked github shader and tried to fix it with DarkStarSwords pattern. Based on the shader headers, I guess that this game uses SSAO not HBAO+.
The first correction works and improves it a bit but then the shader multiplies something with a 3x3 view matrix and the fixes don't work for the other texture samples. Sometimes they need a positive, sometimes a negative fix to improve it a bit but it does not result in a complete fix.
I tried different things like multiplying the fix with the matrix as well but without success. Does anybody has some ideas?
It also seems that the fix needs a distance dependency but I'm not sure with this. This effect may be caused by the errornous correction.
PS83BFC6D9:
My 3D fixes with Helixmod for the Risen series on GitHub
Bo3b's School for Shaderhackers - starting point for your first 3D fix
http://helixmod.blogspot.co.uk/2016/11/event0.html
Yet, know when I search for it, I can't find it:(( I was thinking, maybe add a link to it in HelixMod at guide section? :)
In the meantime PLEASE can anyone point me the web-page where I can find it?
Thank you!
1x Palit RTX 2080Ti Pro Gaming OC(watercooled and overclocked to hell)
3x 3D Vision Ready Asus VG278HE monitors (5760x1080).
Intel i9 9900K (overclocked to 5.3 and watercooled ofc).
Asus Maximus XI Hero Mobo.
16 GB Team Group T-Force Dark Pro DDR4 @ 3600.
Lots of Disks:
- Raid 0 - 256GB Sandisk Extreme SSD.
- Raid 0 - WD Black - 2TB.
- SanDisk SSD PLUS 480 GB.
- Intel 760p 256GB M.2 PCIe NVMe SSD.
Creative Sound Blaster Z.
Windows 10 x64 Pro.
etc
My website with my fixes and OpenGL to 3D Vision wrapper:
http://3dsurroundgaming.com
(If you like some of the stuff that I've done and want to donate something, you can do it with PayPal at tavyhome@gmail.com)
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
What game is that shader from?
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword