3Dmigoto now open-source...
  34 / 143    
[quote="bo3b"]I haven't looked at our memory usage in any detail, so correct me if I'm wrong, but I would not expect those structures to grow indefinitely. I'd expect them to grow only as big as every shader seen in a game, but not be leaking/growing.[/quote]Not exactly - a lot of those maps use the virtual address of the original shader as the key and are never freed, which means we can well keep increasing our memory well beyond what the game would do in some circumstances - think about what happens if the game frees a shader and later reallocates the same one, but it ends up at a different virtual address. I don't think memory usage is what is contributing to the 10 minute crash on Far Cry 4 - I've monitored RAM and VRAM usage and I don't see any significant increase leading up to the crash. It might well be contributing to the memory leaks on alt+tab, but I have not investigated those. What has me more concerned about these is that we are holding pointers to structures that the game has already freed. For the most part this should be fine because we shouldn't be dereferencing any unless the game is still using them, but holding invalid pointers is always risky. Of course changing this back to the old behaviour we still are holding invalid pointers, just less of them. I'm also concerned that some of those invalid pointers will later point to a different shader if they are freed and reallocated, which might lead to e.g. VertexShaderReplacementMap holding a replacement shader for a since released shader, but later on a different shader gets allocated and assigned the same virtual address... We don't clear it out of the replacement map, so the next time the game uses it we replace it with something totally inappropriate, which could definitely lead to a crash or at least graphical glitches.... I haven't been able to catch the 10 minute crash in a debugger (thanks to uplay), so I don't know exactly what is happening there, and I have only done one test with hunting=0 since changing these back, so I could have just got lucky (however, the 10 minute crash has otherwise been 100% reproducible, but I was playing the game a little more actively in that run, so I can't rule out that something else may have delayed it, like perhaps a loading screen). Plus we never saw this with 3DMigoto 1.0 even when hunting was enabled, so it seems there must be more to it. [quote]Just as an aside- does anyone have a preference for the way Helix did development, with a specific release version? We could of course adopt that model.[/quote]I think what we have is working and don't see any reason to change. [quote]That model would remove the need for the reload mechanism and on/off hunting and a few other bits.[/quote]I don't see how it would remove either of these - we'd still need to be able to reload shaders and config in the debug build, and just config in the production, plus I'd still want the on/off toggle in the debug build (heck, I even use the toggle in Helix Mod (print screen), but 3DMigoto's toggle is more useful).
bo3b said:I haven't looked at our memory usage in any detail, so correct me if I'm wrong, but I would not expect those structures to grow indefinitely. I'd expect them to grow only as big as every shader seen in a game, but not be leaking/growing.
Not exactly - a lot of those maps use the virtual address of the original shader as the key and are never freed, which means we can well keep increasing our memory well beyond what the game would do in some circumstances - think about what happens if the game frees a shader and later reallocates the same one, but it ends up at a different virtual address.

I don't think memory usage is what is contributing to the 10 minute crash on Far Cry 4 - I've monitored RAM and VRAM usage and I don't see any significant increase leading up to the crash. It might well be contributing to the memory leaks on alt+tab, but I have not investigated those.

What has me more concerned about these is that we are holding pointers to structures that the game has already freed. For the most part this should be fine because we shouldn't be dereferencing any unless the game is still using them, but holding invalid pointers is always risky. Of course changing this back to the old behaviour we still are holding invalid pointers, just less of them.

I'm also concerned that some of those invalid pointers will later point to a different shader if they are freed and reallocated, which might lead to e.g. VertexShaderReplacementMap holding a replacement shader for a since released shader, but later on a different shader gets allocated and assigned the same virtual address... We don't clear it out of the replacement map, so the next time the game uses it we replace it with something totally inappropriate, which could definitely lead to a crash or at least graphical glitches....

I haven't been able to catch the 10 minute crash in a debugger (thanks to uplay), so I don't know exactly what is happening there, and I have only done one test with hunting=0 since changing these back, so I could have just got lucky (however, the 10 minute crash has otherwise been 100% reproducible, but I was playing the game a little more actively in that run, so I can't rule out that something else may have delayed it, like perhaps a loading screen). Plus we never saw this with 3DMigoto 1.0 even when hunting was enabled, so it seems there must be more to it.

Just as an aside- does anyone have a preference for the way Helix did development, with a specific release version? We could of course adopt that model.
I think what we have is working and don't see any reason to change.

That model would remove the need for the reload mechanism and on/off hunting and a few other bits.
I don't see how it would remove either of these - we'd still need to be able to reload shaders and config in the debug build, and just config in the production, plus I'd still want the on/off toggle in the debug build (heck, I even use the toggle in Helix Mod (print screen), but 3DMigoto's toggle is more useful).

2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD

Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword

Posted 10/25/2015 01:04 PM   
I did not understand all the new features of the last versions, shall they help me to solve my problem ? In DCS some object (HUD and MFD) share same shaders and textures. I increased the HUD depth because it was not collimated at infinite, but unfortunately this will also increase depth of parts of MFD, making them unusable. I already used texture filtering, but as object shared the same texture, I am stuck. Is there a way to cope with this kind of situations ?
I did not understand all the new features of the last versions, shall they help me to solve my problem ?
In DCS some object (HUD and MFD) share same shaders and textures.
I increased the HUD depth because it was not collimated at infinite, but unfortunately this will also increase depth of parts of MFD, making them unusable. I already used texture filtering, but as object shared the same texture, I am stuck.
Is there a way to cope with this kind of situations ?

Posted 10/25/2015 06:04 PM   
I'm not sure what exactly you mean by "MFD" - is that some kind of screen inside the game? If so, what exactly is being adjusted - the screen, or the things on the screen? A screenshot might help to explain the problem. If it's a display on a screen in the game that is being adjusted, your best bet is render target size filtering so you only adjust the HUD when it's being drawn directly to the screen: [code] [Device] ... get_resolution_from = swap_chain ; If swap_chain does not result in the correct values for res_width and ; res_height when using a non-native resolution, try depth_stencil instead: ;get_resolution_from = depth_stencil [ShaderOverrideHUDMFD] ; Inject the current render target dimensions and screen resolution into the ; shader: Hash = xxxxxxxxxxx x1 = rt_width y1 = rt_height z1 = res_width w1 = res_height [/code] And in the shader: [code] // Only adjust UI depth when we are drawing to a render target that matches the // screen resolution: float4 rt_filter = IniParams.Load(int2(1,0)); if (rt_filter.x == rt_filter.z && rt_filter.y == rt_filter.w) { // Move to infinity: float4 stereo = StereoParams.Load(0); o0.x += stereo.x * stereo.y; } [/code] If the vertex shaders are shared but the pixel shaders are not, you can filter based on the partner shader (there's a few ways this can be achieved, this is the best way if you are using 3DMigoto 1.2.5 or later, as it is the most flexible and can work with any type of shader): [code] [ShaderOverrideHUDPixelShader] Hash = xxxxxxxx x = 1 post x = 0 [/code] Then in the vertex shader you can check IniParams.Load(0).x, which will be 1 if that pixel shader is active and 0 otherwise. I don't think this is your problem, but if it's that you need to separate out textures beyond what can be done with regular texture filtering (e.g. if a game uses a single texture containing a number of icons or HUD elements and you only want to match one), you can check the texture coordinates so long as the texture does not change during the game: [code] // If v1 is the texture coordinate... if (v1.x >= 0.7 && v1.x <= 0.75 && v1.y >= 0.2 && v1.y <= 0.3) { // texture coordinates match, move to infinity: float4 stereo = StereoParams.Load(0); o0.x += stereo.x * stereo.y; } [/code] You can use either trial & error to find the coordinates, or frame analysis to dump out the texture and work it out from there.
I'm not sure what exactly you mean by "MFD" - is that some kind of screen inside the game? If so, what exactly is being adjusted - the screen, or the things on the screen? A screenshot might help to explain the problem.

If it's a display on a screen in the game that is being adjusted, your best bet is render target size filtering so you only adjust the HUD when it's being drawn directly to the screen:
[Device]

...

get_resolution_from = swap_chain
; If swap_chain does not result in the correct values for res_width and
; res_height when using a non-native resolution, try depth_stencil instead:
;get_resolution_from = depth_stencil

[ShaderOverrideHUDMFD]
; Inject the current render target dimensions and screen resolution into the
; shader:
Hash = xxxxxxxxxxx
x1 = rt_width
y1 = rt_height
z1 = res_width
w1 = res_height


And in the shader:
// Only adjust UI depth when we are drawing to a render target that matches the
// screen resolution:
float4 rt_filter = IniParams.Load(int2(1,0));
if (rt_filter.x == rt_filter.z && rt_filter.y == rt_filter.w) {
// Move to infinity:
float4 stereo = StereoParams.Load(0);
o0.x += stereo.x * stereo.y;
}



If the vertex shaders are shared but the pixel shaders are not, you can filter based on the partner shader (there's a few ways this can be achieved, this is the best way if you are using 3DMigoto 1.2.5 or later, as it is the most flexible and can work with any type of shader):
[ShaderOverrideHUDPixelShader]
Hash = xxxxxxxx
x = 1
post x = 0


Then in the vertex shader you can check IniParams.Load(0).x, which will be 1 if that pixel shader is active and 0 otherwise.


I don't think this is your problem, but if it's that you need to separate out textures beyond what can be done with regular texture filtering (e.g. if a game uses a single texture containing a number of icons or HUD elements and you only want to match one), you can check the texture coordinates so long as the texture does not change during the game:

// If v1 is the texture coordinate...
if (v1.x >= 0.7 && v1.x <= 0.75 && v1.y >= 0.2 && v1.y <= 0.3) {
// texture coordinates match, move to infinity:
float4 stereo = StereoParams.Load(0);
o0.x += stereo.x * stereo.y;
}


You can use either trial & error to find the coordinates, or frame analysis to dump out the texture and work it out from there.

2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD

Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword

Posted 10/26/2015 02:18 AM   
Thanks for your answers. MFD means "Multi Function Display". They are screens in cockpit displaying informations or camera view of weapon or pods. Here is some expamples of the problem I want to fix : [list] [.]MFD with same increased depth than HUD : [url]https://shar.es/1uShW9[/url] [/.] [.]cokpit object (digital clock in lower left part) with same increased depth than HUD : [url]https://shar.es/1uShvc[/url] [/.] [/list] The vertexshader, pixelshader and texture are the same for the HUD,the MFD and the clock. I unsuccessfully tried to use render target size filtering but not as the way you wrote above, so I will try it again, it may help for MFD problem. I also tried the texture coordinates for the clock, but only with X component, so the only thing I obtained was texture warping. I will try with both x & y as you advise me.
Thanks for your answers.
MFD means "Multi Function Display". They are screens in cockpit displaying informations or camera view of weapon or pods.
Here is some expamples of the problem I want to fix :



The vertexshader, pixelshader and texture are the same for the HUD,the MFD and the clock.
I unsuccessfully tried to use render target size filtering but not as the way you wrote above, so I will try it again, it may help for MFD problem.
I also tried the texture coordinates for the clock, but only with X component, so the only thing I obtained was texture warping. I will try with both x & y as you advise me.

Posted 10/26/2015 09:06 PM   
@lefuneste I think the best solution is the last solution posted by DarkStarSword, you can define a big square only when you want the fix is apply, see screenshot: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/66700/[/img] The cross indicate the coordinates. ONly in that texture apply this code, like this: [code]if (o0.y > -0.1) { Fixing code here }[/code] been "o0" the position. Also there is a chance that you need to normalize the position coordinate. In case the negatives values are taken as positive. you can try with (o0.y > 0), the middle of the screen horinzontal is "0"...should work fine also.
@lefuneste

I think the best solution is the last solution posted by DarkStarSword, you can define a big square only when you want the fix is apply, see screenshot:

Image

The cross indicate the coordinates.


ONly in that texture apply this code, like this:

if (o0.y > -0.1)
{
Fixing code here
}



been "o0" the position. Also there is a chance that you need to normalize the position coordinate. In case the negatives values are taken as positive. you can try with (o0.y > 0), the middle of the screen horinzontal is "0"...should work fine also.
Attachments

Example.jpg

MY WEB

Helix Mod - Making 3D Better

My 3D Screenshot Gallery

Like my fixes? you can donate to Paypal: dhr.donation@gmail.com

Posted 10/26/2015 09:33 PM   
The target size filtering is not working, even with depth_stencil option. I managed to solve 2 problems with the texture coordinates, but unfortunately I was not able to discriminate the texture used for the MFD and the HUD. It looks like they are the same... I can not use o0 coordinate filtering, because you can move your virtual head all around the cockpit and zoom, so MFD will not have the same place on screen... Anyway fix is not perfect, but things are far better than default. I will publish the fix in Helix blog after more testing.
The target size filtering is not working, even with depth_stencil option. I managed to solve 2 problems with the texture coordinates, but unfortunately I was not able to discriminate the texture used for the MFD and the HUD. It looks like they are the same...
I can not use o0 coordinate filtering, because you can move your virtual head all around the cockpit and zoom, so MFD will not have the same place on screen...
Anyway fix is not perfect, but things are far better than default.
I will publish the fix in Helix blog after more testing.

Posted 10/30/2015 09:46 PM   
@bo3b / mike / DarkStarSword I move the comments from FAllout 4 thread in here, because i related to the wrapper and is affecting at least 3 particular games: Fallout 4, Cod BO3 and Batman AK. 1) Related to the crash in Fallout [quote="DarkStarSword"]Ok, VS2015 can stop FC4 in the debugger, so at last I have a stack trace of the 10 minute crash: [code]ntdll.dll!RtlpWaitOnCriticalSection() Unknown ntdll.dll!RtlEnterCriticalSection() Unknown dxgi.dll!CDXGISwapChain::GetDesc(struct DXGI_SWAP_CHAIN_DESC *) Unknown d3d11.dll!HackerDXGISwapChain::GetDesc(DXGI_SWAP_CHAIN_DESC * pDesc) Line 1448 C++ FC64.dll!000007fedbd72f7a() Unknown[/code] So, looks like something has gone wrong with the original swap chain and we crashed on a NULL pointer dereference in the lock (maybe it's been released prematurely?) - I'll keep investigating.[/quote] 2) Related to "ld_structured_indexable" instruction issue [quote="DHR"]Also there is another possible bug with the "ld_structured_indexable" instruction....there are possible shaders that are not dumped or can't be hunted because of this issue..... Of course i can be wrong. My theory is shaders with "ld_structured_indexable" instructions with a warning or a not critical error, are dumped in shadercache or anothers also can be hunted normally with NUMPAD and dumped. But the ones with critical error, can't be dumped or hunted. I don't know if 3Dmigoto is preventing to dump ASM shaders, if detect some problem with the HLSL shader. Result will be not dumping ASM and HLSL. Happens in Black Ops 3 and Fallout 4, both games with shaders with the "ld_structured_indexable" instruction.[/quote] [quote="mike_ar69"]Also Batman Arkham Knight - I am convinced as well. I am at work so can't provide screenshots, but in Fallout 4 there are lighting effects that have two different shaders which kick in at different angles/distances from the lights. So if you fix one of them it looks fine, but as you move away from the light, or change angle, all of a sudden it looks broken again because the second shader kicks in (and the first one disables) that has not been fixed. For most of these, I used a sledgehammer approach of bulk fixing all shaders with the right pattern, and I fixed most of these issues. But a few remain. So on the assumption that I missed them in bulk fixing, I try and hunt them manually - but there is absolutely no VS or PS that has any effect on the broken light shaders. I rechecked my bulk fix and even manually went through the shadercache - no more shadow/light shaders left to fix. I have discussed this with bo3b before regarding Batman Arkham Knight but I was hoping if both yourself (Darkstarsword) and Bo3b could look at this and maybe find out if DHR's theory is correct - i.e. if there is a shader that has a critical error, the wrapper does it's best not to fail by simply catching the exception ignoring that shader? I will also try the "verbose" log to see if anything else is logged, but that gets massive and is hard to parse through. I do want to point out as well that a number of HLSL shaders *are* created with the ld_indexable stuff in, they just don't compile properly or load - this was bo3b's originally solution to at least make sure they don't crash the game. I fix these using the ASM files. (Note to self - check if there are ASM files without HLSL files in the shadercache...).[/quote]
@bo3b / mike / DarkStarSword

I move the comments from FAllout 4 thread in here, because i related to the wrapper and is affecting at least 3 particular games: Fallout 4, Cod BO3 and Batman AK.

1) Related to the crash in Fallout

DarkStarSword said:Ok, VS2015 can stop FC4 in the debugger, so at last I have a stack trace of the 10 minute crash:
ntdll.dll!RtlpWaitOnCriticalSection()	Unknown
ntdll.dll!RtlEnterCriticalSection() Unknown
dxgi.dll!CDXGISwapChain::GetDesc(struct DXGI_SWAP_CHAIN_DESC *) Unknown
d3d11.dll!HackerDXGISwapChain::GetDesc(DXGI_SWAP_CHAIN_DESC * pDesc) Line 1448 C++
FC64.dll!000007fedbd72f7a() Unknown



So, looks like something has gone wrong with the original swap chain and we crashed on a NULL pointer dereference in the lock (maybe it's been released prematurely?) - I'll keep investigating.


2) Related to "ld_structured_indexable" instruction issue

DHR said:Also there is another possible bug with the "ld_structured_indexable" instruction....there are possible shaders that are not dumped or can't be hunted because of this issue..... Of course i can be wrong.

My theory is shaders with "ld_structured_indexable" instructions with a warning or a not critical error, are dumped in shadercache or anothers also can be hunted normally with NUMPAD and dumped. But the ones with critical error, can't be dumped or hunted.

I don't know if 3Dmigoto is preventing to dump ASM shaders, if detect some problem with the HLSL shader. Result will be not dumping ASM and HLSL.

Happens in Black Ops 3 and Fallout 4, both games with shaders with the "ld_structured_indexable" instruction.


mike_ar69 said:Also Batman Arkham Knight - I am convinced as well.

I am at work so can't provide screenshots, but in Fallout 4 there are lighting effects that have two different shaders which kick in at different angles/distances from the lights. So if you fix one of them it looks fine, but as you move away from the light, or change angle, all of a sudden it looks broken again because the second shader kicks in (and the first one disables) that has not been fixed. For most of these, I used a sledgehammer approach of bulk fixing all shaders with the right pattern, and I fixed most of these issues. But a few remain. So on the assumption that I missed them in bulk fixing, I try and hunt them manually - but there is absolutely no VS or PS that has any effect on the broken light shaders. I rechecked my bulk fix and even manually went through the shadercache - no more shadow/light shaders left to fix.

I have discussed this with bo3b before regarding Batman Arkham Knight but I was hoping if both yourself (Darkstarsword) and Bo3b could look at this and maybe find out if DHR's theory is correct - i.e. if there is a shader that has a critical error, the wrapper does it's best not to fail by simply catching the exception ignoring that shader?

I will also try the "verbose" log to see if anything else is logged, but that gets massive and is hard to parse through.

I do want to point out as well that a number of HLSL shaders *are* created with the ld_indexable stuff in, they just don't compile properly or load - this was bo3b's originally solution to at least make sure they don't crash the game. I fix these using the ASM files. (Note to self - check if there are ASM files without HLSL files in the shadercache...).

MY WEB

Helix Mod - Making 3D Better

My 3D Screenshot Gallery

Like my fixes? you can donate to Paypal: dhr.donation@gmail.com

Posted 11/12/2015 05:24 PM   
I take a look at fallout 4 log, and found this: This is a successful dumping The shader is dumped in ASM and HLSL [code]HackerDevice::CreatePixelShader called with BytecodeLength = 2796, handle = 000007FEEEA74120, ClassLinkage = 0000000000000000 bytecode hash = f324a8bf156d077a shader registered for possible reloading: f324a8bf156d077a_ps as bin returns result = 0, handle = 0000000051178418[/code] This case i have an *****Exception decompiling shader**** This shader is only dumped as ASM, not HLSL [code]ackerDevice::CreatePixelShader called with BytecodeLength = 14324, handle = 000007FEEEA3EEE0, ClassLinkage = 0000000000000000 bytecode hash = f2a8f7adfbcbda45 creating HLSL representation. ******* Exception caught while decompiling shader ****** error while decompiling. shader registered for possible reloading: f2a8f7adfbcbda45_ps as bin returns result = 0, handle = 0000000051178318[/code] Also there are a lot of "hash_number_ds" type of shaders with the same exception. At the end of the log, there are 38 of this ***Exception***, without hash number or anything...maybe those 38 shaders are not dumped or can't be hunted? it's an error calculating the hash [code]******* Exception caught while calculating crc32c_hw hash ****** [/code] Same happens in CoD: BO3, but here are more than 38 with an additional message: [code]failed result = 80004002 for 000000006BC7FC00 [/code]
I take a look at fallout 4 log, and found this:

This is a successful dumping
The shader is dumped in ASM and HLSL
HackerDevice::CreatePixelShader called with BytecodeLength = 2796, handle = 000007FEEEA74120, ClassLinkage = 0000000000000000
bytecode hash = f324a8bf156d077a
shader registered for possible reloading: f324a8bf156d077a_ps as bin
returns result = 0, handle = 0000000051178418



This case i have an *****Exception decompiling shader****
This shader is only dumped as ASM, not HLSL
ackerDevice::CreatePixelShader called with BytecodeLength = 14324, handle = 000007FEEEA3EEE0, ClassLinkage = 0000000000000000
bytecode hash = f2a8f7adfbcbda45
creating HLSL representation.
******* Exception caught while decompiling shader ******
error while decompiling.
shader registered for possible reloading: f2a8f7adfbcbda45_ps as bin
returns result = 0, handle = 0000000051178318


Also there are a lot of "hash_number_ds" type of shaders with the same exception.

At the end of the log, there are 38 of this ***Exception***, without hash number or anything...maybe those 38 shaders are not dumped or can't be hunted? it's an error calculating the hash
******* Exception caught while calculating crc32c_hw hash ******


Same happens in CoD: BO3, but here are more than 38 with an additional message:
failed result = 80004002 for 000000006BC7FC00

MY WEB

Helix Mod - Making 3D Better

My 3D Screenshot Gallery

Like my fixes? you can donate to Paypal: dhr.donation@gmail.com

Posted 11/12/2015 05:37 PM   
@DHR - thanks for pulling this together. I will stay in this thread and out of the Fallout 4 thread as well. I will update my batch thing to look at ASM files, not HLSL, to see if I can identify any more lighting shaders. I have fixed about 10 of them already in ASM, so it's a piece of cake when I have them.
@DHR - thanks for pulling this together. I will stay in this thread and out of the Fallout 4 thread as well.

I will update my batch thing to look at ASM files, not HLSL, to see if I can identify any more lighting shaders. I have fixed about 10 of them already in ASM, so it's a piece of cake when I have them.

Rig: Intel i7-8700K @4.7GHz, 16Gb Ram, SSD, GTX 1080Ti, Win10x64, Asus VG278

Posted 11/12/2015 07:18 PM   
Thanks for summarizing those pieces. I can say with assurance that the problem of not being able to hunt shaders is not related to the Decompiler problems, and also not the exceptions when decompiling. We always tend to dump the entire shader to ShaderCache, but that's not required. If that part is disabled, then you can still hunt shaders with nothing in ShaderCache nor ShaderFixes. It uses the original binary objects to do hunting there. In the example above, that's what the "registered for possible reloading ... as bin" means. Even though it couldn't parse the file properly, it's still in the list of shaders for hunting. Always possible there is a bug in some path there of course, so as a double check you might want to try to hunt the mystery shader with no dumping enabled. As I understand the problem, you never even see the shader wink out while hunting, it's skipped. Another thing to try there is the marking_mode=pink, as I think there are scenarios where having them go blank doesn't change anything on screen. (I know for sure I had this happen in Psychonauts with HelixMod). This is actually why I added pink mode. The exceptions caught while calculating the crc32c hash are possibly related, and I'll double check that code path. If it gets an exception there, it should just be marked with a texture/index buffer hash as zero, and should not impact the vertex or pixel shader hunting. On the ld_structured_indexable problem, these also should have no ability to block shader hunting, as they only happen upon shader dumping, either to ShaderCache or after Mark. The goal there was to get them out of causing exceptions, even though it could not create proper code, and give us something we can hand fix. After some study, I'm pretty sure I've worked out how to generate proper code for ld_structured_indexable, and I'll be adding this as time permits. It's fairly complicated so it's going to take me at least a week or two. I'm going to focus upon the missing hunting first though. Not sure about the new exceptions you are seeing, I had all exceptions fixed, so must be a new shader instruction or something with this game. Generally I can fix that sort of problem quickly, so I'll fix that this weekend.
Thanks for summarizing those pieces.

I can say with assurance that the problem of not being able to hunt shaders is not related to the Decompiler problems, and also not the exceptions when decompiling.

We always tend to dump the entire shader to ShaderCache, but that's not required. If that part is disabled, then you can still hunt shaders with nothing in ShaderCache nor ShaderFixes. It uses the original binary objects to do hunting there.

In the example above, that's what the "registered for possible reloading ... as bin" means. Even though it couldn't parse the file properly, it's still in the list of shaders for hunting.

Always possible there is a bug in some path there of course, so as a double check you might want to try to hunt the mystery shader with no dumping enabled. As I understand the problem, you never even see the shader wink out while hunting, it's skipped. Another thing to try there is the marking_mode=pink, as I think there are scenarios where having them go blank doesn't change anything on screen. (I know for sure I had this happen in Psychonauts with HelixMod). This is actually why I added pink mode.


The exceptions caught while calculating the crc32c hash are possibly related, and I'll double check that code path. If it gets an exception there, it should just be marked with a texture/index buffer hash as zero, and should not impact the vertex or pixel shader hunting.


On the ld_structured_indexable problem, these also should have no ability to block shader hunting, as they only happen upon shader dumping, either to ShaderCache or after Mark. The goal there was to get them out of causing exceptions, even though it could not create proper code, and give us something we can hand fix.

After some study, I'm pretty sure I've worked out how to generate proper code for ld_structured_indexable, and I'll be adding this as time permits. It's fairly complicated so it's going to take me at least a week or two. I'm going to focus upon the missing hunting first though.

Not sure about the new exceptions you are seeing, I had all exceptions fixed, so must be a new shader instruction or something with this game. Generally I can fix that sort of problem quickly, so I'll fix that this weekend.

Acer H5360 (1280x720@120Hz) - ASUS VG248QE with GSync mod - 3D Vision 1&2 - Driver 372.54
GTX 970 - i5-4670K@4.2GHz - 12GB RAM - Win7x64+evilKB2670838 - 4 Disk X25 RAID
SAGER NP9870-S - GTX 980 - i7-6700K - Win10 Pro 1607
Latest 3Dmigoto Release
Bo3b's School for ShaderHackers

Posted 11/13/2015 01:49 AM   
Thanks bo3b for the response....i will try to hunt the shaders i can't find in CoD:BO3 using marking_mode=pink But i'm sure is not listed when i hunt (VS or PS or CS)...is a lensflare light type that is over the half of the screen, if i'm close. anything you need to test, let me know.
Thanks bo3b for the response....i will try to hunt the shaders i can't find in CoD:BO3 using marking_mode=pink

But i'm sure is not listed when i hunt (VS or PS or CS)...is a lensflare light type that is over the half of the screen, if i'm close.

anything you need to test, let me know.

MY WEB

Helix Mod - Making 3D Better

My 3D Screenshot Gallery

Like my fixes? you can donate to Paypal: dhr.donation@gmail.com

Posted 11/13/2015 10:40 AM   
I have an update. It's generally good news, though it leaves me not knowing the solution yet. I found a *compute* shader that is responsible for the bad light. I can't quite work out how I found it, I disabled lots of fixes to put the game back to looking like crap then it seemed to be easier to identify. So - this maybe takes the pressure off the "hunting is not working" problem, but I don't want to rule it out, as I still can't identify any VS for the light. Now it turns out I had autofixed all the compute shaders already, but whereas the fix puts the lights in the correct place, there is a residual (and incredibly prominent) clipping effect. This might be a scissorrect thing I am not sure. I feel like I need a VS to sort this out but there is no VS that has any effect on the light. Also, the compute shader HLSL is wrong, so I am having to use the ASM and am having an issue creating a temporary variable, but that's probably because I have not done ASM for a while now. Still - I have a shader, and that is something to work on :-)
I have an update. It's generally good news, though it leaves me not knowing the solution yet. I found a *compute* shader that is responsible for the bad light. I can't quite work out how I found it, I disabled lots of fixes to put the game back to looking like crap then it seemed to be easier to identify.

So - this maybe takes the pressure off the "hunting is not working" problem, but I don't want to rule it out, as I still can't identify any VS for the light.

Now it turns out I had autofixed all the compute shaders already, but whereas the fix puts the lights in the correct place, there is a residual (and incredibly prominent) clipping effect. This might be a scissorrect thing I am not sure. I feel like I need a VS to sort this out but there is no VS that has any effect on the light. Also, the compute shader HLSL is wrong, so I am having to use the ASM and am having an issue creating a temporary variable, but that's probably because I have not done ASM for a while now. Still - I have a shader, and that is something to work on :-)

Rig: Intel i7-8700K @4.7GHz, 16Gb Ram, SSD, GTX 1080Ti, Win10x64, Asus VG278

Posted 11/13/2015 03:00 PM   
In my case with Black Ops 3, i try: 1) Mark mode = pink, not positive results hunting VS / PS / CS 2) I check again all VS / PS / CS with mark mode = skip...nothing 3) Also i do a batch search and fix with ASM files (15 shaders with the pattern), there are like 328 VS in ASM and 97 in HLSL...but still the shader i want to found is missing :(
In my case with Black Ops 3, i try:

1) Mark mode = pink, not positive results hunting VS / PS / CS
2) I check again all VS / PS / CS with mark mode = skip...nothing
3) Also i do a batch search and fix with ASM files (15 shaders with the pattern), there are like 328 VS in ASM and 97 in HLSL...but still the shader i want to found is missing

:(

MY WEB

Helix Mod - Making 3D Better

My 3D Screenshot Gallery

Like my fixes? you can donate to Paypal: dhr.donation@gmail.com

Posted 11/13/2015 09:37 PM   
I have identified the source of my "clipping" issue - its a game bug. I found a site where this effect is reported in 2D, in widescreen resolutions. There is an ini setting to correct it.
I have identified the source of my "clipping" issue - its a game bug. I found a site where this effect is reported in 2D, in widescreen resolutions. There is an ini setting to correct it.

Rig: Intel i7-8700K @4.7GHz, 16Gb Ram, SSD, GTX 1080Ti, Win10x64, Asus VG278

Posted 11/14/2015 06:07 AM   
Hi, I have another problem with DCS. The horizon has a dark color only for one eye, and this eye is changing regarding the roll angle of the plane. At 0° of roll there is no artifact. [url=https://shar.es/159R9u]Here is the problem [/url] I found two shaders that may cause the problem: [url=https://shar.es/159PQW]23eb49961f704621[/url] [code]void main( uint v0 : SV_VertexID0, out float4 o0 : SV_POSITION0, out float2 o1 : TEXCOORD0) { float4 r0; uint4 bitmask, uiDest; float4 fDest; o0.zw = float2(0,1); r0.xy = (int2)v0.xx & int2(1,2); o0.xy = r0.xy ? float2(1,1) : float2(-1,-1); o1.xy = r0.xy ? float2(1,0) : float2(0,1); return; }[/code] and [url=https://shar.es/159Mqt]bda5788465aff6b0[/url] [code]cbuffer _Globals : register(b0) { float4x4 viewProjInverse : packoffset(c0); } Texture2D<float4> StereoParams : register(t125); Texture1D<float4> IniParams : register(t120); void main( uint v0 : SV_VertexID0, out float4 o0 : SV_POSITION0, out float4 o1 : TEXCOORD0, out float3 o2 : TEXCOORD1) { const float4 icb[] = { { -1.000000, -1.000000, 0, 0}, { 1.000000, -1.000000, 0, 0}, { -1.000000, 1.000000, 0, 0}, { 1.000000, 1.000000, 0, 0} }; float4 r0; uint4 bitmask, uiDest; float4 fDest; o0.zw = float2(0,1); r0.x = v0.x; o0.xy = icb[r0.x+0].xy; o1.xy = icb[r0.x+0].xy * float2(0.5,0.5) + float2(0.5,0.5); r0.xy = icb[r0.x+0].xy; r0.z = 1; o2.x = dot(r0.xyz, viewProjInverse._m00_m10_m30); o2.y = dot(r0.xyz, viewProjInverse._m01_m11_m31); o2.z = dot(r0.xyz, viewProjInverse._m02_m12_m32); return; }[/code] It seems that the second shader is causing the problem, the first one looks more like additional effects. The o2 output should be linked to the problem. I tried some things with r0 stereoization for o2 but it didn't work, I can define a fix for a positive roll angle of the plane, but when I roll on the other side the problem is showing again. The more the roll angle is, the wider the dark area is. If you can help...
Hi, I have another problem with DCS.
The horizon has a dark color only for one eye, and this eye is changing regarding the roll angle of the plane. At 0° of roll there is no artifact.
Here is the problem
I found two shaders that may cause the problem:

23eb49961f704621
void main( 
uint v0 : SV_VertexID0,
out float4 o0 : SV_POSITION0,
out float2 o1 : TEXCOORD0)
{
float4 r0;
uint4 bitmask, uiDest;
float4 fDest;

o0.zw = float2(0,1);
r0.xy = (int2)v0.xx & int2(1,2);
o0.xy = r0.xy ? float2(1,1) : float2(-1,-1);
o1.xy = r0.xy ? float2(1,0) : float2(0,1);
return;
}


and bda5788465aff6b0
cbuffer _Globals : register(b0)
{
float4x4 viewProjInverse : packoffset(c0);
}

Texture2D<float4> StereoParams : register(t125);
Texture1D<float4> IniParams : register(t120);

void main(
uint v0 : SV_VertexID0,
out float4 o0 : SV_POSITION0,
out float4 o1 : TEXCOORD0,
out float3 o2 : TEXCOORD1)
{
const float4 icb[] = { { -1.000000, -1.000000, 0, 0},
{ 1.000000, -1.000000, 0, 0},
{ -1.000000, 1.000000, 0, 0},
{ 1.000000, 1.000000, 0, 0} };
float4 r0;
uint4 bitmask, uiDest;
float4 fDest;

o0.zw = float2(0,1);
r0.x = v0.x;
o0.xy = icb[r0.x+0].xy;
o1.xy = icb[r0.x+0].xy * float2(0.5,0.5) + float2(0.5,0.5);
r0.xy = icb[r0.x+0].xy;
r0.z = 1;
o2.x = dot(r0.xyz, viewProjInverse._m00_m10_m30);
o2.y = dot(r0.xyz, viewProjInverse._m01_m11_m31);
o2.z = dot(r0.xyz, viewProjInverse._m02_m12_m32);
return;
}


It seems that the second shader is causing the problem, the first one looks more like additional effects.
The o2 output should be linked to the problem.
I tried some things with r0 stereoization for o2 but it didn't work, I can define a fix for a positive roll angle of the plane, but when I roll on the other side the problem is showing again. The more the roll angle is, the wider the dark area is.
If you can help...

Posted 11/14/2015 11:10 AM   
  34 / 143    
Scroll To Top