News:

One Minute Game Review by The Happy Friar: https://ugetube.com/@OneMinteGameReviews
Also on Rumble: https://rumble.com/c/c-1115371

idTech 4 (aka Doom 3 tech) Discord Server! https://discord.gg/9wtCGHa

Main Menu

Ambient cubemap support for sikkmod

Started by Zombie, October 20, 2014, 04:12:25 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

motorsep

All I can say is "wow". Before delving into the code, it would be nice to examine the assets. Shaders are written in Cg, and engine automatically converts Cg shaders into GLSL, HLSL and Cgb. The latter one isn't included as it's for PS3.

During runtime engine uses GLSL shaders.

nbohr1more

Point taken.

I do not own Doom 3 BFG so I based my presumptions on Fabien Sangarland's article since he seems reputable though I guess
he may not have spelled-out the whole chain of operations.

Nonetheless, (again) my code is exactly copying the bind operations in native BFG so it shouldn't matter if it's ARB, Cg, or GLIDE.
Same bind syntax == same execution.

motorsep

Quote from: nbohr1more on November 10, 2014, 12:48:48 PM
Nonetheless, (again) my code is exactly copying the bind operations in native BFG so it shouldn't matter if it's ARB, Cg, or GLIDE.
Same bind syntax == same execution.

That would be true for GLSL/HLSL, but not for ARB (not sure what GLIDE is). Doom 3 BFG doesn't work with ARB shaders and the entire rendering system is different from old idtech 4. We had a few things implemented for idTech 4 renderer and we had to rewrite it from ground up for BFG engine.

revelator

GLIDE  :o eh i sure hope its not the old voodoo glide api there refering to  ;D

nbohr1more

LOL, yes I was being hyperbolic there. I was tempted to use FORTRAN but really the point stands. I'm using the same bind
that the newstage parser uses. So if you can use any custom shaders without editing the source in BFG, my example should
work.

motorsep

Quote from: nbohr1more on November 11, 2014, 08:59:13 AM
LOL, yes I was being hyperbolic there. I was tempted to use FORTRAN but really the point stands. I'm using the same bind
that the newstage parser uses. So if you can use any custom shaders without editing the source in BFG, my example should
work.

I am not so sure. You are proposing several interactions per surface, and the engine maybe not designed to work that way. Just because you add a new global key to the light's material doesn't make it automatically work as long as you throw a custom interaction shader on it.

Why not to simply test it, nbohr1more, show the results, and only then throw code snippets around ? Grab original interaction shader, mess with it to make it do what you need, and then test it.

nbohr1more

Here are the facts:

1) RB-DOOM3-BFG already has multiple paths for interaction shaders depending what hardware support, light entity type, and keywords are used
and I followed the same orthodoxy to branch my path

2) RB-DOOM3-BFG uses the same bind command I used elsewhere in the code to bind user-made shaders in newstage

3) GL Program Bind commands don't take Cg

So, in conclusion. The native Cg shaders must be converted before they hit the bind portion. I am skipping that in my code
and taking the material defined GLSL and binding it directly.

Sure, I'd love to test all this out. Maybe I'll get Doom 3 BFG for X-mas and I'll be able to. I just made these changes because 7318 wanted them.
If Treb or another engine coder wants to clarify what I am doing wrong, I'm all ears. :)

motorsep

That's not what I am talking about. Interaction shader is the same for any and all lights. It's not about format in which shaders are made. It's not about custom shaders that can do various post process effects and stuff. There is only one surface-lighting-shadows interaction type shader.

nbohr1more

Look at my comments after "//" :



// select the render prog
if( lightShader->IsCustomLight()) // this is my (nbohr1more) change
{
renderProgManager.BindShader( newStage->glslProgram, newStage->glslProgram );
}
else
{
if( lightShader->IsAmbientLight() ) // this was already in Doom 3 BFG and is the same branch method we use in The Dark Mod. This literally means "if this light has the AmbientLight keyword, use this alternate interaction type)
{
if( surf->jointCache )
{
renderProgManager.BindShader_InteractionAmbientSkinned();
}
else
{
renderProgManager.BindShader_InteractionAmbient();
}
}
else
{
if( r_useShadowMapping.GetBool() && vLight->globalShadows )  // This was Robert's addition to toggle the light type depending on cvar
{
// RB: we have shadow mapping enabled and shadow maps so do a shadow compare
if( vLight->parallel )  // here's another change by Robert. It literally means "if this light entity is parallel, use this shader instead"
{
if( surf->jointCache )
{
renderProgManager.BindShader_Interaction_ShadowMapping_Parallel_Skinned();
}
else
{
renderProgManager.BindShader_Interaction_ShadowMapping_Parallel();
}
}
else if( vLight->pointLight )
{
if( surf->jointCache )
{
renderProgManager.BindShader_Interaction_ShadowMapping_Point_Skinned();
}
else
{
renderProgManager.BindShader_Interaction_ShadowMapping_Point();
}
}
else
{
if( surf->jointCache )
{
renderProgManager.BindShader_Interaction_ShadowMapping_Spot_Skinned();
}
else
{
renderProgManager.BindShader_Interaction_ShadowMapping_Spot();
}
}
}
else
{
if( surf->jointCache )
{
renderProgManager.BindShader_InteractionSkinned();
}
else
{
renderProgManager.BindShader_Interaction();
}
}
}
}



Does that make sense?

motorsep

#39
When you make an example out of it, then we can all see what you are trying to achieve. Right now all I see is you added a new custom key to the light material. I requires some hypothetical shader. What that shader will do, and how it will work with ambient and normal lights interactions shaders is not explained.

I admire the energy and all, but it just doesn't work like that with idTech 4 engines. Good example is shadow mapping in Doom 3 BFG. It's the slowest, almost impossible to optimized algorithm that was implemented into Doom 3 BFG. The reason I am saying that is that we took it, implemented into our engine fixing some of the poor decisions, optimized shader, optimized almost anything that could have been optimized and we still get ~22-30ms GPU drain on a simple test map on GF 670GTX. And the reason being that tr3b's method binds FBO 7 times! Darkplaces for example binds FBO only 2 times. All because cubemap for point lights needs to be rendered into atlas, not into 6 images.

Sure, when someone owns i7 CPU and R9/GF8xx or 9xx series GPUs, you don't feel it. But most of the people don't have that kind of PCs. So when someone just slaps something into idtech 4 / BFG engine without testing it, optimizing it accepting feedback from people, I get skeptical. Let's not just go out there and proclaim "I implemented this!", without testing it.

nbohr1more

These are branches, the engine doesn't bind all these at once it just binds the one that a specific light calls for or
binds all lights to one of many choices depending cvar or hardware boolean.

There's far more branching in the code than seen here so I would be skeptical that this would be performance impacting
but then again... drivers might not like this especially if they are poorly written and try to do shader replacement by detecting
the executable...

RB-DOOM3-BFG's performance probably has more to do with the vast amount of buffer data needed to create the shadow maps.
Most engines with shadow-mapping use large (stationary) scene covering lights which generate one or two distinct shadow maps
at most, not unique shadow maps for every object for multiple lights. The fact that he gets the performance he does alone is
a miracle.

oneofthe8devilz

#41
Quote from: motorsep on November 11, 2014, 11:41:39 AM
Good example is shadow mapping in Doom 3 BFG. It's the slowest, almost impossible to optimized algorithm that was implemented into Doom 3 BFG. The reason I am saying that is that we took it, implemented into our engine fixing some of the poor decisions, optimized shader, optimized almost anything that could have been optimized and we still get ~22-30ms GPU drain on a simple test map on GF 670GTX. And the reason being that tr3b's method binds FBO 7 times! Darkplaces for example binds FBO only 2 times. All because cubemap for point lights needs to be rendered into atlas, not into 6 images.

That is actually a very interesting topic that I always wanted to tap into as well (and it should probably get its own thread in order to avoid further derailing of this "ambient cubemap" one)

But just recently watching gameplay footage of the most recent to be released AAA titles (Far Cry 4, Dragon Age: Inquisition, Assassins Creed Unity and many more) I can't help but notice how "lame" the indoor visuals are and all the ssao shading in world cannot compensate for the lack of realtime shadow-casting pointlights within the indoor environments of those new AAA game titles...

It actually gives the "The Evil Within" dev-team even more value for what they have achieved with idtech5...

So it seems that after all the years of the big talk about how much more efficient GPU rendered shadowmapping is compared to the stencil shadows approach of idtech4... in the end when it comes to simple numbers... it looks like in a typical doom3 level scenario, stencil shadow pointlights outperform  the shadowmapped cubemapped pointlights by roughly 400%.

It pretty much reflects the performance results that I get on my machine when I switch between shadowmapping and stencil-shadows running the RBDOOM-3-BFG build. Plus the fact that the RBDOOM-3-BFG soft-shadows have really serious bias problems (maybe due to bad implementation ?) which result in light bleeding/shadows missing all over the place which cannot stand a chance compared to the properly calculated stencil shadows (things get worse for shadowmapping regarding "contact-shadows" the further away you look from the center of the shadow casting pointlight).

The "only" true advantages I see for shadow buffers over stencils are:
1. Control over the shadow penumbra (fast gpu based shadow-filtering of any kind is possible)
2. Shadow filtering through alpha tested shaders
3. Advanced LOD control of shadows

But in a typical Doom3 indoor scenario where you have your 75-300 shadow casting pointlights per level, shadowmapping should ALWAYS loose performance-wise versus stencilshadows...

Eventually it might be worth to think about a clever hybrid solution to get the best of both worlds but please feel free to correct me if I just wrote something fundamentally wrong since my graphics programming skills are literally non-existent...   
I got six little friends and they all run faster than you ;)


Check out our mods at
moddb or the SPS Homepage

nbohr1more

Yeah, the assets in Doom 3 weren't quite made for shadow-mapping but it may also be continuity issues which some of the
newer ARB clamp extensions fix (I'll have to add that to a tracker for RB-DOOM3-BFG). So, good shadows everywhere will require
you to trial 'n error geometry until this is fixed. :(

This is presuming you aren't getting code rot from the compiler "optimizing out" parts of the calculation like what happened to Revelator.

Yeah, for years folks have said Doom 3 would perform better with shadow maps and that was possibly true if they chose low resolution
maps like Dead Space but after all this time a bunch of the claimed superiority from other engines is still left in question. UE3 "can" do dynamic
lighting but commercial games ship with baked lighting, same with just about every other engine. The possible exception being Crytech but they
wear their heavy hardware requirements on their sleeve. Still, at least BFG has 10x the performance of vanilla :)


motorsep

The only reason you think shadow maps have no advantage in Doom 3 BFG (or in general) is because implementation is incorrect (rather it's technically correct, but unoptimized and the slowest that it can be).

If shadow mapping was implemented the way it is in Darkplaces engine or Teserract, performance difference would be quite noticeable.

Example: http://youtu.be/3fOpHRrOBJQ?t=2m38s

Shadow volumes depend on models/world polycount. So you can't have Gears of War like models / world in Doom 3 BFG. Shadow mapping is decoupled from polycount - you can have high poly models, and that will not affect performance on the shadowing side. That's why modern games all use shadow mapping.

motorsep

Quote from: nbohr1more on November 11, 2014, 01:43:27 PM
Yeah, the assets in Doom 3 weren't quite made for shadow-mapping

Please enlighten me what is "made for shadow mapping" for assets means. There is no such thing.