Viewport in Particle Editor broken after changing texture

[caedes]

I'm working on bringing the Windows-only MFC tools to dhewm3 (see https://github.com/DanielGibson/dhewm3/tree/tools if you're interested) - it's based on the older dhewm3-based SteelStorm2 code, so thanks to motorsep for giving me that code!

Seems to work overall, however there is an issue with the Particle Editor that also exists with the original 1.3.1 binaries (at least on my System: Win10, Intel Haswell CPU with Iris Pro GPU):
When selecting a different  texture (and clicking "Ok" or whatever you click to close the texture selection window), a new small viewport turns up in the lower left corner of the window and is used exclusively (the rest of the window doesn't seem to really get updated anymore), looks like this:

Has anyone encountered this before? Were you able to implement a fix or at least workaround?

[The Happy Friar]

I don't remember a workaround when that would happen to me.  Maybe I didn't care, maybe I had a key bound to "reloadengine" and that fixed it.    It didn't happen often though, only very rarerly.

[caedes]

Happens for me every time, I just:
1. start the testmaps/test_box map
2. start the particle editor (editParticles)
3. click the material selection ([...]) button
4. select a different texture
5. click ok (or whatever it's called, I forgot)
6. it looks like on the screenshot I posted

In Doom3 a workaround seems to be to switch to a non-Doom3 window and back to Doom3, for dhewm3 that doesn't work (vid_restart does though).
My first guess is that somehow the wrong GL context is selected after closing the material selection window..

I'm still hoping someone already fixed that in their own port of the engine so I don't have to get familiar with MFC and the editor code and figure it out myself - that code is not really pretty on first sight...

[motorsep]

I don't remember having this bug at all (not in BFG tools at least) :/

[caedes]

Maybe it only happens (or happens more often) on Win10 or Intel drivers or something else that differs between our systems (FWIW, Revility originally told me about the problem, so at least it doesn't only happen on my machine).

I guess if this problem had always existed (like it does for me even with original Doom3 binaries) I would have been able to find more information about it and people here would have been more familiar with it, so it might indeed be due to some changed behavior in Windows or drivers

[The Happy Friar]

ok, happened to me in stock D3 binaries.  It could very well be that I normally have multiple windows open so alt-tabbing is so normal I never noticed.

[caedes]

Ok, I figured it out, see https://github.com/DanielGibson/dhewm3/commit/5c1e1d770883996eb5bb6a73118bd6d41295d282

The problem was that the widget used to render the preview of the texture in the material picker calls renderSystem->BeginFrame(w, h), with w and h being the size of that preview area.
BeginFrame() will set glConfig.vidWidth/Height, which are otherwise expected to have the window resolution (and are also used in renderSystem->GetScreenWidth()/Height()).
This is never reset (I would have expected that to happen in EndFrame()), so from then on the main (ingame) window will render at that resolution, looking like on the screenshot above.
(I now just restore glConfig.vid* after rendering that preview area.)

[motorsep]

Ok, I figured it out, see https://github.com/DanielGibson/dhewm3/commit/5c1e1d770883996eb5bb6a73118bd6d41295d282

The problem was that the widget used to render the preview of the texture in the material picker calls renderSystem->BeginFrame(w, h), with w and h being the size of that preview area.
BeginFrame() will set glConfig.vidWidth/Height, which are otherwise expected to have the window resolution (and are also used in renderSystem->GetScreenWidth()/Height()).
This is never reset (I would have expected that to happen in EndFrame()), so from then on the main (ingame) window will render at that resolution, looking like on the screenshot above.
(I now just restore glConfig.vid* after rendering that preview area.)

Cool! I feel like the tools were slapped together by ID just to get things going and never actually polished

[motorsep]

@caedes: Btw, I think it would be better to grab Storm Engine 2 tools (MFC related stuff at least) since, for example, GUI Editor was upgraded and fixed there, but not in dhewm3 fork. This way you have better base to work with that has a lot of bugs already fixed.

[The Happy Friar]

Cool! I feel like the tools were slapped together by ID just to get things going and never actually polished

Except for D3 Edit, didn't they admit that years ago by saying they didn't really use the tools, they use notepad instead?  Tools were more dev's for Raven's Q4?

[motorsep]

Except for D3 Edit, didn't they admit that years ago by saying they didn't really use the tools, they use notepad instead?  Tools were more dev's for Raven's Q4?

Oh, ouch :/

[caedes]

@motorsep: I might look at that code (and maybe merge it) later, right now I'm glad I have working tools at all
Do your tools work in 64bit builds? Do you know if any changes were necessary for that?
I declared mine 32bit-only for now because I don't really trust that code and I don't have time to extensively test them to find out if things break in 64bit..

BTW, this is now officially in dhewm3 and I just did a pre-release (incl. Win32 binaries) containing it:
https://dhewm3.org/#a-first-prerelease-of-dhewm3-1.5.1

[motorsep]

@motorsep: I might look at that code (and maybe merge it) later, right now I'm glad I have working tools at all
Do your tools work in 64bit builds? Do you know if any changes were necessary for that?
I declared mine 32bit-only for now because I don't really trust that code and I don't have time to extensively test them to find out if things break in 64bit..

Yeah, 32bit and 64bit both work (Storm Engine 2 based off BFG). Also I just remembered we did some fixes for inlined meshes in dmap (Storm Engine 2 based off BFG). What was done for 64bit to work, I have no clue, sorry

BTW, this is now officially in dhewm3 and I just did a pre-release (incl. Win32 binaries) containing it:
https://dhewm3.org/#a-first-prerelease-of-dhewm3-1.5.1

\o/ yay, congrats! Now you just need to find people to use them and test them

[caedes]

BTW, I tried building dhewm3 with tools for Win64 and it doesn't build at all, needs lots of changes in the MFC code for 64bit.
A handful of cases where functions take or return (u)INT_PTR instead of (unsigned) int, but the biggest problem is the usage of SetWindowLong()/GetWindowLong() instead of S/GetWindowLongPtr() with GWL_USERDATA and GWL_WNDPROC instead of the GWLP_* equivalents.
If you think "Haha, just use search and replace" - nope, not *that* easy, because the arguments passed to those functions (pointers to functions I think?) are explicitly casted to LONG but you need to cast to LONG_PTR instead - but just there, not in other cases where other things should still be casted to LONG..

So while making this work shouldn't be impossible, it's not that easy and I won't work on this any further for now; 32bit Windows builds are good enough anyway.

(Yes, I should eventually look at the Storm Engine 2 code which probably has all this fixed if it indeed has a working editor in 64bit builds, but OTOH there I'll have to make sure not to merge BFG-specific stuff)