Thursday, March 31, 2011

DirectQ Update - 30th March 2011 (Part 2)

I haven't mentioned it yet, but a few days ago I added in optional dynamic light coronas. you can't really see them too well in a regular screenshot, but here's one anyway.



They're cvar-controlled, of course. I haven't really decided on the best way to handle them just yet, but I'm thinking that something copied from DarkPlaces makes sense; at least it's a well-established standard for this kind of thing. Personally though I would have extended the acceptable values of gl_flashblend to take a "2".

So there are 3 modes for dynamic lights now: coronas-only (just like GLQuake!), lightmaps-only, and coronas + lightmaps. In the third mode the size of the corona is reduced somewhat so that it doesn't overpower the scene and so that you can still see the regular dynamic lightmap update. There are also the usual loads of cvars to make things more user-controllable, all beginning with r_corona.

Whenever I implement (or work on) a feature like this I normally tend to look at all the "magic numbers" that ID Quake uses for it (or that could be used for it) and decide on whether or not they should be exposed via cvars. I think it's nice to give people lots of fine-tuning ability in this way, and it means that whenever anyone asks "is there a way of changing the size/time/number/etc of X" I can normally answer "yes" right away. Most of the time they don't get exposed in menus (an exception is if there's something new that I particularly want to draw attention to) but you can consider them little Easter Eggs for you to find and use.

cmdlist and cvarlist are your friends here, by the way. Give them an optional filename to dump the lists to disk.

Back to the subject of coronas and dynamic lights. You shouldn't consider them to be a viable alternative to regular dynamic lighting that can speed things up: dynamic lighting is already so fast that on most hardware the gains will be marginal at best. So in coronas-only mode they're something you can use if you prefer the old GLQuake look, in lightmaps + coronas mode they're just gratuitous eye candy.

Wednesday, March 30, 2011

Thoughts about Sound

While my stated intention with DirectQ's sound code has been to port from Quake II, there are a few other options available, and lately - following on from a query on an earlier post - I've been thinking them through.

Some options - such as OpenAL - might look attractive on the surface, and might even be technically superior. But the big problem with these is that they would require a good chunk (I'd expect a sizable majority) of DirectQ's user base to install additional libraries on their machines. That defeats one of the objectives of DirectQ which is that you don't actually need to do this. Just install Quake, download DirectQ, make sure that you're on a reasonably up to date version of DirectX and you're off.

You'll note the precedent in my web download code, my PK3 support, my external texture loaders, and other examples all over the place. None of them need additional libraries on your machine. All the code is either contained in the executable or uses native OS components. So it's not a case of "you need libweenie 1.3.12 and it's your tough shit amigo if that conflicts with anything else you might have that needs a different version".

A reasonable argument could be made that such libraries do support portability, but since DirectQ is a Direct3D program anyway, portability is not even on the radar. I have the luxury of being able to say "I'm using the Windows API for this, GDI+ for that, etc" without having to worry about consequences, so the argument becomes irrelevant.

So these options are Just Not Going To Happen.

Porting Quake II's code was the leading option for a number of reasons, with the strongest being that it has the most similarity to Quake's (but with some essential extra features added), but there was a third option lurking in the back of my head which was to rewrite the entire thing from scratch.

Am I mad? Well I've already done it for the renderer, so why not the sound code too?

One school of thought says "it's working code, it's fine, don't mess with it". That's a very valid argument from one perspective, but it doesn't take in the whole story.

Quake's sound code is The Way It Is because it had to run on machines of the time. 1996, MS-DOS, Pentium 60, 8 MB of RAM, etc. DirectSound was grafted on for WinQuake, but the core sound code remains an abstraction layer that just calls down to whatever lower level API it happens to be using, but is otherwise much the same.

Besides, it's not really the way you write DirectSound code. Real DirectSound code looks something like the structure described in the First Steps in DirectSound Programming article in the DirectX SDK. Lots and lots of Secondary Buffer objects all of which are mixed together automatically by the runtime. No locking overhead, no mixing in software, no requirement to do extra updates if running slowly, everything nice and clean.

So I'm coming around to thinking that this is a viable approach. Aside from anything else, it will also allow for more advanced sound FX than Quake originally supported should I ever decide to go down that route in the future. I'm thinking stuff like room ambience, echo, underwater gargle, and so on.

DirectQ Update - 30th March 2011

Been working some on lightmaps and timers. I've fairly well gutted the old lightmap code (which had that horrible class in it), cleaned out a lot of mess along the way, fixed up some stupid things, and have gotten a nice turn of improved speed as a result.

Timer-wise I'm getting damn close to being at a stage where I'd feel reasonably confident about releasing with decoupled timers enabled as default. I've played through a lot of maps, fine-tuned a lot of behaviours, and have most things working nicely the way they should. Some interesting consequences of having a server and client not running at the same speed have come up, with changes being needed to enable parts of the code that were previously Multiplayer/non-local server only (CL_LerpPoint is a good example).

One thing I remain slightly paranoid about is some input code I ported from ProQuake; the whole synthetic lag and CL_SendLagMove thing. I can see what the code does, for sure, but what I'm not seeing is the intention behind it, nor what problem (if any) it was intended to solve, nor what advancement (if any) it was intended to offer. It might be necessary to disable some of this code if running a local server, so more experimentation will be required.

Right now the big question is: do I release a "work done so far" version (with decoupled timers disabled by default, and an advisory that you enable them at your own risk) or do I push ahead and complete the job? Completing the job by this stage involves going through every function that uses a time or frametime variable, deciding what the most appropriate way of handling it is (which could be "do nothing"), making the changes required, and a whole bunch of testing.

In terms of releasing I'm thinking that it makes sense to change the versioning. It's getting a bit silly with the whole RC1/RC2/RC3 and now potentially RC4 stuff, so instead what I'm proposing to do is a sub-version based on the build date. The main version is still at 1.8.7, so that's going to remain the same.

Monday, March 28, 2011

Your mother was a hamster and your father smelled of elderberries!

Yes, it's them Kniggits again, and this time I've got them to almost 420 FPS.



I'm suspecting that I'm right at the limit of what I can pull out of the hat with MDL rendering now; about the only thing left of major interest is exploring options for muzzleflashes.

Sunday, March 27, 2011

Menu Work and More

The next one is going to add two new submenus off the "Options" menu: one for "ProQuake Options" and the other for "User Interface Options". At the risk of going back to the old overloaded menus, let me explain what's happening here.

"ProQuake Options" is a gathering place for all options that I've ported from the ProQuake code. It's convenient to have them all together, and it makes it clearer what the purpose of these options is. Right now it's probably looking a little bare, but hopefully it will grow over time. Thinks: this one could probably alternatively move to Player Setup as it's mostly Multiplayer-specific.

"User Interface Options" is one that is not strictly necessary, but it does give better organization and grouping. I'd always found it slightly weird that these options were split between the Video menu and the Effects menu, so now they've been pulled together. Doing this also gives a little more onscreen real estate for anything I might want to add to either menu in the future too.

The old "Sound Options" menu is still Missing in Action. I've a stated intent to port Quake II's sound code, but at present it's low on the priority list - mostly because what I've currently got works just fine within the bounds of what ID Quake did. It will happen though as soon as I get the necessary impetus together, and at that time Sound Options will reappear.

This gives a much more sensible grouping and if you want to configure something via the menus it should be immediately obvious where to go: Video, UI, Sound (in the future), Effects or PQ (which may yet move to a more logical place).

If you've already set up your config the way you like it none of this is relevant to you, of course.


I've more or less finished the MDL work, and it's working nicely. The code did turn into a slight mess as it grew, so I'm thinking that it probably needs another pass over it to complete a final clean-up. In the end I went with the Fitz method of handling viewmodel interpolation; it's not perfect but it is the one that relies on the least assumptions and as such is least likely to cause trouble in the future. I do have another plan cooking for this, but it's going to require a little bit more work, so I'll hold it off for a few releases.

There are some other things I want to work on too, but I'll leave talking about those for later.

Quake Gameplay Notes

I'm coming close to the end of what must be my first full Quake playthrough in almost 10 years. This was initially done to test out my new code in a real life situation, but after a while turned into something I did just for fun.

I'm taking the episodes in order, and trying to spend at least 20 minutes on each map, just savouring the experience. Taking one or two maps per day, with the occasional day when there are none, adds to it all.

Right now I'm on the Pain Maze. I was initially sceptical about whether or not I'd even do e4, but as I moved from testing to fun I really started getting into it, and - to my surprise - it's turning into what's probably been the most enjoyable episode.

It's true that Sandy Petersen didn't have the same aesthetic sensibilities as a Romero or a Willits, but by damn his maps are fun. You just cannot fault the gameplay here - absolutely top notch stuff. Some interesting observations about the style.

Each weapon only appears once in the episode, with - I think - the exception of the Lightning Gun, which makes a second showing in the Nameless City. Miss it and it's gone, but Sandy did think enough of this to pop up a message saying "are you sure you want to exit now? you left something important behind". Sheer class.

Jump into a map and almost immediately you get access to a Megahealth. OK, either you've just taken a pounding and you need it right now, or you're about to take a pounding. This happens in every single one of Sandy's maps.

You commonly see a monster - with it's back to you - walking off. An easy first kill, and it helps to create the sense that this is an inhabited world in which the monsters do stuff other than just wait around for you to kill them.

Architecturally it's all weird angular stuff with a crazy hodge-podge of texturing styles and lots of huge gloomy shadows. e4 must be the most minimally lit episode in all of Quake. It just feels truly alien, like something for which there is just no real-world counterpart. Weird. Eldritch. Elsewhere.

Structurally there are lots of weird slightly out of place elevators, and you commonly have to make two or three passes over most of the map in order to complete it. It's packed full of weird little pits full of monsters, crazy walls that open out in odd ways, interesting battle arenas with plenty of scope for using the environment, and curious tucked-away nooks and crannies which are always packed full of goodies but also often full of baddies to make you earn your loot.

So have I just found a new favourite episode? Quite probably; for all it's flaws e4 must be the most quintessentially Quake-like of all the episodes. Who woulda ever thought?

Saturday, March 26, 2011

Thoughts on Muzzleflashes

Been thinking some about muzzle flashes on view models. If you know your history, you'll be aware that the combination of frame interpolation and muzzle flashes is evil, and can give visual glitches such as the flash moving back and forth between the two barrels on the Nailgun.

Up to now DirectQ has had a solution (of sorts) where it could selectively disable interpolation for parts of a model on a per-vertex level. However I've always been wary of this as it's based on a reasonably fuzzy heuristic which may not always be valid. The fact that it has been valid so far is neither here nor there: someday a model will appear that invalidates it.

I'm also doing some more optimization work on MDLs which breaks this solution (of sorts) so matters are coming to a head with regard to how to deal with it. I could of course undo that work, say "it's fast enough already", and put things back the way they were, but that wouldn't do anything about the risk of the solution (of sorts) becoming invalid in the future.

So I'm thinking that it's better to do something now rather than wait until it does break and then have to rush a fix. The options available seem to include these.

Just don't interpolate view models

This may yet be a viable approach, but in reality it's just exchanging one visual degradation in quality for another. If no other solution proves workable it is however what ID Quake did, so it's got a lot going for it.

Interpolate them anyway - it's a content bug!

DarkPlaces does this, and it's also viable from one perspective. However I don't see it as being an absolute content bug, as the setup of muzzle flashes was valid within the original parameters of ID Quake.

Copy what FitzQuake does and disable interpolation for two frames when the poses change

This fully resolves the muzzleflash problem in exchange for a small loss in smoothness. It seems a fair tradeoff, and has the advantage that it's an already accepted solution.

Do something sexy with the models that works with the new optimizations

I seriously thought about things like removing the muzzleflashes from the models and replacing them with a sprite, or drawing viewmodels twice - once without the flash vertexes and interpolated, once with only the flash vertexes and not interpolated. Both of these (especially the latter) seem attractive and interesting, but both of them also depend on that fuzzy heuristic to identify the flash vertexes which might break in the future.

So right now I'm leaning towards copying from Fitz as being the lesser of all evils, and it's probably sufficient as a shorter term solution. Longer term I am genuinely interested by the idea of drawing the models twice, and perhaps adding some user-configurability for the heuristic and for viewmodel interpolation in general.

Friday, March 25, 2011

DirectQ Update - 24th March 2011 - More Fun with Hardware Instancing

Hardware instancing rocks. Check out the polycounts.



And all that in a mere 17 draw calls too. Neato.

Thursday, March 24, 2011

DirectQ Update - 24th March 2011 - Fun with Hardware Instancing

Taking a short break from timer and input work to explore the potential of hardware instancing a little more. I've seen potential in using this to accelerate MDLs a little, so today I wrote up some test code to just count how much of a saving I can get from using instancing (particularly in combination with my current MDL code which can do massive VBO reuse).

Results were quite surprising to say the least. Even in the most common cases - like the Necropolis demo - maybe 90% of the time there are significant enough numbers of the exact same MDL using the exact same frames, vertexes and textures being drawn. I expected some for sure, but I didn't expect that much. In extreme cases - like my 400 knights map - batches of up to 150 or so of the MDLs being drawn can be instanced. I expect that something like the Quoth longtrch model in ne_tower (where there could be up to 80 or so of them in the PVS and view frustum at any one time) would also benefit hugely from this.

So there is potential benefit to be had here and it's an avenue worth exploring.

Of course this will only work if your hardware actually supports instancing; hardware that doesn't needs to go through the slower path, but at least that can benefit some from the VBO reuse I have going so it's not all bad news.

DirectQ Update - 23rd March 2011 (Part 2)

I've added a contrast cvar (called "contrast", oddly enough) and menu option. RMQ has had this for a long time, and there was really no reason for DirectQ not to have it aside from laziness and oversight on my part.

One thing you might not know about gamma in DirectQ is that it's possible to adjust red, green and blue independently of the main gamma. The cvars are called "r_gamma", "g_gamma" and "b_gamma", have been there for 2 years or so, and the same now applies to contrast ("r_contrast", etc). This should be helpful for those who like to tweak this kind of setting to the ultimate level.

DirectQ is actually full of little hidden settings like this, so I'd encourage you to have a look at the new cvar and command lists I've posted to your right. I haven't added descriptions yet, but you may be able to pick up some hints from some of the names.

Multiplayer-side I've fixed one serious bug with ProQuake message parsing whereby the notification area was spammed with ping and status results when you bring up the scoreboard. This was annoying, and I've written before about the quality of this code, but today I'm going to mention comments.

In the original source the comment on the line containing the bugfix looked like this:

// added this
My comment describing the fix looks like this:
// set cl.console_ping = true before sending a ping to the server
// if you don't want this to swallow the result
Now, I don't claim to be the best commenter in the world but which do you think is most useful to someone porting or maintaining the code? I don't care how technically excellent the code is, this kind of thing is utterly disgraceful.

Wednesday, March 23, 2011

DirectQ Update - 23rd March 2011

Been fine-tuning the timers and framerate decoupling quite a bit, which is really a matter of ensuring that everything is working off the correct timer, which circumstances to use which timers in, that all subsytems have their time and frametime correctly isolated, and that any assumptions I may have made are valid.

Interesting things are coming out of this exercise.

Some of the client-side subsystems need to run off the server-side timer. In particular I'm talking about stuff like particle updates and the old view.c stuff. These are heavily framerate dependent, and running off the (potentially) faster client-side timer causes problems for them.

In timedemo mode the client time is advancing faster than the client timer, so it needs lots of special handling.

For stuff like sound updates it makes sense to run them off the slower timer. You don't need to update your sound buffers 250 or more times per second; 72 is plenty, thanks. There's a good performance improvement in this too.

If a game isn't currently active you can just drop back to the slower timer, irrespective of the value of host_maxfps.

For multiplayer games reverting everything back to the original way seems to make sense.

I'll probably ship a release sometime later on this week or early next week with an experimental first cut of this in, but it will be disabled by default and you'll need to opt-in. host_decoupletimers 1 will be what you need, and as it's an experimental first cut you shouldn't be too upset if things don't work 100% right.

On the input front I've been pondering some on pitch drifting. This is an old relic of the days of keyboard looking, and of having to hold down a "look" key, and is the means why which your view automatically re-centers over time. I'd really love to take a knife to this code, but I suspect that there may still be valid cases where it's useful to have (I'm thinking joysticks and controllers) so for now I'll just disable it completely if we're mouselooking. Of course this should happen automatically anyway, but with everything potentially running at different framerates it just seems to make more sense to handle it more explicitly.

More later.

Firefox 4 Impressions

Last week I installed IE9, so this week it was the turn of Firefox 4. I must say I was deeply underwhelmed by FF3 but was still cuirious enough to bite at 4.

OK, so it's not an "oh wowsie, this is the best browsah evah" moment, but it's definitely a vast improvement in many areas on 3.x, which was starting to look, feel and behave a little clunky for me (especially in comparison to the ultra-slick IE9). It's quite interesting to compare both browsers side-by-side, and note that in terms of the user experience at least they're now far more similar than they are different. Each has small and subtle points that make it better than the other, but on balance you'ld really have to look twice to notice.

Specifically talking about FF4, one genuinely crushing disappointment is the treatment of the bookmarks toolbar. I've always loathed this monstrosity, and unfortunately in FF4 it seems to have orchestrated an even more obnoxious hostile takeover than before. A quick search on Google reveals that I'm far from the only one who hates it, but somebody on the FF team obviously is unwilling to take this kind of feedback onboard. A "do not use the bookmarks toolbar" option would be nice to have, as would "do not show the bookmarks toolbar on the bookmarks menu".

Speaking of which, it's still not possible to just right-click/sort by name in the bookmarks menu. Sigh.

Overall though, and despite these gripes, it is definitely a significant and valuable jump ahead of 3.x, and I'm looking forward to making more use of it as time goes by.

Tuesday, March 22, 2011

A Brief History of Time

Instead of fixing up some multiplayer issues (which I should be doing) I've been experimenting with timers again. I believe that I've now taken the final steps required to properly decouple all of the subsystems in the engine, and now it's just a matter of properly determining which timer they should run on each frame (and under which circumstances: local server, active timedemo, etc).

Right now I have a build with 3 such timers running each frame, with one (which I'll call the "Client Timer") ticking according to the value of host_maxfps, another (which I'll call the "Server Timer") ticking at up to 72 FPS and the third (which I've called the "Steady Timer") ticking at up to 500 FPS. Depending on the mode that the engine is running in, various events during a frame (reading client input, rendering a screen, running the server, updating sound, etc) fire off based on one of these timers.

This has all got me thinking more about exactly how often certain events should fire off. All of this is purely "thinking aloud", and there are no final decisions made, but it is interesting to think about them.

First up is the renderer. I'm wondering if the renderer should ever actually run at more than - say - twice your refresh rate. What exactly is the value of running the renderer (and remember that I'm only talking about the renderer here, not client input, for example) at 97 bajillion frames per second versus running it at 120? On the other hand, framerate locking the renderer would result in complaints that host_maxfps isn't working and that timedemos are unusually slow, so on balance leaving it unlocked seems better. Sometimes you have to do the Wrong Thing in order to do the Right Thing.

Second up is the server. Traditionally this has been subject to a cap of 72 FPS, but I've come to believe that the Quake server (particularly in single player games) should actually run a good bit slower than that. Look at it this way: back in 1996 most people were getting 20-30 FPS from Quake, and that was considered good performance. Quake development would have involved tuning the behaviours of physics/etc to give the intended results at this kind of framerate. As Quake has framerate-dependent physics, based on this logic even 72 FPS would be too fast and would be incorrect.

So why was 72 selected? A comment in the original source code ("don't run too fast, or packets will flood out") indicates that it's intended to be an absolute upper limit, rather than "the framerate that Quake is supposed to run at" (which I believe to be actually nearer 25). And why 72, rather than something like 70 or 75? The most reasonable explanation seems to me to be that 72 Hz was a common (if not the most common) refresh rate for monitors at the time, and it was just arbitrarily chosen on that basis.

This is all conjectural, of course, and I must dig through the old .plan archives to see if I can find anything that has a bearing on this.

So like I said, no final decisions made yet, and more testing of everything is required before I'd even consider a release with decoupled timings again, but it is a very interesting topic.

Monday, March 21, 2011

DirectQ Update - 21st March 2011

OK, so the Spring Equinox was officially yesterday according to here, but anyway, whether official or traditional it means that summer is now on the way, so with lots of sunshine, ice cream, warm days and pleasant feelings coming up, it's time for a DirectQ update.

(Of course this sucks if you live in the Southern Hemisphere, but you can't have everything.)

The next release should see a significant improvement in performance with scenes containing lots of liquid textures. I already had a sine wave lookup for the underwater warp, and I was able to reuse it for the standard water warp too (I've you've looked at recent source code you'll see that I had a note to do this). Because this is a 2D texture with a sine wave in both the red and green channels (blue and alpha contain something completely different) I'm able to get two sin calculations at the cost of one lookup, which is just perfect for water warps.

I've also moved a lot more calculations which were previously per-pixel to per-vertex, and calculations which were per-vertex to one-time-only. This involves no loss of quality; anything which can be linearly interpolated correctly is a candidate for such a move. A sine wave can't be linearly interpolated - at least at such a coarse level as per-vertex - which is one of the primary causes of GLQuake's crap water.

I've removed the clamping of the cvars associated with this, so you can create some really weird effects by playing with their values. They all begin with r_warp, by the way.

A few bugs came out in the last one which I'm also in the process of fixing.

I wonder what people think of the current rapid-fire release schedule? I'm quite happy to keep it up for the time being, but is it too often, not enough, or what? I wouldn't be comfortable with something as regular as nightly builds, one of the main reasons being that the engine is often completely broken at the end of a day, but once every week or two feels OK to me. It also means that I can respond to bugs a little more quickly.

Friday, March 18, 2011

1.8.7 RC3 is out

Download it from here.

DirectQ Update - 17th March 2011

It's just as well that I didn't release as planned yesterday as there were a few late-breaking bugs and areas in need of enhancement that came to light while I was doing a run through e1 to e3 (I'm not sure if I'll do e4 - all those Tarbabies - eeewww).

So I've got the memory saving and map loading speedups done. The items in question were edges, surfedges and vertexes in BSP models, which - if you've got your polygon building structured appropriately - can be taken directly from the on-disk versions instead of having to allocate memory, read them in, etc. They're only needed during polygon building, and once that's done they're never used again, so they went.

I also took the opportunity to sanity-check one element of polygon building, which was triangle strip conversion. DirectQ basically converts all world surfaces to a triangle strip layout at load time, which may be faster on some hardware owing to vertex reuse and less hopping around in the buffer. Popping the conversion into GLQuake (and using GL_TRIANGLE_STRIP instead of GL_POLYGON) confirmed that it's cool.

This is something that may be of benefit to GLQuake based engines in general; in theory it should be possible to add degenerate triangles and build a big chain of strips for drawing, which would enable them to get some of the benefits of batching without so much of the code restructuring overhead. MDLs in GLQuake could also benefit from it as the conversion code I have can convert any GL_POLYGON or GL_TRIANGLE_FAN layout to a strip.

This is all 1998-level optimization - indexed triangle lists rule on modern hardware (and are what DirectQ uses) - but I could see an engine that aims to still support hardware from that era reaping benefits for sure. I'm thinking that it's an area I'll most likely explore further in the near future, as I'm finding it tickling my interest a little.

What else is done is that I've fixed a nasty little bug that occasionally broke TAB autocompletion (this took far too much time to identify the cause of) and added a session-to-session command history. I've also tidied up the Finale (end of episode) screens so that the banner is aligned better with respect to the text.

Release on track for tomorrow. Look out!

Wednesday, March 16, 2011

DirectQ Update - 16th March 2011

Today's planned release isn't going to happen, and it will now most likely be Friday at the earliest.

No problems here; just that I'm after discovering a whole class of data that doesn't actually need to be loaded at all. This is inherited from GLQuake, and getting rid of it will result in improved memory usage (not that big a deal) and faster map loading times (very much a big deal).

This will be another "Release Candidate", meaning that if no problems arise with it I consider it suitable for release and general usage, but that there will be a lot of new code in it that will need to go out into the wild before I feel comfortable calling it an actual full Release.

Tuesday, March 15, 2011

DirectQ Update - 15th March 2011

Got some speed boosts from recalculating the node and leaf bounding boxes for brush models. I was actually totally unaware that this was a problem, but reading through hmap2's source code alerted me to it, so now there's some much more efficient frustum culling going on.

Been having more fun with MDLs. Before I say anything else, I need to stress that all of the following is optional, OK. So let's not be getting paranoid about any of it.

First of all I put mesh optimization back on MDL files, but this time did it right. What this does is rearrange vertexes so that they're in a more optimal layout for faster rendering. To be honest it's really only of major benefit for huge datasets, but it feels like the right thing to do, so it's done.

This does cause a small (i.e. barely even noticeable) delay in map loading, so if it's objectionable to you, you can always disable it by using r_optimizealiasmodels 0. This one is enabled by default, so it's opt-out. I would encourage you to leave it at 1 and only disable it if it causes you any specific problems; the extra millisecond or so to load each MDL should otherwise not be a problem, and you may get a performance boost from it.

The second one is the return of the dreaded "mesh files", also known as "the glquake directory" (although I've called it "mesh", and the files are a new .ms3 format). DON'T PANIC - this one is optional, remember, and in this case it's strictly opt-in (i.e. disabled by default). Set r_cachealiasmodels to 1 to enable it (the default is 0, which is disabled, remember) and you'll offset some of the speed loss from mesh optimization by loading an already optimized mesh from disk.

What's different here is that the format has been changed to conform to my new in-memory layout for MDL files (introduced 2 years ago), and also to resolve problems with mismatched meshes (try saying that fast after a few beers). The important bit is that the new .ms3 file contains an MD5 checksum which is taken from the original MDL and used to verify that the .ms3 file is valid. The other important bit is that this is disabled by default, or did I already mention that?

So if you don't mind having extra files clutter up your Quake directory, and if you want slightly faster loading times (although on todays hardware the gain will be marginal at best, and may even be slower if disk IO is a bottleneck for you) you can opt-in to this, otherwise it won't even happen at all and you can cheerfully carry on as if it never even existed.

In other news, I tested the new fullbrights code on Intel today and it worked Just Fine, thangyewvuhmush, so I'm thinking of releasing this puppy tomorrow.

Monday, March 14, 2011

DirectQ Update - 14th March 2011 (Part 2)

The new fullbright handling code is in, so to celebrate here's some crunchy pixels.



I haven't tested this on Intel yet so I won't know the full story on performance for a while.

Tip of the day: if you're going to release a Windows binary, and if you're including a readme file, and if that readme file is a .txt file, then the least you can do is use Windows line-endings. I guess I've just found one reason why people might not read readme files.

DirectQ Update - 14th March 2011

The "cd" command now works with replacement audio; I've also added TAB autocompletion for all of the options.

Loaded pointfiles should be a lot easier to see.

I've removed the multithreaded surface sort code and reworked some things deep in the renderer backend; this should give a 5% to 10% speedup (depending on hardware and other system config, of course).

I'm going to add TAB autocompletion to the "bind" command sometime, but it requires a slight reworking of how some of it ("it" being autocompletion, not the "bind" command) works so it may not happen soon. But it'll be another strike in favour of player-friendliness.

I had a version of DirectQ running that did AVI capture some time back, but I junked the code owing to weird bugs in places. I may restore some of it but don't hold your breaths for this one.

That's about it for now. Release in the second half of this week, depending on how testing goes.

Update

You can now turn fullbright colours on and off using the gl_fullbrights cvar. The default is on, of course, because this was how software Quake did it. However, being able to turn them off is desirable in cases when the texture artist targetted GLQuake and either didn't bother/didn't care to test in software Quake or was unaware that the original intended look of Quake was slightly different.

An interesting glitch with fullbrights is that they can actually appear darker than the non-fullbright version under certain circumstances. You want an example? Here we go.



The obvious solution is to sample the textures with fullbrights both on and off and take the maximum, but it will come with a performance penalty. How much? Don't know yet, but I suspect that it will be acceptable enough.

Now, this is definitely not how software Quake did it, and it will lead to some slight over-saturation in places, I know, but it comes back to what I mentioned before about doing something a little different "when the original is crap".

Sunday, March 13, 2011

DirectQ Update - 13th March 2011

Some slight jerkiness in the timers has been fixed so things should now be running even smoother.

One issue with vsync has been fixed. This related to detecting the vsync setting from a video card's control panel, which is not possible under D3D. Ironically, I was able to do it by adding some OpenGL code during startup. This doesn't use anything beyond basic OpenGL 1.0 functionality, so it should be compatible with everything, but it is still quite experimental. I'm interested in seeing how it works out for people.

With r_waterwarp 1 enabled you might have noticed that as you turn around, the warp changes while turning. I had a fix for this, but it wasn't compatible with drawing the view model (which doesn't rotate) so it had to go. The lesser of two evils prevailed I guess.

I've got a slight speedup with fullbright rendering. This goes back to the way 1.8.666 and earlier did it, but it doesn't affect anything else to any major degree.

Some more fixes and updates but you'll need to follow the comments in other posts to pick them up.

I think the next release will happen sometime over the next few days. Stay tuned.

Wednesday, March 9, 2011

DirectQ Update - 9th March 2011

This one didn't make it into the RC2 release, but I now have FOV correction for modes that are taller than they are wide.



I'm not sure how this is going to react with values of the "fov" cvar other than 90, so remember that crazy combinations of settings are entirely at your own risk.

There's a bug in RC2 that only affects multiplayer games which happens if DirectQ needs to download a map and you have r_waterwarp set to 1. The new code for this attempts to get the player's position in the map before the map has fully downloaded, and so will cause an error, which in turn causes the connection to fail.

The temporary fix for now is to set r_waterwarp to 0 or 2 before connecting, then set it back to 1 after you connect. I've updated the CodePlex download page with a note to this effect. It will be fixed for real in the next one, of course.

DirectQ 1.8.7 (RC2) is Out

Download me baby, oh yes, do it now, oh yes yes yes.

Tuesday, March 8, 2011

DirectQ Update - 8th March 2011

I haven't updated on DirectQ for a while as I've been stuck in the depths of some messy coding work for RMQ, but today I did a little more.

We now have arbitrary window resizing, so you can grab borders, move them around, hit the maximize button and so on. This obviously only applies to windowed modes, but it's quite cool nonetheless.

A window resize event needs to also resize the backbuffer, so it's effectively a video mode change, but this is a fast enough operation (especially for windowed modes where a resolution change isn't involved) so it doesn't really matter so much.

It's not 100% feature-complete now, in that it doesn't co-operate with your Video Options menu, and the changed size isn't saved to your config, but I'm in two minds about that. Do I want to make it complete, or do I say that it's not meant to be? Answers on a postcard, please.

Friday, March 4, 2011

DirectQ Update - 4th March 2011

Been reworking my MDL lighting again. I was starting to get quite annoyed by it earlier on, so I decided to refocus my objective a little, and am now well pleased with the result.

Way I see it there are a number of possible goals with MDL lighting. One obvious one is to be faithful to GLQuake, and another is to be faithful to software Quake. Up to now I had been aiming for the latter, but ran into trouble with various scaling and clamping factors, in particular with attempting to work overbrights vs no overbrights into it.

Thinking back, the solution should have been quite obvious. Instead of aiming for either of those two goals, let's define a third one instead: be consistent with the world model lighting. This automatically solves a lot of problems: MDLs now slot perfectly into place with no weird brightnesses (or darknesses) jumping out at you, dynamic lighting matches, and the whole scene just looks a lot better now.

Being faithful to the original is a worthwhile goal, but sometimes when the original is crap you've got to rethink things a little.

Update

A screenshot of the revamped MDL lighting was required so here we go. There's not really much to see here, it's when models and lights start moving that things get interesting.



And I would have gotten away with it too if it wasn't for you pesky kids!

Thursday, March 3, 2011

DirectQ Update - 3rd March 2011 - Part 2

I've switched waterwarp back to my old software-Quake-alike render to texture warp. The reasoning for this is that I frankly missed it; it was nice to be able to say that I had an underwater warp that looked like software Quake's.

I have however retained the more recent projection matrix mangling warp as an option for slower machines; this is selectable by choosing r_waterwarp 2.

The render to texture warp has been made highly configurable; look for cvars beginning with the text "r_waterwarp" and play with the values. None of these are exposed via menus and I'm not going to tell you what they do; they're toys for you to discover and play around with. :)

DirectQ Update - 3rd March 2011 - Protocols and Other Strangeness

A short note about protocols as I may not have mentioned this before. In the past DirectQ has supported the original Quake protocol, the FitzQuake protocol, the work-in-progress RMQ protocol, 3 BJP protocols and my own hacked-up modification of one of the BJP protocols. Maintaining this mess became a little silly, so for 1.8.7 I ripped them all out aside from original, Fitz and RMQ (and also updated RMQ to reflect what's actually used by the RMQ engine).

I've now gone back and added back support for the 3 BJP protocols, but these are now client-side read-only protocols and are supported solely for the purpose of playing demos. It's not possible to set them on the server, and trying to do so will have no effect.

The default protocol for DirectQ has gone back to being the FitzQuake protocol as it's the most sensibly designed of those that are actually out there "in the wild". It will of course automatically set itself to whatever protocol the server is using, so none of this needs to bother you if you have concerns about how it's going to interact with MP games. The server will always control the protocol, and you don't need to do anything special yourself - DirectQ will Just Work.

Speaking of "Just Working", I should probably mention that setting -heapsize on the command-line has no effect in DirectQ. It automatically configures it's memory usage based on the requirements of the map you're playing, and will automatically use as much as you need (and no more).

This goes back maybe about two years, but old habits might die hard.

Also worth mentioning that the "game" command ("gamedir" also works and has the same effect) can take multiple parameters. "game hipnotic quoth warpspasm" is a valid command, for example, and will load the specified games in the order you give them. Up to now only the first game has tab autocompletion on it, but from the next release onwards they all will.

"game hipnotic", "game rogue", etc have the same effect as using -hipnotic, -rogue, etc too.

1.8.7 Release Candidate 1 is out

Here we go.

This is now the recommended DirectQ release.

A rant about r_mirroralpha

When GLQuake was released it came with a few additional features over software Quake - shadows, water alpha and mirrors. If you're one of those strange people that actually reads readme files, you'll know that these are intended as novelty features. If not, let me quote the relevant part of GLQuake's readme at you:

novelty features
----------------
These are some rendering tricks that were easy to do in glquake. They aren't very robust, but they are pretty cool to look at.
Now, it should be quite clear what the intention is here, and there's lots of things wrong with these features. Water doesn't depth-sort properly, shadows only work on flat planes, and mirrors have all manner of restrictions. However, and for some bizarre reason, over time these have come to be seen as somehow "essential" features of GLQuake, despite their original stated intention. People want to see their crap shadows and they want to see their crap mirrors. And will sometimes get very upset if they can't.

As a generalisation, with a novelty feature you have maybe 3 options. You can work on it and build a more robust implementation. You can accept it as a novelty and leave it as it is. Or you can drop it. With DirectQ I've taken all 3 approaches.

For water alpha I did the work and added proper depth sorting against both other water surfaces and other alpha objects. For shadows I just said "whatever" and left them largely as they were; I did remove the angle to resolve one rendering glitch when a MDL goes offscreen but it's shadow remains onscreen, slightly de-crappified them by using the stencil buffer and allowing intermediate values of r_shadows, but otherwise I did the bare minimum. And I killed mirrors until they were dead.

I don't know about the rest of you, but r_mirroralpha was one of those things that I turned on once, looked at the crap mirror, then turned off again and carried on getting on with the rest of my life.

The problem with Quake mirrors is that they need a more sensible standard built up around them. Otherwise it's just not possible to build a proper implementation, and they will always be crap. Let's quote from that readme again:
This changes one particular texture (the stained glass texture in the EASY start hall) into a mirror. The value is the opacity of the mirror surface.
Well whoop-de-fucking-doo indeed. This just isn't good enough; they need a proper texture naming convention, proper usage guidelines for mappers, and this just isn't it.

So, as a postscript, I'll finish by saying that I occasionally get requests from people for stuff. I haven't had a request for restoring the crap mirrors yet (which kinda confirms that most others don't see them as a big deal either), but just in case I ever do, this is to save you the time and effort required to ask, and to save me the time and effort required to answer.
No.
I won't.
And that's final.

Wednesday, March 2, 2011

DirectQ Update - 2nd March 2011

gl_texturemode is now saved to your config file. I've also fixed a bug in it where GL_NEAREST was always selected irrespective of what you asked for. I guess that most people who used it were using it to select GL_NEAREST anyway, so it may have slipped through the cracks. This only affected gl_texturemode when used from the console; the corresponding video menu options always worked right.

This command has tab autocompletion enabled on it, by the way, just in case anyone hasn't noticed. I've always found GL_NEAREST_MIPMAP_NEAREST (my preferred mode for when I get a hankering for crunchy pixels) a little too long-winded to type, so I did it a few versions back.

The D3D effects framework is proving to be a minor pain, and something of a performance bottleneck. Warning - technobabble follows. The primary problem is with the ID3DXBaseEffect::SetTexture method, which does an AddRef and Release on your IDirect3DTexture9. Normally that's OK by the rules of COM (highly recommended if you're copying a pointer, because otherwise there is a risk that something else may Release the object before you use the copy), but in this case the AddRef and Release don't actually accomplish anything as there are no D3D calls between them. To compound the misery, the AddRef call on IDirect3DTexture9 is fucking expensive. Between these AddRef and Release calls, D3D is spending almost 3 times the amount of CPU time that it's spending on everything else combined.

Obviously something is happening here that goes beyond simply incrementing or decrementing a reference counter, and I suspect based on this evidence that MS have coded some additional logic into the AddRef method for IDirect3DTexture9. Needless to say I'm exploring options for working around this shoddiness.

______________________________


Update: I've resolved the texturing problem so that CPU overhead is now gone. Digging into PIX it seems as though there's lots more evil stuff going on behind the scenes with the effects framework, so over time I'll be cleaning that out too. It's a pity because the framework is really handy for compiling and managing shaders, but in use it's a strange black box where you never really know what's happening. This confirms my initial observations on it from about 2 years ago, when I noted that framerates had dropped a few percent when compared to raw shaders.

I'm playing with the idea of putting mouse updates into a separate thread. the real problem here is that Quake's default 72 FPS framerate just doesn't match to typical mouse polling rates, which does impact on overall smoothness.

I did have some problems with single CPU/single core machines when I moved a portion of the renderer to another thread, but I think I can pull this one off without so much bother.

Tuesday, March 1, 2011

DirectQ Update - 1st March 2011 (Part 2)

Keyboard autorepeat broke as a result of some new input code (and me trying to be too clever for my own good); this is now fixed.

I've removed the renderer decoupling as there were some glitches and edge cases I'd failed to catch. No big deal; I've lost nothing by trying it and it was definitely worth trying to see how it would behave. Guess that I'm almost there with it, but not close enough that I can see the whites of it's eyes yet. It's worth another round later on, but right now I'm not going to let it hold things up.

Some shader optimizations should see some more performance improvements coming through. I forgot to mention that Beta 3 got about 25% extra (depending on hardware/etc) in heavy scenes, which was nice. I might be able to pull a few more percentage points from it.

I think the next one out is going to be a Release Candidate, and will happen towards the end of the week.

Beta 3 is now available

Here be the link.

Things have really started coming good now, so I'm thinking of promoting it to a full release for the next one. This is of course assuming that no further unforseen bugs appear.

Thanks to everyone who downloaded, tested and gave feedback on the previous beta releases. I haven't gotten round to implementing all of your requests yet, but I hope to tackle at least some of them shortly.

DirectQ Update - 1st March 2011

Beta 3 will be out later on today; just a few things to finish up.

Been playing around with the new input code a little. Just running around in maps, spinning around, pressing keys, etc. It's quite beautiful and even eerie in-use; it's extremely responsive and smooth. Yeah, I'm happy with it.

I've also finally completed the full decoupling of the renderer, sound and client input code from the rest of the subsystems so these are now running at full speed while everything else runs at the standard 72 FPS. Because of this I've restored the 72 FPS lock for single player games. Note that this only applies to the server portion of the host; the client runs at full speed now.

There are probably still valid reasons for running a MP client at higher than 72 FPS so I've left things as they are there.