Friday, October 30, 2009

The FIRST thing you should do if DirectQ crashes

Despite my best efforts DirectQ may sometimes crash.

This sucks, but there is something easy, quick and painless that you can do before reporting it to me.

DirectQ is a Direct3D9 application, but you should never assume that if you have Direct3D10 or 11 that you also have the most up to date version of Direct3D9.

Direct3D10 was first released in late 2006, Direct3D11 was first released in late 2009, but Microsoft kept releasing updates to Direct3D9 after both of those dates. These updates are not included on either Windows Vista or Windows 7, and just having Direct3D10 or 11 may not be enough.

The most recent update to Direct3D9 was released in February 2010 for example. There is probably - if things go on their usual schedule - another one due sometime in April 2010.

You can safely upgrade Direct3D9 without affecting either 10 or 11; they live in completely different system files on your machine.

You may not need to update. If DirectQ didn't crash as soon as you ran it, then you very probably don't. If it did crash though, you should.

Thursday, October 29, 2009

W_LoadWadFile: couldn't load gfx.wad

Some people seem to have trouble with installing Quake properly. Granted, it's an old program from the DOS days when people were reasonably expected to be familiar with arcane crap like command-lines and directory structures, so this is perfectly understandable.

To resolve this, I've just given DirectQ the ability to be able to run the executable from anywhere on your PC (or network). You can even download the latest exe (when it becomes available - hyuk hyuk) to your desktop, open it and run it directly from the Zip file (thanks to the magic of Windows Compressed Folders) and it will find where you have Quake installed and just run away happily.

Those of you who have been paying attention will remember that I had something similar in my WinMHColour program, but this is the all-singing, all-dancing version. First it checks the current directory, then a few "well-known locations" (like C:\Quake) to see if it can find it fast. If it can't, it does a full traversal of your HD (which is actually quite fast) and all other drives (including network drives) to try find it (notifying you that it's searching so that you won't think the program has hung).

Finding Quake is just a matter of looking for some stuff in ID1: pak files, maps, models, cfg files and so on: basically the same code that I had previously used for detecting a game dir.

The real reason I wrote this was purely selfish. I move between PCs semi-regularly, and I find that VC++ will always lose the "Working directory" setting from the Debug properties. So it's a convenience for me that just happens to be a bonus for everyone else.

Documentation Bugs

While writing a standalone utility to convert all D3D enum members to their string representations, I came across the following bugs in the SDK documentation (version used: March 2008; currently downloading latest version to double-check if they have been fixed; will update status later).

  • D3DDECLMETHOD does not include a FORCE_DWORD
  • D3DDECLTYPE does not include a FORCE_DWORD
  • D3DDECLUSAGE does not include a FORCE_DWORD
  • D3DDISPLAYROTATION does not include a FORCE_DWORD
  • D3DERR is not an enum
  • D3DQUERYTYPE_ResourceManager should be all upper-case
  • D3DQUERYTYPE does not include a FORCE_DWORD
  • D3DRTYPE_CubeTexture should be all upper-case
  • D3DSTT_1D is not a member of D3DSAMPLER_TEXTURE_TYPE
  • D3DSAMPLER_TEXTURE_TYPE is not documented as having a FORCE_DWORD value of 0x7fffffff
  • D3DSCANLINEORDERING does not include a FORCE_DWORD
  • D3DVERTEXBLENDFLAGS does not include a FORCE_DWORD
Now, this is just one very specific example, and it does fit in with a lot of the ranting I've done in the past about the quality of the documentation.
Update:

In the August 2009 SDK (latest version) the only one of these that is kinda fixed is D3DERR. It's still included in the "Enumerations" section in the Reference, but is at least documented as if it were no longer an enum.

Wednesday, October 28, 2009

Guess what?

Knew you'd be pleased.



There's still some tweaking to be done, but I was able to lift the code almost completely from my QII engine, so it was a very simple basic implementation.

More fun with the far clipping plane

Handling of the far clipping plane has evolved yet again. I've switched it over to dynamic at runtime based on visible nodes for the current frame and subject to a minimum of 2048. I haven't imposed a maximum on it, despite halfway thinking that I probably should. Bottom line though is that if a map is designed to have brushes visible 10 million units away from the player, then that's the way the map is intended to be and the engine should support that.

One final change I might make in this area is to switch the 2D stuff to D3DFVF_XYZRHW. This mode will cause Direct3D to bypass the T&L stage and go direct to the fragment (or pixel, depending on your chosen flavour of terminology) stage. The intent is to gain some additional performance by having less internal driver state changes/updates and matrix transforms at runtime. It's not going to suddenly gain a significant amount, but when you're coding for a resource constrained environment even all the little 0.5 percents add up. (Update: done, and slightly more that 0.5% too!)

I've also been doing some looking at the DXUT framework that ships with the SDK, and am tempted by the thought of moving my window and device management to it. This area has always been a little hairy for me, so having some code that is known to work written by the people who know D3D the best is an attractive idea to me, and may go a long way towards squashing some lingering bugs (not to mention cleaning up some codepaths). I think I can get away with it in GPL-land as it's demonstrably an external library intended for general use in other programs, rather than being part of the DirectQ program itself. There's also plenty of prior art available (just search on Google for DXUT and GPL) to support the case.

Tuesday, October 27, 2009

Not too much done today

Been writing code all day in work, so I didn't feel like doing much DirectQ when I got home. Made a few changes though; I've replaced the previous r_infiniteperspective cvar, and the whole handling of the projection matrix, with an older technique from a previous MHQuake. I now measure the size of the map at load time (basically while building surface verts) and set a dynamic far clipping plane depending on that, which is upper bounded to 32768 and lower bounded to 2048. I've also reworked the sky slightly so that it's properly compatible and so that fog looks right (and consistent) on it at variable far clipping distances.

So this means that DirectQ will no longer use an infinite projection matrix, but I'm wondering how much of a big deal it is. It's something that's really only practically useful for shadow volumes, and the main reason I had one was to allow me to draw the sky at an arbitrary distance without having to worry about overlapping geometry. Now that I've fixed that, and given that I don't have shadow volumes, I'm not losing too much sleep over it.

Of course, the real reason to do this is for better compatibility with 3D cards that may barf on infinity. It's also why non-infinite is the new way.

The beauty of this approach is that the far clipping plane will now dynamically scale depending on the size of the map (I assume that maps with dimensions > 32768 will be reasonably partitioned so that you don't see the full distance at any one time), so it handles arbitrary sized maps from small to huge with equal ease, and with no user intervention required. A tweak might be to use the visdata in the map to evaluate the max distance that can actually be seen, then build the far clip based on that.

One final thing I've done here is draw the sky at twice the standard far clipping plane, primarily to avoid any "poke-through" situations. It doesn't seem required, (the Z-fail technique used for all sky drawing should prevent it in most cases), but it was trivial to implement and feels right.

Now I really need to get back to the ProQuake messaging system...

Monday, October 26, 2009

New 1.7.4 features

I've about 80% of the software I need on now, so I fired up Visual C++ and added some more new features to the engine. Some of the menu options in the Video Options menu have now been restored to simple mode, there is an r_infiniteperspective cvar for switching off the infinite projection matrix in case it gives you trouble (I have yet to tune sky for it) and I've also added compatibility for DarkPlaces-style alias model skin replacement textures.

With regard to the last item, I'm of the opinion that DarkPlaces does one thing wrong and one thing right. The wrong thing is the directory location; it's just so much cleaner all round if you store all replacement textures under "/textures". That way, you can easily locate them, move them out of your Quake folder if you want, and so on. However done is done, and while we might now have an enforced de-facto standard that is essentially "wrong", and unintuitive, it must be supported.

The right thing is retaining the ".mdl" in the texture name; primarily because "/progs" is used for two different types of content - alias models and sprites. In ID1 there is already a case where there is an "s_light.mdl" and an "s_light.spr"; without something to differentiate them there is a risk that a texture for one type could be loaded for the other.

DirectQ used to not add the extension (dunno why, as my previous MHQuakes did), but now it does. The upshot of this however is that skin textures without ".mdl" in them are now unsupported in DirectQ. A general rule is that if it works in DarkPlaces it should also work in DirectQ (with obvious exceptions such as gloss/specular/normal/etc maps), but if it doesn't work in DarkPlaces, while it might work in DirectQ, I make no guarantees.

DirectQ on Windows 7 RTM

Just got the development environment up and have done a full test rebuild and benchmark. It runs, and it's fast; not as fast as on XP unfortunately, but not so slow that you'd notice. It's just a different baseline speed to get used to for my own benchmarks primarily.

It's a bit annoying that even with Direct3D 11 you still have to go back and install the 9.0c post-release updates; this is something that should have been gotten right.

Anyway, off to reinstall the rest of my software...

Moving to Windows 7

I'm currently in the process of moving my primary development machine to Windows 7. This is a HP dv1000 Laptop with Intel 915GM Integrated Graphics, a deliberate choice so that I can test on what I consider to be an important target platform.

This machine had a ~4 year old XP installation which had been fairly seriously abused, but which certainly gave lie to the old myth of registry bloat and Windows slowing down over time. It was still a good and viable OS install, and one which could have easily kept on trucking for a few more years yet. However, in terms of security, hardware support, and ongoing software support, it just wasn't in the same ball game any more.

As a result of this, I haven't had the opportunity to do much development work on DirectQ lately. I'm hoping that later on today I'll be in a position where I at least have a development environment that I can install and test DirectQ on built, but I've still got a fairly big job ahead of me just to reinstall all my other apps and bits and pieces.

I'll post when I'm done about DirectQ performance on this new build.

Friday, October 23, 2009

Latest batch of 1.7.4 Updates

OK, it's past my bedtime so here's what's happened lately:

  • Added v_gunkick cvar (default 1 to replicate ID Quake) and moved all kick control cvars to archive.
  • Added bestweapon.
  • Removed baseline updating from PROTOCOL_VERSION_MH (any demos recorded with this version may become invalid).
  • Added +/-quick* commands to keybindings menu and removed ancient/rarely used stuff (keyboard look, look up, etc) from that menu.
  • Switched gamma handling from mixed to pure GDI.
  • Added say/team %l/%t/etc formatting (made it case-insensitive).
  • Added ProQuake message parsing (quite crude and incomplete ATM).
  • Added "vid_defaultmonitorgamma" command to revert monitor gamma to linear if it screws up.
  • Modified COM_Parse to enable parsing of full lines as well as tokens (for .loc files).
  • Added hud_centerhud cvar to control centering of entire HUD.
The full ProQuake messaging/etc system is quite non-trivial to add; I don't think I'm going to be able to take it all the way as I don't know all of the little nooks and crannies in the code where stuff may be required, but I'll see how far it goes. Unfortunately stuff like scoreboard ping, iplog, etc is dependent on it, so I can't make any guarantees here. Right now the best case seems to be that it will probably take a few releases to get there. More likely is that I do a partial implementation and abandon the rest of it to other priorities.

Other news: I've found out why my D3D8 port was slower: for some reason mipmapping had become disabled in it. I still have the 1.7.3 code (slightly pre-release) ported to D3D8, and will continue to look over it every now and then. If I can get the correct functionality up and running properly, I'll then do another performance comparison and make a decision about porting the main codebase.

Thursday, October 22, 2009

Lots of fun with LOC files

OK, this is for the benefit of anyone who's designing a new plain-text format for use with Quake:

FOR THE LOVE OF GOD, PLEASE PLEASE PLEASE MAKE SURE THAT IT'S READABLE BY COM_PARSE!!!

This is not a joke. There is a perfectly good, tried and trusted format that is used by almost all Quake files - entities, configs, even huge chunks of savegames. Why not use it? Why go design your own funky format that requires a custom reader? There is nothing wrong with COM_Parse.

Just sayin'.


Update:

LOC file support is now added. I can't say I'm overly happy here; I had to modify COM_Parse to read a full line if an optional flag was passed to it. I'm not joking about files being properly parsable by the perfectly good built-in parser; building custom formats that require their own custom parser is just playing silly buggers for the sake of it, and being awkward. At least I've got it largely going through an evil combination of sscanf and COM_Parse now, so it's kinda conformant to the Quake standard that's used for almost everything else. Somebody badly needs a slap in the face for this.

There may be (update: is) a change of plan

I think I'm probably going to release a 1.7.4 over the next week or two. I've just added a lot of really nice stuff to the 1.7.3 codebase (including the much demanded bestweapon) that has me wishing I had waited before releasing that version, and I really want to make this available to people. I also want to see can I get any luck with that NVIDIA rendering bug.

Look on it as a nice present that's going to be the 1.7.3 with all the stuff you want in it.


Update:

1.7.4 is definite. Currently adding lots of ProQuake server compatibility, including message parsing, %h/%a/%r/%l/etc formatting, .loc support, scoreboard ping and other stuff, ProQuake precise aim, and so forth. .loc files are EVIL - fscanf should burn in hell. OK, I accept that using plain text allows you to tweak the values subsequently, but binary is just so much easier (and less error-prone) to parse in a program (yes, I'm being incredibly selfish about this).

Comments on Some Feature Requests

I frequently receive feature requests for DirectQ, so in this post I'm going to list some of these, as well as some comments I have on them. I'm going to add a new "Sticky Posts" category on the right-hand side and include this post, and anything else that I want to have more easily accessible, to it.

Improvements to the Name Maker
I'm open to requests for this. What I currently have is very much a "first attempt" which was also broken for a good chunk of time, so fixing it was more important than adding extra functionality.

Bestweapon Support
I am of the opinion that this belongs in QC, although I might hack it into the engine at some point in time. I am however incredibly wary of crossing engine/QC boundaries, as I've learned in the past that it's one quick and certain way to break mod compatibility.

Scoreboard Ping
Doing this right requires a protocol change, e.g. an svc_pingtime message followed by a byte for playernum and (possibly) a short for ping time. In standard protocol 15 Quake only the server is aware of ping times, the client just does not know them. I am aware that other engines do it, and I have looked at their code, but I find the implementation to be quite hacky. If I decide to do it, I might at the very least copy something from ProQuake or QRack, but it would be nicer if everyone could just agree on standardising the protocol change instead. It's another boundary to cross, this time the client/server boundary, and once again there is a risk of breaking standard functionality in order to add a feature that may be desirable but is not utterly essential.

CSQC Support
I have looked at the CSQC sample implementation, and porting it to DirectQ is a non-trivial job. Doing so would mean putting all other engine work on hold for a substanatial period of time. Owing to the scarcity of actual real-world usages of CSQC (outside of the theoretical or tech demo arena), the return on investment would be quite low. If a major new mod was released that utterly required CSQC, and that everyone was using, then it might become worthwhile.

Linux or Mac OS Support
No can do owing to the use of Direct3D for the renderer. Run it under WINE instead. I would love to build a Linux box so that I could at least test performance and functionality under WINE, but I don't have the time or resources to invest in it (and the quality of documentation and real-world practical examples usable by someone who has zero Linux exposure is something I find to be quite pathetic). Best case scenario is that someone else volunteers to do this job and notifies me of what code changes may be required.

Enhanced Particle System
This is something that I play with every major release, and I'll probably do so again for the next one. I have yet to find a satisfactory solution for certain effects, and I am aware of a number of edge cases where an enhanced particle system would break down. To my mind, a robust and consistent look and feel is more important. The DirectQ particle system in it's current state (as of 1.7.3) is very easily extensible to an enhanced system - it's just a matter of changing a few params for each particle type - so engine hackers can build their own from it if they wish.

Bringing back HLSL
I won't do this, sorry. The only reason I added HLSL to earlier versions was to do better waterwarps, but that created a situation where at least one person was unable to run the engine, and where it didn't play nice under WINE. The whole bumpy/shiny/specular/reflective thing can look nice, and is a viable approach for other engines, but it's just not something that's in my own personal objectives.

Removing (or making optional) Fullbrights and/or Overbrights
I won't do this either, so sorry again. Fullbrights and Overbrights are an essential feature of software Quake and are explicitly and deliberately set (and not optional) in software Quake's colormap. If your map or textures look bad in DirectQ they will also look bad in software Quake, and I'm not going to let you abdicate responsibility for doing things right, or offload that responsibility to the engine.

Any other features
I'm always open to reasonable requests, so long as there is understanding that I can't promise to implement everything, that I do have certain design goals (which I must summarise in a single post sometime) which some feature requests may not be compatible with, and that other priorities may arise that may push new features to the back of the queue. I do however evaluate everything on it's own merits, and I also do try to think carefully about implications and edge cases before deciding to implement or not implement something. I won't just say "no" for the sake of saying "no", but likewise I won't say "yes" to everything. Correct basic functionality is always more important than new features, and sometimes I might just forget something I had indicated I would do. Other times if a new feature interferes with basic functionality and if I can't reasonably fix it, I might remove it.

If you do submit a feature request I would like to see some indication that you have thought about it yourself first. It's not mandatory, but it is always nice to see.

Wednesday, October 21, 2009

Changes so far for 1.8.0

Obviously I've only been working on it for half an hour or so, so there's not too much here! It's a pity I didn't think of these before releasing 1.7.3 as it would have been nice to include them, but ah well.

  • Increased console buffer to 256k.
  • Removed dosisms.h (unused).
  • Added protocol selection to Multiplayer/New Game menu (included descriptive text for each protocol).
  • Improved texture resource management slightly.
  • Relaxed restriction on size of sky textures - can use 4x2 or upwards.
I don't intend the sky texture thing to be seriously used; it's more of a handy feature to have in case a mapper is experimenting with something non-standard.

It's always nice beginning work on a new version as it means that I'm more free to make radical changes without too much fear of disrupting a working codebase in the face of a looming release deadline. It's also nice to do some small things like this as a way of easing into the new version.

I'll post more on the major things I have planned as it begins to take shape.

And it's out!

Get it here.

If you're the first person who downloaded it you get a special prize! I forgot to include the change log with the executable download, so you now have a unique collectable item - the only DirectQ 1.7.3 executable download in the world without a change log included! Keep it for 20 years then sell it on eBay!

Everyone else gets the standard release with a change log.

1.7.3 Release is Imminent

Just been doing some more tweaking with various parts, and I'm thinking that release is actually nearer than I had originally supposed.

One interesting thing that's recently cropped up is that I've started getting the dreaded C2471 - unable to write PDB bug. Some searching around indicates that VS 2008 actually shipped with this bug present, and that it wasn't fixed until SP1 (despite MS apparently being well aware of it). Ugh, very bad form.

Anyway, I'm on VC++ Express 2008 RTM so I'm assuming - as it uses largely the same codebase - that it's also present there, but the odd thing is that I've been developing DirectQ on this platform for almost a year now, and it's never happened before.

I've an OS rebuild coming up soon, so I'm thinking that I may very well make the jump to SP1 at that time (unfortunately for SP1 Express requires a reinstall rather than just an upgrade). For now I've switched my debug info to C7 format, not really a big deal. I am however quite puzzled that it took a year to materialize, and then came totally out of the blue with no changes or anything having been made to potentially cause it.

All this, together with my previous experiences (multithread CRT issues, b0rked intellisense, etc) seems to confirm that 2008 was a pretty poor misstep in several crucial areas. I am however looking forward to seeing what 2010 has to offer.

Despite all it's quirks (and they are quirks which I am used to) I do believe that Visual Studio is probably the best IDE out there for certain kinds of development, and 2008 has some nice features which definitely and measurably enhance my own productivity, and which I would find it hard to live without. When all is said and done I'm a pragmatist rather than a purist.


All that aside, watch this space for release news!

Tuesday, October 20, 2009

Alllllmost there...

1.7.3 is getting very near ready to be unleashed. I've just put the latest touches on the simple menus, so I think I'm pretty much done with them for now. The major outstanding item is the name maker, which has been broken for all of the 1.7 line.

I almost started diving in and rewriting the surface refresh, but the realisation that it's a fairly huge job drove me off. This is still reserved for 1.8. Another thing I want to add for 1.8 is support for 1 TMU and 2 TMU cards, which has never been in DirectQ. The major codepaths affected by this are brush surfs, alias models and sky. It's not overly critical though, as there are really very few of those left; it's only people with really really old hardware (that would struggle with the basic render anyway) that this is for, but all the same it's on the "nice to have" list.

Another item I've been considering is paletted textures. Direct3D seems to have some pretty good support for dynamically changing palettes at runtime, and that would be a rather neat solution to the whole issue of colormaps. Unfortunately I can't even get the API working without a crash, and the documentation is scattered across various different parts of the SDK with no links between the related items (again!) so it's difficult at this stage to say which version - if any - they might appear in.

One final big surprise for 1.8 might be a reversion of the renderer to Direct3D 8. I even caught myself out thinking along those lines, but it does mean broader hardware support, and it's possible that the API might be faster. Right now though it appears as though the D3D8 headers and libs won't even link correctly under VCPP 2008 Express, but if I can get past that hurdle it's something that might be worth trying. No promises.

Just did a fairly heavy run through some of SDQuake to test some limits and stability; everything is good. Man, this mod is mental! I don't know whether some of the features in it are just plain annoying or not (especially some sound effects), but the gibbing and firefighting is quite intense.

If nothing dramatic comes up I might bring the release forward to this coming weekend. Wait and see.


Update:

The name maker is now fixed and restored to the Multiplayer/Setup menu.

I've managed to successfully compile and run some of the DirectX 8 samples on VC++ 2008 Express, so that switchover is looking more and more likely. I might take a copy of the code and do a test changeover right now (i.e pre-release) so that I can do some performance comparisons.

Paletted textures require support for D3DFMT_P8 which I believe is quite rare (my primary dev/test box doesn't have it either) so that's not an option.
Update 2:

I've ported a copy of my current codebase (rather crudely) to Direct3D 8, and it's actually a little slower than 9, which is a strange result as I was assuming that the earlier version of the API would have simpler code paths/etc. Of course, some of it may be down to the crudeness of the port, but all the same I think that the actual runtime stuff is well enough done. My current best guess is that 9 contains significant optimisations over 8 in the basic code paths.

For all that Direct3D has a reputation for changing the API completely with each new version, 8 and 9 are incredibly similar. There are some new params in some places with 9, some restrictions have been relaxed, and some flags have moved around a little, but it's nothing too dramatic.

I don't think 8 is going to be viable though (I may have accepted the speed loss in exchange for broader hardware support) as it lacks D3DTSS_CONSTANT/D3DTA_CONSTANT (it may have an equivalent which I just didn't find) which I make quite extensive use of.

All the same, it was fun and I might go back and do it seriously again if I can find an adequate replacement way of doing the constant stuff. At least I've established that it can be done without any pain, which was important.

Monday, October 19, 2009

More Menus

OK, been working on the simple menus option and it's just about done. Here's what I've done with it so far:

  • Restored the chunky pictures, spinning cursor and Quake plaque for Main, Single and Multi.
  • Stripped out all options from Main that weren't in ID Quake.
  • Removed the Help/Ordering option from "advanced" menus.
  • Made simple menus the default.
  • Restored the simple Options menu to ID Quake items, without lookspring and lookstrafe, but with Save Current Configuration and Mouse Look.
  • Removed all items from the simple Video menu aside from resolution and anisotropic filtering.
  • Added a menu_simplemenus cvar to control the whole thing.
The menu_simplemenus cvar is exposed as a menu option on both the simple and advanced Options menu. The way I reason it is that sooner or later a player is gonna want to try something that's not exposed in the menus, and this is one item where they will get a hint "ah-ha - maybe if I look here I might find something".

At the end of the day menu options - even quite overwhelming ones - are far less hostile to players who may not be familiar with Quake weirdness than the command-line or cvars can ever be, and I want to be able to leave a way open for these people to record demos, change games, play with fog, and suchlike if they are that way inclined.

Toggling Simple Menus off will immediately switch on the full effect, which - if anyone finds it scary - they can just switch off again quite easily.

So here's what we have:



I'm still open to suggestions along the lines of "if you're going to be playing with those other options, you really should also be familiar with cvars anyway", but I'd like it if any such suggestions also came with ideas for how to make the likes of the Demos menu more accessible.

There may be one or two other items to add to the simple Options menu (and possibly one or two to remove, like Screen Size - does anybody use that anymore?), but I think I'm pretty much there with it; I'm definitely not going to build a "simple menu" and then start going overboard with it!

Menus!

One item of feedback that I've now received twice is that my menus can be a little - how shall we say - overwhelming. Now, I deliberately made them the way they are because I wanted to expose a lot of options to the player that normally one would have to go fooling around with either the command-line or the console for. Whatever about the menus, I believe that the command-line and the console are unduly hostile to folks who just want to play the damn game without having to delve into arcane crap.

However, I am taking the feedback on board as I'm also aware that a lot of people do just want to play the game with a set of sensible defaults, just need the stuff that ID originally provided, and couldn't give a flying one about the other options I've exposed.

The solution I've decided on is to add a "Simple Menus" toggle. This will essentially strip the Main Menu and the Options Menu of most of the extra stuff I've added - these are the two I believe are the biggest culprits here. I may also remove some stuff from the other menus in "Simple" mode - we'll see. The toggle is going to be available in both "Simple Mode" and "Full Mode", so you can switch between them if you don't like what you currently have.

It will require some small rearchitecting of the menu system - basically moving it to a stack rather than using a fixed previous pointer - in order to support ESCing back to an arbitrary previous menu. Or I might do it another way.

I might also restore the big chunky friendly Main Menu picture in "Simple" mode, but I'm not fully decided yet, so don't rely on it happening.

Two questions I'm currently pondering are: (1) should I even bother displaying the toggle in a menu or should I leave it as a cvar only, and (2) what should the default be (currently leaning towards "Simple")? What do you think?

1.7.3 Latest Updates

Here's the list so far for 1.7.3, including updates from the last few days:

  • Standardised file I/O further.
  • Prevented PK3 support from creating a folder structure under %USERPROFILE%\Local Settings\temp.
  • Removed nehahra cutscene cvars from checking/enumeration for non-Nehahra games.
  • Added new PROTOCOL_VERSION_MH extending coordinate system to 3.402823466e+38F.
  • Improved accuracy of angle quantization for old protocols.
  • Changed angles to quantized shorts instead of full floats for new protocol (bandwidth and packet size).
  • Added improved baseline comparison/updating to new protocol.
  • Optimized translated player skin uploads to resolve multiplayer stalls and hitches during intense firefights with lots of players.
  • Added autocompletion for alias commands.
  • Restructured command execution for better performance (binary search instead of linear search, etc).
  • Increased command buffer to 1MB.
  • Removed last occurrances of std::vector and std::algorithm.
I'm still intending to release this one by the end of October, which probably means sometime next week! I do need to go back and remind myself of a few things that need to be done for it though: one of the bad things about taking a break...

Saturday, October 17, 2009

Inside3D Review/News Item and consequent bugfix

It's rare enough that I look at the main news page of Inside3D, but today I did and saw that there was a news item/review on DirectQ 1.7.2 there (thanks folks!) One point of interest to me was a note that the engine was quite slow on the bigass1 demo. Now, I can't resist a challenge like this, so I downloaded the demo and played it, and yup - really slow, with lots of hitching and stalling.

So on a hunch I went to my player skin translation routine and stuck in a Con_Printf to see if that was the culprit, and yes, it seems as though DirectQ is running a new skin translation on every respawn (rather than when the skin actually does change). I don't know if this is inherited from original GLQuake, but I more than half suspect that it is (the original comment about doing it fast instead of sending through GL_Upload8 in the ID code is the giveaway here).

A further test was to just put a return at the start of this function and rerun the demo which confirmed that things were nice and smooth and with no hitching or stalls.

This has now been fixed by a combination of caching, some memory optimizations and a few other small things. The difference is like night and day; sweet.

For the record, the original Inside3D item that prompted this can be viewed here.

Once again, thanks due to Inside3D in general and scar3crow in particular for this one, without which a fairly evil performance bug would have continued into future releases. I've said it before but I'll say it again: I love being notified of issues like this as it draws my attention to areas of the engine I need to do better on.

Thursday, October 15, 2009

More Particles (Yet again!)

I've decided to drop the idea of an enhanced particle system.

At the end of the day, full and proper support for it in Q1 is something that I feel is flaky at best. Having a consistent and robust look is just far more important than any amount of flashy effects, and where particles are concerned there are just too many edge cases where things can go wrong. This is not a problem for ID1, where everything is well defined within the limits of the standard engine code and QC, but is a problem for mods which may do any one or more of the following:

  • Replace particles with their own sprite-based effects.
  • Use replacement content that does not conform to the ID1 conventions.
  • Spawn particles at unexpected places and with unexpected params that may conflict with assumptions made in the engine code.
  • Use different model types for effects.
  • Anything else I can't foresee.
The way I see it now is that I would be placing the engine in a situation where I'd be playing constant catch-up with any mod that comes out and does something new, and that is just not worth it. There are far more important things to be prioritising.

I could offload responsibility to the user (or modder) by e.g. adding a facility for supporting custom particle scripts, but that is not consistent with the design goals of DirectQ (it is a valid architectural choice for other engines though).

I'm going to leave the full support as of 1.7.2 level for enhanced particle effects in the engine though, so if anyone wants to hack on this code to build their own enhanced effects they are free to do so. It's just a matter of modifying some parameters in the particle spawning functions, so it should be a trivial enough task.

Where this places things is that I can now proceed with the other changes and fixes I have planned for 1.7.3 with a view to releasing by end October. This will be the final release of the 1.7 line, as I need to start knuckling down to the cleanup work on the world refresh which, as I had mentioned before, is now in a state of fairly advanced ugliness.

Wednesday, October 14, 2009

More Particles

I ended up getting nothing done yesterday aside from reverting my QMB particle system changes back to classic (1.7.2) style. I'm still very much in thinking mode about particles and how best to handle them, primarily on account of potential concerns with explosions.

The real problem with explosions is that sprite that's spawned from QC. Finding a particle explosion effect that looks good and plays nice can be quite a challenge here. Now, in more naive days I would just check the sprite model name and not bother rendering it, but the problem with that is the fact that the actual model that's rendered is determined by QC, and is therefore outside of the engine's control. There's nothing to guarantee that it will always be s_explod.spr, and it could even be an alias model or a BSP model in some mods.

I'm not really satisfied with the thought of building an enhanced particle system but leaving explosions be as to my mind the visual discontinuity this would generate would be far worse than just retaining the classic system on everything. On the other hand, I don't know (yet) how QMB handles this, so there may already be a solution there waiting for me.

Overall, on the QMB front, my current estimate is that completing the job would require about a month. This time would include porting to Direct3D, bringing the classic system on top of that, re-integrating with a lot of the new DirectQ stuff, and all the time bugfixing and error checking as I go. At the end of that time I would likely be sick to death of the code and possibly not caring too much if I never saw it again. Hence the rollback and rethink.

As it stands, the current 1.7.2 particle system is actually quite capable of some very enhanced effects without any code changes. So all that's required to build an "enhanced" system is adjusting a few parameters for each particle type. On reflection, going QMB was one of those nice ideas that just don't work out in practice.

But the problem of explosions remains something to be addressed, so while I may experiment with some effects in some places, I might not finish the system off for 1.7.3 (hey, I do wanna get the new code changes out before 2010!)

Monday, October 12, 2009

Thinking about particles

Now that I've had a break I'm intending to get some more work started tomorrow. Sometimes backing away from the code is a good thing as it let's you view things from a different perspective. I've realised that I'm probably going about implementing the QMB particle system in the wrong way, and that instead of trying to port it on top of my current system I should instead drop-n-go it, fix all errors, re-implement infinite particles, then port the original particle system on top of it as the default option.

More news as it happens!

Friday, October 9, 2009

Wow! I'm back!

That was awesome! Gonna break from DirectQ for a few more days, but start expecting to see updates appearing again shortly.

Thursday, October 1, 2009

1.7.3 Updates So Far

Before I sign off for a week and a half, here's what's been done to date for 1.7.3:

  • Standardised file I/O further.
  • Prevented PK3 support from creating a folder structure under %USERPROFILE%\Local Settings\temp.
  • Removed nehahra cutscene cvars from checking/enumeration for non-Nehahra games.
  • Added new PROTOCOL_VERSION_MH extending coordinate system to 3.402823466e+38F.
  • Started porting QMB particle system.
  • Improved accuracy of angle quantization for old protocols.
  • Changed angles to quantized shorts instead of full floats for new protocol (bandwidth and packet size).
Perhaps the single most noticeable change here is the use of quantized shorts (instead of quantized bytes) to represent angles in transmission. One effect of this is that hipnotic rotating brushes are no longer jerky but instead incredibly smooth. You will be able to see this for yourself on the hipnotic start map.

Unfortunately zerstorer cutscenes are still jerky. I might experiment with full floats to see if that makes any improvement, but if so I will need to increase the max packet size as we would be talking about 4 x data size for angles!

One important point to remember about protocols is that the server controls the protocol. If you use DirectQ to connect to a protocol 15 server, all you'll get is standard protocol 15 behaviour.

That's about it until the 15th October or so.