Sunday, August 31, 2008

Game Changing Revisited

I've had a rethink about my approach to game changing, and what's allowed and what's not allowed, and have decided to back off a little on my more proscriptive stance. The games menu has been reworked to allow for this, but I still need to go back and rework the functions that actually change.

Now, MHQuake will divide games into two categories: "Expansions" and "Games". An "Expansion" is something that was previously hard-coded into the engine, or was written with the intent of being used in conjunction with a game. This includes "Hipnotic", "Rogue", "Quoth" and "Nehahra" (I don't have full support for the latter 2 yet). Specifically, it's replacement content. A "Game" is just a directory under "Quake" that contains maps, models, progs, etc. In other words, additional content.

Using this model, you can now select any number of "Expansions" that you want (all 4 if you're feeling lucky, punk), as well as one game. This perfectly replicates the stock ID Quake command-line behaviour. If you select more than one "Expansion" you'll get a warning to only proceed if you know what you're doing (I'm still convinced that they're not meant to be used together). If you select any "Game" other than ID1 in conjunction with an "Expansion", you'll get a similar warning. I hope to get rid of this latter warning if I can devise a means of detecting whether a "Game" is meant to work in conjunction with an "Expansion".

The order of loading is "ID1" first, then "Expansions", then the "Game", just like in the original.

If this sounds more complex than it need be, don't worry as I've hidden most of the fiddly stuff away. A "Menu of the Day" might explain things better:



(Note no Nehahra in the above as I haven't started that yet...)

I detect a valid game directory by looking for one of progs.dat, config.cfg or pak0.pak in a subdirectory off "Quake". There's a few it's going to miss, of course, but it's better than adding in too many (not everything under "Quake" is necessarily a game!), and there's always the console to deal with that. I might add in a few more checks to be sure of getting as many as possible.

Right now the console "game" command is more limited - it just lets you change to a single game. I have plans to make it even more powerful though (as the console should be) by adding in the ability to add as many extra game directories as you want completely on the fly.

The command-line is still fully supported, of course, so typing freaks can still get their jollies there (joke! it's also good if you specifically want to start up in a game other than ID1).

Overall I'm happier with this setup, as it replicates the original more accurately (perfectly, in fact) and reduces the possibility of mod authors complaining. Which is always a good thing.

An interesting realisation...

Currently reworking my game changing code, and I've just hit on a very interesting realisation - in stock ID Quake it's possible to have up to 4 games loaded simultaneously!

The following is a valid command line:

quake -rogue -hipnotic -game mygame
By doing this, you'll have ID1, rogue, hipnotic and "mygame" (whatever that might be) all simultaneously loaded. This of course will cause all sorts of interesting side-effects and conflicts. How do you know which progs.dat are you using? What if you have a model conflict? What happens when you type "map start" at the console?

Now, I'm pretty certain that I'm not the first to realise this (and that I am in fact fairly late to this particular party), and that there are mods out there that rely on this buggy and unpredictable behaviour to work. So here is my official position on it:
  • I do not support loading both rogue and hipnotic simultaneously.
  • I do support loading an additional game on top of either rogue or hipnotic, but only from the command-line.
  • I do not support loading more than one additional game on top of ID1 from the in-game menu or console.
This way, if you are running a mod that acts as an "expansion pack to an expansion pack", it will work, but only if you explicitly do it from the command-line. I assume that if you take the trouble to type it, you know what you're doing and are prepared to take any consequences. If you're trying to do something genuinely weird like the command-line example I gave above, it will not work - this is intentional as it would have likely either crashed the engine or given an unpredictable result to begin with. If you're switching games from the menu, none of this weirdness will work, and you'll only have the game you've explicitly selected active.

I am pretty certain that there is no way on earth ID would have intended this behaviour, and that it's one of those wacko exploits that mod authors seem to love. Unfortunately we're stuck with it, and I've learned from past mistakes not to go "fixing" things like this. Oh well...

Thursday, August 21, 2008

This was certainly unexpected...!

I've just added back in the alias mesh cache - you know, the old "glquake" directory. However, rather than keeping it on disk, I now keep it in memory, which enables me to have the advantage of faster alias model loading, but without the disadvantage of clashes owing to the same model name in different games. I can also flush the entire cache on-demand, as well as flush out least recently used models as the engine runs. Total cost never exceeded about 150K during tests on ID1, so even in the most extreme mod you're unlikely to go over half a meg of overhead.

Hey, it's 2008 - machines have more memory, so let's use it!

Sunday, August 17, 2008

The death of features....

It's killing me to do this, but it looks as though I'll probably have to remove torch coronas. They were one nifty bit of graphical gloss that really added to the atmosphere, but I've run a few test maps with lots of torches in the scene, and they really do murder the framerate. I might see if I can concoct another way to do them (in fact I already have one in mind) that doesn't involve some of the more CPU and GPU intensive methods I've been using.

Friday, August 15, 2008

Demos Menu Finished

It's been a while since we had a "Menu of the Day"...



In other news... I've changed the autogenerated save game name from a GUID to a date/time, so I can sort them in reverse order giving you the most recent save on the top. They're still not sorted by last write time though, just by creation time, so I need to fix that. Always busy, oh well...

Wednesday, August 13, 2008

Latest Updates

Some days I get loads of things done, some days I don't. Other days are weird, either I write loads of code with no real result, or I write very little code with a very good result to show for it. Today was one of the latter; the main thing I did was rewrite my file searcher function as a class, which really really cleaned up a lot of ugly code as well as removed certain (fairly high) hard limits I had on things.

I can now replace several hundred lines of code with something like this:

CFileSearcher *ListSearch = new CFileSearcher (path, mask);

while (1)
{
char *FoundList = ListSearch->GetFile ();
if (!FoundList) break;
Con_Printf ("%s\n", FoundList);
}

delete ListSearch;
Works with the full filesystem search paths, PAK files as well as the raw directory structure, cleans up after itself, and gives me a single place to debug. Very nice.

Tuesday, August 12, 2008

More tidying up

Just one small refinement on the maps menu - I've added a check for an info_player_* entity in each map before it's added to the menu, so that item boxes won't get added. Because I create the maps menu dynamically (so I can adjust it if the game changes) this has added a delay to the first load of the menu. On some systems this might be unacceptable, so I've also done a pre-load of the menu.

I've posted a request on the QuakeOne forums for ideas for the Demos menu. If there's something you believe it needs, I'm interesting in hearing about it. I'm just building basic functionality right now, so once I have that complete I'll post a shot so that you can see where it's currently at.

Meanwhile, I rather like this screenshot:

Monday, August 11, 2008

Visual Studio 2008 - The Horror Lives On

There's definitely something bizarre happening with either the compiler, the linker or the CRT. A new bug has now surfaced, causing a "map" command to crash the engine when issued during the startup demo loop. At this stage I consider it a waste of my time to work any further in this version; I'm reverting back to 2003 (where even using the very same project options doesn't have any trouble at all).

Now that I'm back on track with a development platform that can at least be relied on to behave in a predictable manner, I've dug in and done some serious debugging on a few items; mostly in the save/load menu. There were some issues there that had mounted up over a while, so I've finally fixed them. These include:

  • Mapshots not working right on Intel - it turned out that Intel cards used a non-standard page flipping policy, meaning that the previous framebuffer was actually used. The simplest solution was to flip buffers twice (also works fine on non-Intel). I'll probably revisit this, as there's a flicker while the shot is being taken which I want to get rid of.
  • Save Enter/Delete key crashing - I was missing a return!
  • Delete key from the load menu returning you to the save menu - just a mixed up pointer.
  • Added save time to the menus (yyyy-mm-dd hh:mm - there wasn't room for seconds).
The surface refresh is also fixed. Fortunately I didn't have to rewrite it! It turned out that duplicate textures (i.e. same texture data but a different name) in certain maps were the cause here, which when coupled with my using a texture list built at loadtime (rather than cl.worldmodel->textures) in my texture chains caused some texture_t structs to not have texturechains built for them. Just a simple matter of switching back to cl.worldmodel->textures.

Sunday, August 10, 2008

Visual Studio 2008 - Now Working

MHQuake is now working under Visual Studio 2008. The last thing I did was load progs into a temporary buffer, then malloc a permanent home for it and memcpy it in.

Now, it's obvious that so far as memory is concerned, progs is a wildcard. I haven't investigated fully yet, but my own inclination is to leave things this way. I could very well invest a lot of time and effort into tracking down exactly what's happening, all for the sake of a single memory allocation that can easily and cleanly be handled outside of the main subsystem. If it was happening with other things, then it would make sense, but so far (and I've tested a lot) it's only with the progs.

Friday, August 8, 2008

Visual Studio 2008

Most Visual Studio 2008 problems have now been sorted. I'd switched DirectInput to unbuffered some time back, which has completely resolved the threading issues. What's remaining is a PF_VarString crash. I've definitely confirmed that this is coming from the entered/left the game (with frags) broadcast messages, originating in CL_Disconnect, going through Host_ShutdownServer and SV_DropClient, which calls into QC to generate a signoff message (the only case where this call path doesn't apply seems to be a "changelevel" command, but that ends up in PF_VarString with the same crash anyway). It doesn't appear to be a buffer overrun, as even doing strlen on the G_STRING result crashes. There's definitely call stack corruption though; one other symptom is a jump into a random section of code elsewhere.

My current theory is that my new memory management system is at fault. Progs.dat is being loaded on the hunk (it's not really a hunk anymore, but I'll use the familiar terminology for convenience and legibility), and going astray somewhere at disconnect time. I'm going to put it into main memory and hopefully that should resolve things once and for all.

Wednesday, August 6, 2008

Scratching the Itch

I wouldn't claim by any means to be one of the "great hackers" out there, but I can definitely understand and relate to whoever it was wrote about them feeling a need to "scratch the itch". For me it's been menus recently, and I certainly got the itch bad while doing LAN tests yesterday. Those old multi-player menus looked really ugly and clunky when sitting beside my new ones, so I dug in and almost finished them. I'd been putting them off as the originals contained some of the ugliest code in the entire engine, but this was the final straw. The remaining items are to add the maps list to the Game Options menu (no idea how I'm going to handle that yet; right now it just launches dm1) and do some serious debugging on my all-singing all-dancing text input box control. That will see all of the menus moved over to my new system, so everything else is just layers of varnish (and potentially adding new menus - a Demos menu is definitely coming). I'll probably also reorganise the menu code into multiple files, as my menu.cpp is fairly large and awkward to navigate.

Uncovered a few more bugs too, which are going into the to do list.

In the spirit of devoting this week to Real Life, I'm knocking it on the head for the rest of this evening.

Tuesday, August 5, 2008

Play Tests

OK, I said I was taking a break, but one of the objectives in writing this engine is to create a Quake that I would like to play, and so today I played some Quake. First thing I did was set up a LAN and verify basic connectivity using a GLQuake to GLQuake test, which worked a charm. Then it went MHQuake to GLQuake, and finally MHQuake to MHQuake. Glad to say it all worked very well. Some slight glitches in MHQuake's handling of client updates - I need to wipe the entity states after a respawn, but nothing too serious. Also, looking at the old network menus is fairly cringe-inducing - they are pretty ugly compared to the new ones! I'm almost tempted to cut my break short and bring them up to scratch.

I'd like to find a way of identifying when an MHQuake client is talking to an MHQuake server, maybe by means of hiding a "magic bullet" in one of the entity updates, although that sounds a bit hacky. If I can get that, then Bob's yer uncle, and we have fair game at least so far as LAN work is concerned.

Sunday, August 3, 2008

Taking a break

I'm going to be taking a break from MHQuake for a week or two. This is pretty much of my own choice - I feel that I've accomplished a lot and taken the codebase somewhere good over the past month, so it's time to devote some time and energy to Real Life. There's books to be read, people to go drinking with, a whole lot of stuff to catch up on, and a whole lot of fresh air and sunshine to enjoy!

When I come back I'll be starting on what feels like a final push to get this baby released; some of the features I've planned (particularly the sound subsystem overhaul) will likely end up being held over to the one after (gotta have an incentive for the next release!)

Saturday, August 2, 2008

Updates

Haven't done too much these past few days, as I've been caught up in Real Life again (nothing bad or catastrophic, just going out and having fun with friends), but I did manage to add in the Point Parameters based round particles, and a few other things. If memory serves, square particles are still the default for the "Traditional Mode" toggle in the menus, but they can always be set to round subsequent to that (did I mention how easy my new menu system makes it to add new options - no more messing around with _Key and _Draw functions - yippie!) The square particles also use Point Parameters, by the way; they just don't enable GL_POINT_SMOOTH during the setup.


Based on an ongoing discussion on Inside3D I've made some small adjustments to the protocol for Single Player games only. I'm not going to make any adjustments to the Multi-Player protocol until I see what comes out of those discussions, but things seem to be moving in a good direction and this time we might have a chance of something good and useful emerging. The Quake engine community is, and always has been, too fragmented, and now that there are fewer players involved, maybe it's time to start nailing down some good solid standards for things like this. We can still all be free to fly off on our own tangents with a lot of things, but when it comes to something as fundamental as how two different engines talk to each other, we need a standard set in stone. No messing can be allowed.

One concern I have is that it will degenerate into squabbling over which ideas are best, and end up getting nowhere. We have to be prepared to accept that we need a fundamentally solid standard that we can all move forward with, rather than getting lost in the Eternal Quest for Absolute Perfection (TM). That's no use to anyone, and we end up right back where we started with nothing to show for it but a whole heap of wasted time and energy.

Anyway, moving on to what I've done; I've changed the way the server side and the client side work with baselines a little. Instead of setting a once-off baseline on connection, MHQuake now tracks the previous state on a continual basis. Only if the previous state changes is an update sent. Tests show that this halves the amount of communication quite effectively.

As I said before, this change is for Single Player only. As it requires changes on both the client and the server (as both track the baseline) it's not suitable for when two different engines talk to each other. In such cases, the default behaviour is used, and the initial baseline is tracked.

More to come on this front, including increases in the maximum datagram size, which should really help with playing those massive SP maps out there. MP will stay the same as default Quake, pending agreement on a new standard protocol (and also pending agreement on a protocol negotiation method so that both client and server can figure out which protocol they should be using).

One final observation - my removal of the limit on the maximum alias frames isn't worth anything without a protocol change, as the current frame is sent across in a byte value. Still, it feels like it was the right thing to do, so I don't really mind.

Friday, August 1, 2008

Classic Particles!

One of the easiest reversions ever, classic particles (and I do mean classic - they're even square!) have returned. I've had to switch off some particle based effects for them, in particular gun barrel smoke and impact smoke, as they really looked bad.

Depending on if enough people holler for it (and people - you will have to holler - if I can't hear you I won't know if you want it) I might add in round particles, using Point Parameters.