I've decided to add the QMB particle system to DirectQ. It's not quite drop 'n' go for a number of reasons, including the facts that I'm Direct3D, I'm C++, I have infinite particle support, I already have my own half-built system to work around, and I want to retain the classic system as the default option. But at least it's a particle system that's reasonably well known and will give expected results.
In case you missed it above, it's worth repeating: I WILL BE RETAINING THE CLASSIC PARTICLE SYSTEM AS THE DEFAULT. You can stop complaining now.
Anyway, so far I'm just working on the scaffolding around it, which is fun. This includes bringing the QMB textures across as embedded resources (no external files needed), setting up the structs, doing the init code, and so forth. No screenshots yet, of course, but as soon as I have something worth showing I'll put one up.
Typically though, this is happening just as I'm about to leave the country for a while, so don't expect to see anything for two weeks minimum.
Tuesday, September 29, 2009
Particles! At last!
Posted by
mhquake
at
9:49 PM
2
comments
The Joy of Protocols
One area where DirectQ falls short by comparison to some other engines is in it's coordinate system. DirectQ coordinates are fairly standard, with a max of 4096 (it's actually 0.125 less, but we won't quibble). To address this I've just added a new protocol to enable me to extend this out to stupidly huge values. This new protocol is number 10003, and now becomes the default; the other previous protocols (15, BJP, BJP2 and BJP3) remain as selectable options via the sv_protocol command, of course.
Instead of quantizing coords and transmitting as a short, or even as a long, I've decided to transmit the native float data. Coords are stored as floats on both the client and the server anyway, and the quantization/reduction only has effect during message transmission, so why not?
The new max coordinate for DirectQ is 3.402823466e+38F.
I've also done the same to angles, so we'll get more accurate aiming. Otherwise it's functionally identical to PROTOCOL_VERSION_BJP3.
Implementation details of this protocol are subject to change between now and release, but post-release I'll likely end up freezing it and adding any further extensions to a PROTOCOL_VERSION_MH2, retaining the original PROTOCOL_VERSION_MH for compatibility reasons.
Posted by
mhquake
at
7:17 PM
0
comments
Next Release News
I've given the matter some thought, and right now it's looking fairly likely that a Release 1.7.3 actually is going to happen. There are a few places in the recent code where I haven't set things up as cleanly as I would have liked, and so I want to go back and rework them. I also want to fix the name maker. These should have all been done for 1.7.2, but it ended up not being so owing to the requirement to get a bug fixed version out sooner rather than later.
I might also bring some of the changes I have planned for 1.8 forward to it, just so that there's a little bit more present than mere tightening up of code. This would also mean that 1.8 would contain less radical overhauls of code, meaning that it would be simpler and therefore less bug-prone, which is all good stuff.
So until you hear otherwise, you can assume that 1.7.3 will appear sometime towards the end of October or early November.
Posted by
mhquake
at
4:55 PM
3
comments
Monday, September 28, 2009
I think 1.7.2 is done
Pretty much everything has just finished coming together. PK3 support was just recently completed and tested, and all seems to be working correctly. I've added vanilla ".zip" as a viable extension here too, just because it's easier to work with ZIPs than PK3s for testing purposes. ID's rationale in Q3A for changing the extension (to prevent anyone from messing with them by accident) remains valid however.
One thing that is not done is the name maker. This broke when I rewrote the memory handling routines, and it remains broken now. It doesn't crash however, it just doesn't work. For now I've commented out the code that adds it to the menu, as I don't want to delay releasing and it's more of a "fun" feature than something essential anyway.
I need to do some final testing just to ensure that nothing evil crept in with the recent changes, and then hopefully I'm releasing tomorrow (Tuesday 29th) PM (GMT).
Here's the full list of changes for 1.7.2:
- Fixed "illegible server message" on demos recorded on engines that send U_TRANS but don't change PROTOCOL_VERSION.
- Transitioned much file input to Win32.
- Fixed continuous "mode is unchanged" message.
- Resolved more clipnodes > 32767 issues.
- Added host_maxfps cvar for control of Host_FilterTime (default 72).
- Added Nehahra MOVETYPE_FOLLOW support.
- Bumped max nodes to 65535 by implementing arguiRe's hacky fixes.
- Implemented optimized cullents tracing via suggestions from LordHavoc; switched default to 1.
- Implemented variable lightmap sizing thus allowing surface extents to go up to > 32768 on typical hardware.
- Fixed bug where a worldmodel which has angles set will cause incorrect clipping behaviour.
- Fixed bug where link files were not working correctly.
- (Temporarily) Removed name maker from Multiplayer setup menu.
- Added PK3 support.
Posted by
mhquake
at
8:07 PM
1 comments
Fixed it
I wasn't going to let it lie. ;)
The source of the bug was in SV_ClipMoveToEntity, where I had included the Q2 rotating bmodel code. It turns out that e3m3 somehow has an angles[1] of 180, so clipping against the world in this case ran through the rotation code. The simple fix was to add && ent != sv.edicts to each test.
Lesson learned; even if something like this makes no sense at all, one should never assume that it's not actually set.
The good news however is that all I need to do now is finish up my PK3 loading and then I'm releasing!
Posted by
mhquake
at
2:43 AM
1 comments
Tomb of Terror bug
OK, things change and an earlier release is now looking like it will be required. There is quite a nasty bug in e3m3, which needs to be resolved NOW. It's too early to say what the fix is, but as soon as I do fix it I'm releasing.
How this slipped through I'll never know, as e3m3 is one of the maps I test most regularly on. Anyway, plan of action is to go back to the last version that worked and build up from there till I locate the cause.
Meanwhile, I've ported the PK3 loading code from Q3A and have most of it working. The only real remaining thing to do is decide how I'm going to handle files. Do I open them into a temp folder or keep them entirely in memory? I'm going to try a temp folder, but the various SHGetSpecialFolderLocation and other related functions make baby Jesus cry, so these will require a bit of careful massaging to get them behaving themselves properly. Sigh... why couldn't MS have just provided a simpler version that just filled a char * buffer? Integrating them into COM_FOpenFile will be "fun" too...
This is something that I'm aiming to include in 1.7.2 as well as the bug fix and the other minor changes so far.
Posted by
mhquake
at
12:23 AM
0
comments
Sunday, September 27, 2009
Fun with Surface Extents
OK, I've followed up on my threat to revisit the lightmap generation code and implement variable sized lightmaps. DirectQ will now successfully generate a lightmap big enough for any size of surface extents up to the max your card can support. What this means is that maximum surface extents are now 4080 for a 3DFX (good luck running DirectQ on one of those!), up to 32752 for an integrated Intel card, and 65520 for a typical modern card (supporting 4096 x 4096 textures). Most Quake lightmaps are generated at 64 x 64, so we get the speed increases as well as the flexibility to have larger extents if we need them.
Of course, I already had a dynamically sized BlockLights array from a few generations back, which is also a requirement of this.
Posted by
mhquake
at
9:52 PM
0
comments
More 1.7.2 updates coming in
A few more updates were just done, including fixing one extremely rare crash bug. This one only occurred when surface extents were greater than 1008; DirectQ allows for extents up to 2032 (i.e. LIGHTMAP_SIZE * 16 - 16) but when I reduced LIGHTMAP_SIZE to 64 I didn't also reduce the max surface extents. This could have caused a buffer overflow, and consequent crash, in maps that had really really large surfaces. I don't consider it serious enough to warrant a more immediate release, as the bug only occurred in maps that has surface extents well in excess of what ID Quake allowed.
The fix was to increase LIGHTMAP_SIZE back to the standard of 128. Tests indicate that use of 64 is slightly faster, but the increase is somewhat marginal and being able to support larger extents seems preferable to me right now.
I'm probably going to go back to the lightmap code at some point in time and allow variable sized lightmaps, as a more general solution is preferable to my way of thinking. The default will be 64, but a lightmap will be able to be generated at 128, 256, 512 or even higher if required.
Another change came from an Inside 3D discussion here. This is quite a nifty solution to the problem of exponentially slow anti-wallhack, which is also applicable and useful in single-player games. As a result of having done this I've been able to switch the default value of sv_cullentities back to 1. It was originally defaulted to 0 as I had felt that it was just too slow to justify being on by default.
Credit needs to go to LordHavoc for the suggestions and discussion leading to this.
As well as behind-the-scenes changes I like to have something more immediately visible in each release. I've been thinking a little about what this could be for 1.7.2, and right now I'm leaning in the direction of PK3 support. The plan is to port the Q3A code so that there won't be any dependency on external DLLs (something I hate). It's not final yet, but right now it seems to be something useful that could be easily enough done.
Of course, there are other consequences. PAKs are easy to work with as I can just create a FILE * pointer (or a HANDLE in this version of the code) in the main COM_FOpenFile routine and then read normally elsewhere if that's the way I want to load the file. I'll probably end up extracting to your temp directory and creating a HANDLE on that instead, but it's still early days and all liable to change as soon as I start writing code.
As for a release date, I'm happy to say that I finished on my legacy codebase stuff in work on Friday just gone. I am off on holidays/vacation/archaeology-trip-stuff for a while shortly though, so roundabout mid-October I'm planning on being sufficiently unwound and relaxed to launch into this again, with an objective of a late October release. This is a month later than I had originally planned for, but things don't always work out.
Posted by
mhquake
at
8:45 PM
2
comments
Thursday, September 24, 2009
Well I'll be damned...!
I just did some DirectQ! Nothing dramatic, just another clipnodes bug fixed (which in practice didn't have much effect, but it's still good to do) and added cl_maxfps for changing the server-side update framerate.
That really should be sv_maxfps, but other engines use cl_maxfps and so will I.
Posted by
mhquake
at
10:22 PM
5
comments
Wednesday, September 23, 2009
Still no updates
Nothing has been done with DirectQ since the last post.
For the past month or so I've been spending 7 hours a day (in work) debugging, restructuring, and just generally trying to bring some degree of sanity to a horrible legacy codebase. We're talking about real "brain running out of my ears by the end of the day" stuff. There is just no way that I'm going to be able to come back home and do more coding under those circumstances.
The good news however is that I have a closure date of early next week for it, following which I'm taking a coupla weeks off. Hoping to be able to get back to DirectQ and get things moving again towards the end of that time.
Posted by
mhquake
at
6:01 PM
0
comments
Monday, September 14, 2009
Real Life is hitting
I neither expected nor intended this, but right now Real Life is taking some priority. Depending on how things work out over the next while, it's now possible that 1.7.2 will not happen by the end of September. I could just release what I have at that stage, but I don't think that I'll have enough to warrant a release.
Posted by
mhquake
at
10:05 PM
0
comments
Tuesday, September 8, 2009
First changes for 1.7.2
Just 2 things for now, one of them kinda important and the other firmly in the "nice to have" category.
Firstly, I've fixed an "illegible server message" bug; this is the one I posted a while back that only occurs in demos recorded by engines that have a non-standard handling of protocol 15. I don't consider it too important, as you can still play Quake despite it, and it only affects certain demos in certain circumstances. All the same, it's good to fix something that might cause trouble and/or confusion for people.
Secondly, I've transitioned a lot of file loading from fopen/fread/fclose to the Windows API. This has no functional change (so far) and does mean a loss of portability (which - by being Direct3D - I don't have to worry about), but it does mean that I can avail of lower level optimizations in file loading, and also do things like play nice with HSM, NTFS security and so on.
Windows API file IO functions are certainly a rather odd bunch of misfits, I do like the functionality they provide but the API itself has some fairly odd quirks about it.
One thing I think I want to implement is a "run from CD" mode, allowing you to keep most of your Quake data on CD and just have the engine, save games, screenshots, and anything else you might generate on HD. It may be a simple as adding extra search paths, but there may be subtleties to it which I haven't considered yet.
Posted by
mhquake
at
5:02 PM
2
comments
Monday, September 7, 2009
Classification of Features
There are a number of different ways I classify features for DirectQ. First up is features that I know I need. These are commonly something that was present and correct in software Quake but was either missing or a hack job in GLQuake, but are also bug fixes, performance improvements or usability improvements. Something that gives cleaner code can also fall into this category (but not at the expense of correct behaviour) as clean code can generally be less buggy code. I'll finally include ripping out a lot of legacy code (like IPX support - does anyone actually use IPX these days? It's 2009 for crying out loud.)
Secondly are features that I want. These can be allowed to include changed behaviour at times, so long as it's not gameplay affecting. Quake II style noclipping, the modified TAB behaviour, cvar autocompletion, the restructured menus, and so forth fall into here. Sometimes a cool idea occurs to me that I know won't get much use but is sufficiently interesting to me to get me coding (the customizable HUD is one I'll put into here). Some of these features can occasionally disappear and reappear between versions - 64 bit lightmaps recently vanished but I think the old sRGB colour space might make a comeback (it would be a LOT cleaner now as well). Don't get too upset if your favourite one of these vanishes sometime.
Lastly are features requested by other people. I obviously can't possibly implement every single feature request I get (although it's nowhere near as rampant as it was back in the old MHQuake days), and as a secondary consideration (which is actually probably of higher importance) I have very clear design goals with the engine. There are a lot of engines out there that are aimed primarily at mappers and modders, and mappers and modders love them, which is great. I'm aiming DirectQ primarily at players, where an entirely different set of design goals apply. A player isn't going to be interested in robustly loading a partially completed map that has a f--ked (can I say "f--ked" on here?) BSP structure, for example. What a player wants is to robustly run a completed map or mod without having to fool around with most of the arcane crap that Quake normally shoves in their face (why do other engines not also do this?) Of course, if a feature request is compatible with my design goals I will give it serious consideration.
Some features can move from one list to another. Proper alpha entity sorting (which is STILL not working correctly - must fix for 1.7.2) was fairly low priority until maps that had a lot of alpha entities stacked in the Z order started appearing; who knows what will happen tomorrow?
The primary objective of having a version of Quake that can run reliably and run well on hardware that either chokes or is just plain slow with GLQuake remains, of course.
Posted by
mhquake
at
11:48 PM
2
comments
News on 1.7.2
OK, I'm pretty much committed to a 1.7.2 release right now; I have a small list of things that I want to clean up on before moving ahead to 1.8. Haven't started on anything yet, but it will probably happen by end September.
I'm not certain if a 1.7.3 is going to be necessary. The way things look right now is that everything is sufficiently rock solid to be able to press ahead without one, but - of course - if any bugs which I consider serious enough crop up, it will need to happen.
Posted by
mhquake
at
5:38 PM
0
comments
Thursday, September 3, 2009
Paying your Quake taxes
There's a phrase "paying your software taxes" which refers to coding your application to be more robust in circumstances such as running in a non-EN-US environment, on a terminal server, or various other things. Basically I interpret it as doing all the messy things you'd rather not do but have to in order to get a functionally complete application.
Paying your Quake taxes is related: this involves ensuring that your modified engine will stand the best chance of being able to successfully run all the weird and wonderful things that have been made over a > 13 year period.
The name of the game here is that sometimes you have to ditch an elegant architecture in order to properly support something, sometimes you have to forgo tighter validation, sometimes you have to put in a cheap and nasty hack, sometimes you have to deliberately and knowingly do something the wrong way, and sometimes you have to abandon a nice effect, or a significant performance boost, because all of these things can play havoc with popular mods.
There are 3 main categories I assign these to:
- Won't fix. The code change you're putting in is so important that you're prepared to accept loss of some mod compatibility. This is generally reserved for something that otherwise causes a crash bug in another more popular (or sometimes more recent) mod, or that just can't be fixed correctly. Right now I've assigned implementing a correct solution to the problem of Fullbright Colours to this one.
- Nice to fix. You have a mod that does various things, and you can get a basic code infrastructure up to send or receive the data expected by the mod, but going any further (such as actually implementing it) is low priority. For example, DirectQ can now run Nehahra; it sure doesn't actually implement everything in Nehahra, at times it's just got a function that soaks up the data or message and does absolutely nothing with it. Some day I'm going to complete it, but not right now. Right now just enough has been done to ensure - as much as I can - that running Nehahra won't crash the engine.
- Must fix. These are the ones that are generally caused by something that was legal but A BAD IDEA in - more often than not - GLQuake (rather than WinQuake) but that now cause crash bugs or serious glitches in your engine. They may also be resulting from something that was developed in a mod or engine before things were understood to the degree they are today, and which must be supported. They can also be ugly QC hacks that just happened to work because of a bug elsewhere in the ID engines. Generally if they don't crash they get relegated one or two levels down unless the end result is otherwise sufficiently serious.
It transpires that in Ye Olde Dayes of Enginye Developmente, a lot of engines sent a U_TRANS bit from the server to the client but did not bump the value of PROTOCOL_VERSION in order to notify the client that "the data I'm sending you is different to what you might normally expect". Nehahra did this, and I guess other engines followed that lead (I may have even been guilty myself back then).
Normally this is harmless enough in single player; in multiplayer so long as everyone is on the same engine it's also harmless; likewise if the server is not one of those engines it will be harmless - the bit just never gets sent so the unexpected data never arrives. I was bitten by it in a demo which was recorded on one of these engines - DirectQ did client-side validation of the protocol before it would accept the data following a U_TRANS bit. There was no absolute need for this as the server controls the protocol and we're not actually changing the size or format of any data here - it just felt like the right thing to do, but it turned out to be wrong.
I feel that it's still the Right Thing (capitals) but also the wrong thing (lowercase), which is a funny end result here, but unfortunately it had to go.
I guess the moral of this story is that the Quake engine is not a pristine lab environment, and that there's no point in having an elegant - or even Correct (capital C) - architecture even for small stuff like this, cos it ain't worth a spit if it can't handle some of what can get thrown at it in the Real World.
Posted by
mhquake
at
1:03 AM
0
comments
Wednesday, September 2, 2009
Expect 1.7.1 REAL Soon
In fact it will probably be released later on today (UPDATE - OUT NOW).
I've found an infinite texture chain case on some textures in some maps. This seems to happen if the mapper has used a liquid surf in a teleport, the teleport is an inline brush model, and it has a trigger_changelevel buried in it. End result is a total lockup situation.
This wouldn't have happened in classic ID Quake as it didn't add brush models to texture chains, and it won't happen when I do the new renderer for 1.8 as I'm dropping texture chains in linked lists entirely and doing something quite different.
Fortunately the fix was really easy (tracking down the cause took some time though), and I do consider it serious enough to warrant a bugfixed release as soon as possible.
I'll be taking 1.7 down immediately now, as a damage limitation exercise, to prevent too many people from getting it.
Posted by
mhquake
at
10:30 PM
0
comments
Release 1.7.1 Updates
That was fast! Getting 1.7 out was definitely a good thing to do, as I now have a feeling that I have time to spend experimenting with stuff that may or may not work, without fear of delaying an upcoming release.
Here's what we have so far:
- Corrected wrong version number in 1.7 update list.
- Restored default value of r_wateralpha to 1.
- Restored "view is inside the dlight" handling for gl_flashblend 1 mode.
- Definitively fixed stuff showing through water surfs on a contents transition.
- Going from a 1.6.3 config to a 1.7 config may cause some string cvars to become garbled (this is on account of the new memory architecture); it seems to mostly happen with string cvars left at their default value. I thought I had fixed it, but I may not have.
- Direct3D doesn't have a TGA writer so I had to make my own. I haven't really tested it with all possible backbuffer formats though, and there may be subtle bugs in it.
Posted by
mhquake
at
8:06 PM
2
comments
Tuesday, September 1, 2009
OK, I lied
Release 1.7 is here!
I expect that this will be quickly enough followed up by 1.7.1, 1.7.2 and so on until I get a fully stable cut of the 1.7 changes, but anything in those versions will be bugfixes. I'm intending to copy off the code and make some moves to starting on 1.8 reasonably soon though, at least to the extent of bringing on the new changes in an experimental version of the current codebase.
Posted by
mhquake
at
10:37 PM
3
comments