John Carmack .plan entries for 1999

Ok, many of you have probably heard that I spoke at the macworld keynote on tuesday. Some information is probably going to get distorted in the spinning and retelling, so here is an info dump straight from me:

Q3test, and later the full commercial Quake3: Arena, will be simultaniously released on windows, mac, and linux platforms.

I think Apple is doing a lot of things right. A lot of what they are doing now is catch-up to wintel, but if they can keep it up for the next year, they may start making a really significant impact.

I still can't give the mac an enthusiastic recommendation for sophisticated users right now because of the operating system issues, but they are working towards correcting that with MacOS X.

The scoop on the new G3 mac hardware:

Basically, its a great system, but Apple has oversold its performance reletive to intel systems. In terms of timedemo scores, the new G3 systems should be near the head of the pack, but there will be intel systems outperforming them to some degree. The mac has not instantly become a "better" platform for games than wintel, it has just made a giant leap from the back of the pack to near the front.

I wish Apple would stop quoting "Bytemarks". I need to actually look at the contents of that benchmark and see how it can be so misleading. It is pretty funny listening to mac evangelist types try to say that an iMac is faster than a pentium II-400. Nope. Not even close.

From all of my tests and experiments, the new mac systems are basically as fast as the latest pentium II systems for general cpu and memory performance. This is plenty good, but it doesn't make the intel processors look like slugs.

Sure, an in-cache, single precision, multiply-accumulate loop could run twice as fast as a pentium II of the same clock rate, but conversly, a double precision add loop would run twice as fast on the pentium II.

Spec95 is a set of valid benchmarks in my opinion, and I doubt the PPC systems significantly (if at all) outperform the intel systems.

The IO system gets mixed marks. The 66 mhz video slot is a good step up from 33 mhz pci in previous products, but that's still half the bandwidth of AGP 2X, and it can't texture from main memory. This will have a small effect on 3D gaming, but not enough to push it out of its class.

The 64 bit pci slots are a good thing for network and storage cards, but the memory controller doesn't come close to getting peak utilization out of it. Better than normal pci, though.

The video card is almost exactly what you will be able to get on the pc side: a 16 mb rage-128. Running on a 66mhz pci bus, it's theoretical peak performance will be midway between the pci and agp models on pc systems for command traffic limited scenes. Note that current games are not actually command traffic limited, so the effect will be significantly smaller. The fill rates will be identical.

The early systems are running the card at 75 mhz, which does put it at a slight disadvantage to the TNT, but faster versions are expected later. As far as I can tell, the rage-128 is as perfect as the TNT feature-wise. The 32 mb option is a feature ATI can hold over TNT.

Firewire is cool.

Its a simple thing, but the aspect of the new G3 systems that struck me the most was the new case design. Not the flashy plastic exterior, but the functional structure of it. The side of the system just pops open, even with the power on, and lays the motherboard and cards down flat while the disks and power supply stay in the encloser. It really is a great design, and the benefits were driven home yesterday when I had to scavenge some ram out of old wintel systems yesterday - most case designs suck really bad.

---

I could gripe a bit about the story of our (lack of) involvement with Apple over the last four years or so, but I'm calling that water under the bridge now.

After all the past fiascos, I had been basically planning on ignoring Apple until MacOS X (rhapsody) shipped, which would then turn it into a platform that I was actually interested in.

Recently, Apple made a strategic corporate decision that games were a fundamental part of a consumer oriented product line (duh). To help that out, Apple began an evaluation of what it needed to do to help game developers.

My first thought was "throw out MacOS", but they are already in the process of doing that, and its just not going to be completed overnight.

Apple has decent APIs for 2D graphics, input, sound, and networking, but they didn't have a solid 3D graphics strategy.

Rave was sort of ok. Underspecified and with no growth path, but sort of ok. Pursuing a proprietary api that wasn't competetive with other offerings would have been a Very Bad Idea. They could have tried to make it better, or even invent a brand new api, but Apple doesn't have much credebility in 3D programming.

For a while, it was looking like Apple might do something stupid, like license DirectX from microsoft and be put into a guaranteed trailing edge position behind wintel.

OpenGL was an obvious direction, but there were significant issues with the licensing and implementation that would need to be resolved.

I spent a day at apple with the various engineering teams and executives, laying out all the issues.

The first meeting didn't seem like it went all that well, and there wasn't a clear direction visible for the next two months. Finally, I got the all clear signal that OpenGL was a go and that apple would be getting both the sgi codebase and the conix codebease and team (the best possible arrangement).

So, I got a mac and started developing on it. My first weekend of effort had QuakeArena limping along while held together with duct tape, but weekend number two had it properly playable, and weekend number three had it brought up to full feature compatability. I still need to do some platform specific things with odd configurations like multi monitor and addon controlers, but basically now its just a matter of compiling on the mac to bring it up to date.

This was important to me, because I felt that Quake2 had slipped a bit in portability because it had been natively developed on windows. I like the discipline of simultanious portable development.

After 150 hours or so of concentrated mac development, I learned a lot about the platform.

CodeWarrior is pretty good. I was comfortable devloping there almost immediately. I would definately say VC++ 6.0 is a more powerful overall tool, but CW has some nice little aspects that I like. I am definately looking forward to CW for linux. Unix purists may be aghast, but I have allways liked gui dev environments more than a bunch of xterms running vi and gdb.

The hardware (even the previous generation stuff) is pretty good.

The OpenGL performance is pretty good. There is a lot of work underway to bring the OpenGL performance to the head of the pack, but the existing stuff works fine for development.

The low level operating systems SUCKS SO BAD it is hard to believe.

The first order problem is lack of memory management / protection.

It took me a while to figure out that the zen of mac development is "be at peace while rebooting". I rebooted my mac system more times the first weekend than I have rebooted all the WinNT systems I have ever owned. True, it has gotten better now that I know my way around a bit more, and the codebase is fully stable, but there is just no excuse for an operating system in this day and age to act like it doesn't have access to memory protection.

The first thing that bit me was the static memory allocation for the program. Modern operating systems just figure out how much memory you need, but because the mac was originally designed for systems without memory management, significant things have to be laid out ahead of time.

Porting a win32 game to the mac will probably involve more work dealing with memory than any other aspect. Graphics, sound, and networking have reasonable analogues, but you just can't rely on being able to malloc() whatever you want on the mac.

Sure, game developers can manage their own memory, but an operating system that has proper virtual memory will let you develop a lot faster.

The lack of memory protection is the worst aspect of mac development. You can just merrily write all over other programs, the development environment, and the operating system from any application.

I remember that. From dos 3.3 in 1990.

Guard pages will help catch simple overruns, but it won't do anything for all sorts of other problems.

The second order problem is lack of preemptive multitasking.

The general responsiveness while working with multiple apps is significantly worse than windows, and you often run into completely modal dialogs that don't let you do anything else at all.

A third order problem is that a lot of the interfaces are fairly clunky.

There are still many aspects of the mac that clearly show design decisions based on a 128k 68000 based machine. Wintel has grown a lot more than the mac platform did. It may have been because the intel architecture didn't evolve gracefully and that forced the software to reevaluate itself more fully, or it may just be that microsoft pushed harder.

Carbon sanitizes the worst of the crap, but it doesn't turn it into anything particularly good looking.

MacOS X nails all these problems, but thats still a ways away.

I did figure one thing out - I was always a little curious why the early BeOS advocates were so enthusiastic. Coming from a NEXTSTEP background, BeOS looked to me like a fairly interesting little system, but nothing special. To a mac developer, it must have looked like the promised land..

On the issue of railgun firing rates -- we played with it for a while at the slower speed, but it has been put back to exactly Q2's rate of fire.

I do agree with Thresh that the way we had it initially (faster than Q2, but with the same damage) made it an overpowered weapon in the hands of highly skilled players, which is exactly what we should try to avoid.

An ideal game should give scores as close to directly proportional to the players reletive skills as possible. The better player should win in almost all cases, but the game will be more entertaining if the inferior players are not completely dominated.

Quake 1 had really bad characteristics that way -- Thresh can play extremely talented players and often prevent them from scoring a single point. We wouldn't put up with a conventional sport that commonly game scores of 20 to 1 in championship matches, and I don't think we should encourage it in our games.

Eliminating health items is probably the clearest way to prevent blowout games, but that has never been popular. Still, we should try to avoid weapon decisions that allow the hyper-skilled to pull even farther away from the rest of the crowd. They will still win, no matter what the weapons are, just not by as wide a margin.

As you have probably heard by now, the first release in going to be the mac version, probably followed by the linux version, and only then the windows version.

Some of you are busy getting all bent out of shape about this.

We want to get a lot of people playing with the test to find bugs and offer well thought out suggestions, but there are classes of bugs and suggestions that emerge in each order of magnitude of exposed testers.

If a given bug is going to show up when a thousand people have looked at it, but we had released it to a hundred thousand people, then we are going to have a lot of duplication to wade through.

The mac testers will find some obvious problems. We will fix them. The later releases will be better.

Even if we had the windows distribution ready to go right now, I would still seriously consider releasing one of the mac or linux versions first because it will make our tracking a lot easier.

The holdup on the windows side are the issues with updated driver distribution. The game itself doesn't have any holdups.

We could do a windows release for just, say, voodoo2 owners and get some of the benefit of a controlled release, but it wouldn't really work out, because everyone would figure out that it can be made to (almost) work on lots of other cards by tweaking some values. That type of feedback would not be useful, because we KNOW that there are problems with most of the current drivers. We have been working with all of the vendors for the past year to get them all straightened out.

Glsetup is going to be slick -- just run it and it will Do The Right Thing for your video configuration.

We hope it will be done soon, but there are factors out of our direct control involved.

Don't be spiteful. This is just the beginning of the testing and release process.

One conspiracy theory suggests that Apple is somehow getting us to do this.

What we have "gotten" from Apple is a few development machines. No cash payoff. No bundling deal. No marketing contract.

I am looking at this long term. I want to see OS X become a top notch platform for graphics development. I think highly of the NEXTSTEP heritage and I might move my development from NT if it turns out well. There is a lot of groundwork that needs to be laid with apple for this to happen, and my working on the mac right now is part of that. Plus a lot of complaining to various apple engineers and executives. :-)

To be clear:

At this time, there is no mac that is as fast for gaming (or just about anything, actually) as a pentium III with a top of the line 3D card. Period. I have been misquoted by some mac evangelists as saying otherwise.

The new (blue and white) G3 systems are very good systems in many ways, and make a perfectly good gaming platform. However, a high end wintel machine just has more horsepower on both the CPU and the 3D card.

A 400 mhz G3 performs about the same as a 400 mhz PII if they aren't fill rate limited, where the faster cards on the PC will give about a 25% advantage. A 500 mhz PIII with an appropriate card in 30% faster than the best mac you can buy.

The multi colored iMacs, old G3 desktops, and powerbooks can play Quake3, but the RagePro 3D acceleration defines the absolute bottom end of our supported platforms. A serious gamer will not be satisfied with it.

Voodoo cards are not currently supported by the OpenGL driver, which is very unfortunate because many serious mac gamers own voodoo cards. I hope Apple and 3dfx will be able to correct this soon, but I certainly understand Apple's prioritization -- obviously, good support for the OEM video options is of primary importance.

The voodoo performance will still lag the windows platform by some amount, but some strides have been made in that area recently, so I expect good performance,

Gaming is not a reason to buy a mac, but Apple is taking steps so that it may not be a reason to avoid a mac if you have other reasons for wanting one.

MacOS still sucks.

The upper graph (blue/yellow) slides one pixel for every rendered frame. Blue lines below the baseline mean that the frame is interpolating between two valid snapshots. Yellow lines above the baseline mean the frame is extrapolating beyond the latest valid time. The length of the line is proportional to the time.

The lower graph (green/yellow/red) slides one pixel for every received snapshot. By default, snapshots come 20 times a second, so if you are running >20 fps, the top graph will move faster, and vice versa. A red bar means the snapshot was dropped by the network. Green and yellow bars are properly received snapshots, with the height of the bar proportional to the ping. A yellow bar indicates that the previous snapshot was intentionally supressed to stay under the rate limit.

The upper graph indicates the consistancy of your connection. Ideally, you should always have blue bars of only a pixel or two in height. If you are commonly getting big triangles of yellow on the graph, your connection is inconsistant.

In a heavy firefight, it is normal for modem players to see yellow bars in the bottom graph, which should return to green when the action quiets down. If you are getting several red bars visible, you may want to look for a server that drops less packets.

There are a few tuning variables for people trying to optimize their connection:

The most important one is "rate", which is what the connection speed option in the menu sets.

We are fairly conservative with the values we set for the given modem speeds: 2500 for 28.8, 3000 for 33, and 3500 for 56k.

You may actually be connecting faster than that, and modem compression may be buying you something, so you might get a better play experience by increasing the values slightly.

If you connect at 50000 bps, try a rate of 5000, etc.

I err on the conservative side, because too low of a rate will only make the movement of other things in the world choppy, while too high of a rate can cause huge amounts of lag.

Note that the optimal rate will be somewhat lower than a rate for QW or Q2, because I now include the UDP packet header length in the bandwidth estimate.

You can ask for a different number of snapshots by changing the "snaps" variable, but there isn't a lot of benefit to that. Dedicated servers run at 40hz, so stick to divisors of that: 40, 20 (default), 10. A snaps of 40 will usually just cause you to hit your rate limit a lot faster. It may be usefull for tuning rate, if nothing else.

You can adjust the local timing point with "cg_timenudge ", which effectively adds local lag to try to make sure you interpolate instead of extrapolate. If you really want to play on a server that is dropping a ton of packets, a timenudge of 100 or so might make the game smoother.

---

One more addition to net cvars:
"cl_maxpackets" will restrict the maximum number of outgoing packets to prevent client to server rate problems. This does not limit the client framerate. This defaults to 20, which might actually be a bit low. You might try experimenting with raising this to 40.

"cl_maxfps" still exists, but it will never need to be used for networking reasons.

---

* converted cvar allocation to indexes to allow range checking
* cgame converted over to use vmCvar_t instead of cvar_t needed for interpreted cgame
* fixed server crashing string bug
* adjusted scoreboard for 8 players
* show hostname on connection screen
* fixed null model warning on startup
* more space for hostname on local servers screen
* fixed mac Open Transport memory buffer bug, this was causing most of the mac crashes
* made Info_ValueForKey() case insensitive
* sv_privateClients, sv_privatePassword. this allows you to reserve slots on a public server for password access while allowing most to be freely available
* "server is full" message on connect screen
* archive handicap in config file
* cheat protect r_nocurves
* byte order independent zip checksum
* removed cl_stereo, use glConfig.stereoEnabled

* converted sound positioning away from callback method
* increased mac memory zone by 5 megs
* new memory allocator for temporary render use during init
* converted cmodel references to handles and range checked
* converted sound references to handles and range checked
* converted file references to handles and range checked

I put together a document on optimizing OpenGL drivers for Q3 that should be helpfull to the various linux 3D teams.

http://www.quake3arena.com/news/glopt.html

---

* vmtest framework, q3asm work
* converted scene building to procedural style. allows better error checking, better performance characteristics when interpreted, and is a setup stage for SMP optimizations in the renderer if I get around to it
* protected some potential div by 0 areas in cgame

* client side predict item pickups. running over items was one of the few remaining locally perceived signs of lag
* new pont-in-patch test code
* fixed pathname errors when mac users had slashes in their paths: "B/W mac". sigh.

There is one must-fix issue and a couple smaller issues remaining before the release candidate build, then we have to do a lot of testing on it. I made a lot of significant changes in the last week, and I'm sure there are some things we still need to sort out before we inflict it on the general public.

We are aiming for sunday, but understand that that means sunday evening, not sunday morning.

If saturday night / sunday morning testing on the release candidate turns up significant problems, we will put off the release until they are fixed. That could be later sunday night, or it might not make it until monday night.

The previous release delays for win32 were issues out of our control, but this release rests squarely on me. The content and other issues are ready, but we still need to make sure all the new code is solid.

* fixed give item bug
* new first snapshot timing
* moved sun drawing outside of sky shader to fix showtris
* r_drawSun
* handle all shader tesselations in q3map with tjunc fixups
* different flatness epsilons for edge vs border grids
* reorganize sound directories
* removed footsteps on non-metalic and non-water surfaces
* fixed bug with multiple looping sounds
* client side predict teleporters, go to "hyperspace"
* precache remaining liquid sounds
* don't fire jumppad events if upward velocity

A good day of work. I just finished a long test game with all three architectures, and everything looks solid.

As far as I can tell, these are ready to go after making installers and such, but everyone else has an oportunity to find bugs while I sleep...

We won't hold up for minor gameplay issues, but if anyone turns up a repeatable crasher I will rebuild everything.

Barring problems, we should start rolling the releases out tonight.

* fixed crash case on fallback from an unsupported fullscreen
* fixed overrun with very fast system connecting to a very lagged server
* fixed bad Z_Free on sounds not found
* fixed autoswitch with sync clients
* fixed losing console field on positive histories
* fixed demo recording and playback with new net code
* handle signed bit fields in msg code
* fixed playerstate event bit loss on encoding
* fixed "bad clientnum on player entity"
* reenabled corpses sinking into ground

We had to upgrade the crashtest machine to NT sp 5, because some people were attacking it with windows crashers. Those don't count.

Crashtest #3 from [iBO]QWhAX0R was a combination of two problems:

The symptom was disconnecting all clients with an illegible server message. This turned out to be caused by the fact that I was parsing strings out of my net buffers with a MSG_ReadChar() function, and I was checking for EOF as a negative one return value. I had to change this to a MSG_ReadByte() call, because -1 was showing up in the messages, which then caused a parse error because it wasn't really the end of the message.

The actual root of that issue was code like this:

{
	char buffer[MAX_STRING_CHARS];
	...
	strncpy( buffer, input, sizeof(buffer) - 1 );
	...
}

No buffer overruns are possible, but buffer is not forced to be zero terminated if on the stack. I'm pretty sure this was a result of copy-and-paste code where buffer used to be a static with a guaranteed zero, but it made me find several other places where similar things were happening.

I had started using a Q_strncpyz() function a while ago that guarantees a trailing zero and doesn't require the -1, but it turned out that between code I had written a long time ago, and code that either Cash or Brian had added, there were still a lot of normal strncpy calls around. A lot of them were wrong, too. Either missing the -1, or missing the dedicated 0 fill in.

Crashtest #4 from Jim Paris was a variation on the first part of #3.

Only one of these attacks so far has been a server crasher, but I have been giving the bounty for anything that immediately kicks all the players. I probably won't give it for attacks that only overflow some lagged clients, but I'll evaluate as they happen.

I am off to E3 now for a bunch of PR silliness, so if crashtest goes down, it won't be back up for a while...

* fixed crashtest 4
* fixed crashtest 3
* fixed jumping-over-item pickup prediction error
* made "Couldn't load sound" a developer only warning
* fixed demo recording not including portal surface entities
* precache grenade bounce sounds

The SMP support is solid enough to play with now. The only feature that is still broken is light flares.

As a happy consequence, some of the cleanup work I did for SMP gave a couple percent speedup even when running without the separate thread.

On my development system, a dual 300 mhz intergraph realizm II, the low res timedemo scores went from 27.8 to 37.8 with "r_smp 1". This is only a 35% average speedup, but at some times (lots of dynamic lights in complex scenes) the speedup is 90%+. Gameplay is noticably smoother.

The rendering thread is almost always the blocking factor, so the faster the card and OpenGL driver, the larger the speedup will be.

This is explicitly a two thread producer / consumer, so there is no benefit to more than two processors. The app is well behaved, using sleeping syncronization so that you usually still have half a processor free for other operating system functions.

Hopefully we will be able to test with some fast consumer cards sometime soon.

---

A lot of people asked what was done differently this time vs the last time I tried (without benefit) to use SMP.

My original attempt was to make a DLL that intercepted all OpenGL calls and let a separate processor execute them. The benefit would have been that all OpenGL applications could have gone faster. The problem was that the bandwidth required to encode all the commands was enough that the processor overhead was as much as it would have taken to just do the geometry on the main processor.

It would have still been a win if the geometry side was doing lots of work, like multiple lights, user clip planes, and texgens, but for the vast majority of geometry, it didn't balance out. If someone wanted to try that using the PIII or AltiVec streaming memory operations, it could probably still work.

The current SMP code is implemented directly into the renderer, and a lot of things were moved around and double buffered to allow it to use data in place, instead of having to copy it off.

---

Some people expressed surprise that Quake3 wasn't threaded already.

Threading has been presented so often as the "high tech" "cool" way to program, that many people aren't aware of the downsides.

A multi-threaded program will always have somewhat lower throughput when running on a single CPU than a single threaded program that polls in explicit places. The cost of a context switch at the cpu level is negligible, but the damage that it can do to the cache hierarchy can add up to a noticeable amount in bad cases.

The much larger problem is that you lose tight control over when things occur. If the framerates are low enough, it isn't a huge issue, but for applications running at 30+ fps, you really don't want to trust the OS scheduler to coordinate multiple threads and have them all get in every frame. Yes, with explicit sleep() calls you can sort of get it working, but at that point, you might as well not be using threads.

A good example of not-quite-in-sync issues in the windows mouse performance. A PS/2 mouse only samples 40 times a second, so when you get an app updating at around that speed, you will get 0/1/2 scheduling variances.

They are also not terribly portable, and a pain in the ass to debug.

* enable scissor test properly
* archive r_lodBias
* cg_draw3dIcons 0 option
* data cheating protection
* userinfo renamed to clientinfo, added state and current server address
* don't forward commands to a server when playing demos
* fixed NULL extension on dir command
* added one more shotgun pellet
* added CG_Shutdown for cgame cleanup
* fixed jitter in rising smoke
* increase minimum time before reusing an entity slot
* soundinfo reports current background streaming file
* changed IPX separator to . from :, moved port processing to system independant code
* auto port scan wasn't updating the net_port cvar
* attack button presses reset inactivity timer now
* increased the forced respawn time from 10 to 20 seconds
* show smp on gfxinfo, slight reformat

Some reasonable messages have convinced me that a single immediate jump after landing may be important to gameplay. I'll experiment with it.

Strafe jumping is an exploitable bug. Just because people have practiced hard to allow themselves to take advantage of it does not justify it's existance. When I tried fixing the code so that it just didn't work, I thought it changed the normal running movement in an unfortunate way.

In the absense of powerups or level features (wind tunnels, jump pads, etc), the game characters are supposed to be badasses with big guns. Arnold Schwartzenegger and Sigourney Weaver don't get down a hallway by hopping like a bunny rabbit.

This is personal preference, but when I play online, I enjoy it more when people are running around dodging, rather than hopping.

My personal preference just counts a lot. :-)

btw, here are the current weapon effects:


gauntlet: 50 pts, 400 msec / punch
machinegun: 10 pts, 100 msec / shot
shotgun: 11 pellets of 10 each, 1000 msec / shot
rocket launcher: 100 pts direct hit, or 100 pts splash damage falling off
over 120 world units, 800 msec / shot
plasma gun: 20 pts direct hit or 15 pts splash damage over 15 units,
100 msec / shot
railgun: 100 pts, 1500 msec / shot
lightning gun: 8 pts, 33 msec / trace, max range 768 units
grenade launcher: 100 pts direct hit, or 100 pts splash over 150 units,
800 msec / shot.
bfg: 40 pts instant splash damage over 100 units, 100 msec / shot
flamethrower: to be determined, but short range / wide angle

Splash damage is calculated from the edge of the player's box, unlike
quake1, where it was calculated froom the player's origin.


* ignore cl_maxpackets on LAN
* changed cl_packetdup to 1 by default, and archived
* defaulted com_maxfps to 100 and archived, automatically disabling during timedemo. It was possible to lag out some client connections on ultra fast systems even with cl_maxpackets set fairly low due to a huge number of individual commands being created
* 250 msec minimum time between landing and jumping again. I hate having players bouncing around all the time...
* fixed bug with large r_picmip values (white shotgun sight bug)
* new lightning and rail beam drawing
* player torso twitches with pain sounds
* r_drawstrips changed to r_primitives and archived, with changes: default "0" uses glDrawElements if compiled vertex arrays are present, or strips of glArrayElement if not. "1" forces strips, "2" forces drawElements, "-1" skips drawing
* increased rocket speed to 900 from 800. decreased direct hit damage from 120 to 100. splash damage same as 1.05
* removed sound-in-use dialog, auto skip after second try
* made userinfo persistant on server across level changes
* allow different servers to respond to a challenge, allowing redirecting server proxies
* notice bad ip addresses for connection: 192.1234.123.122
* removed neck length pivot to prevent view poking into low subdivided curves. Also make aiming when looking up or down more precise.

Some timedemo numbers (a new demo, not comparable to previous scores):

Run at 640*480*16 bit color to emphasise the cpu/driver performance rather than the hardware fill rate.

                    K7-600  K7-550  PIII-500
TNT2 ultra 16 bit     73.9    68.5      53.8
Voodoo3 3000 16 bit    70.5   65.2      46.0

This is with K7 optimized drivers vs seperate PIII optimized drivers.

There is still wiggle room there in that it is possible that more effort was expended to make the AMD drivers perform better. That is perfectly valid from a consumer's point of view, but muddies the technical CPU comparison.

On identical code run on the systems, there was some more interesting data:

On my map processing tools, the K7 was faster than the PIII, but only slightly more so than the reletive clock rate increase. I would guess that this is due to larger data sets that don't fit in cache as well.

On the matrox OpenGL drivers, which have not been optimized very much and (to my knowledge) contain no PIII specific code, the K7 was a LOT faster.

The bottom line is that I feel comfortable standing behind the statement that the K7 is faster than the PIII. I will have to wait for some stuff to come out of NDA to provide a more detailed technical analysis.

Architectural cleverness is all well and good, but if AMD can't keep the clock speed up with intel, they will still fall behind. A 700 mhz PIII would probably find a lot of applications (especially integer apps) where it would outperform a 600 mhz K7.



* stabilized cg_showfps
* added append support to module file opening
* automatic logging of game scores to games.log
* fixed guantlet firing action
* force a vid_restart on WM_DISPLAYCHANGE messages
* fixed sticking on stairs on very fast framerates
* fixed sticking on stair when jumping
* fixed sticking in corner while falling physics bug
* fixed slide down steep slope physics bug
* r_showimages texture use debugging tool
* cg_freezeDemo cvar
* cg_drawSnapshot cvar
* fixed warnings after demo playback
* changed "stopdemo" to "stoprecord"
* fixed phantom windows on task bar after exit
* check for unset player animation
* fixed the snap-down-look-up bug with very high sensitivities
* reduce inflicted damage by handicap
* all pmove results as events for proper demo playback

* log all client transitions, item pickups, and kills
* changed joystick axis to act just like arrow keys so they can be bound in the controls menu for strafing. Yes, this does remove slow walking from joystick movement, but it makes everything a lot cleaner.
* fix up PantherXL trackball support
* removed bad clear command from dedicated console clear button
* tournement queuing of spectators to enter the game
* track wins and losses as long as you stay on a tourney server
* spectators are now in fly mode instead of noclip, and use teleporters
* pass serverTime instead of msec for command timing, prevents timescale cheating
* track dual eventParms on player state
* draw crosshair and name in spectator mode
* fixed rcon
* r_colorMipLevels texture visualization tool
* don't allow weapon select and cycle when in follow mode
* archive cl_yawspeed and cl_pitchspeed
* don't draw place line on scoreboard when spectating
* fixed console chatting during intermission
* better recursive error handling
* fixed curve surface flags (no impact when landing on a curve bug)

* changed RF_PORTALSURFACE to an entity type
* add sprite indexes in strip order
* changed builtin models to refentity types
* prevent respawn until resting on ground
* fixed unecessary dlight shader checking
* no splashe when lg hitting sky
* removed cvar_restart on version change, caused problems with dedicated server startup
* fixed stupid error that made skies double render
* fixed stupid error that made skies draw an uneeded bakground pass
* support add-mode multitexture for skies

case OP_NEGI:
    EmitString( "F7 1F" );      // neg dword ptr [edi]
    break;

case OP_ADD:
    EmitString( "8B 07" );     // mov eax, dword ptr [edi]
    EmitString( "01 47 FC" );  // add dword ptr [edi-4],eax
    EmitString( "83 EF 04" );  // sub edi,4
    break;

case OP_SUB:
    EmitString( "8B 07" );     // mov eax, dword ptr [edi]
    EmitString( "29 47 FC" );  // sub dword ptr [edi-4],eax
    EmitString( "83 EF 04" );  // sub edi,4
    break;

case OP_DIVI:
    EmitString( "8B 47 FC" );  // mov eax,dword ptr [edi-4]
    EmitString( "99" );        // cdq
    EmitString( "F7 3F" );     // idiv dword ptr [edi]
    EmitString( "89 47 FC" );  // mov dword ptr [edi-4],eax
    EmitString( "83 EF 04" );  // sub edi,4
    break;

06214DD0 83 C7 04             add         edi,4
06214DD3 C7 07 00 4D 0A 00    mov         dword ptr [edi],0A4D00h
06214DD9 8B 1F                mov         ebx,dword ptr [edi]
06214DDB 81 E3 FF FF 1F 00    and         ebx,1FFFFFh
06214DE1 8B 83 30 00 DC 05    mov         eax,dword ptr [ebx+5DC0030h]
06214DE7 89 07                mov         dword ptr [edi],eax
06214DE9 83 C7 04             add         edi,4
06214DEC C7 07 2C 00 00 00    mov         dword ptr [edi],2Ch
06214DF2 8B 07                mov         eax,dword ptr [edi]
06214DF4 01 47 FC             add         dword ptr [edi-4],eax
06214DF7 83 EF 04             sub         edi,4
06214DFA 8B 1F                mov         ebx,dword ptr [edi]
06214DFC 81 E3 FF FF 1F 00    and         ebx,1FFFFFh
06214E02 8B 83 30 00 DC 05    mov         eax,dword ptr [ebx+5DC0030h]
06214E08 89 07                mov         dword ptr [edi],eax
06214E0A 8B 07                mov         eax,dword ptr [edi]
06214E0C 29 47 FC             sub         dword ptr [edi-4],eax
06214E0F 83 EF 04             sub         edi,4
06214E12 83 C7 04             add         edi,4
06214E15 C7 07 40 00 00 00    mov         dword ptr [edi],40h

The current plan is that we will have another test release around the middle of next month. This version will be running game/cgame/ui with the virtual machine, and will include single player play against bots. No new maps.

I will be releasing the source for all the VM modules with it, and setting the executable up so that it will allow modified modules to be used.

The modified LCC and q3asm will be available both in source form and precompiled, so a professional development environment will not be requried. Using MSDEV to debug binary dll's does make exploration a lot easier than adding prints to interpreted code...

Some minor porting work on the tools will be necessary to do development under linux. The effort would be greater for mac development, because the tools are inherently command line based.

The map editor and tools will not be released until after the game hits store shelves. To be completely clear: you are not legally licensed to create or use addon maps with the test.

I am hoping that this public review will turn up bugs before we complete the game. 50k lines of code is quite a bit to go over, but people familiar with previous games will have a good head start in the game module.

The best possible situation would be if exploration of the code evolves into a tier system, with either moderated or limited access lists that I can follow without being swamped. I can't afford to be too involved in helping everyone figure out the basics. I have plenty of confidence in the mod communities ability to work that out. :-)

---

Some people have been mistaking memory swapping in 1.08 for network problems.

We did controlled, back to back tests against the previous versions, and the networking is identical if you have enough memory.

The addition of the new character model and all the new menu code and graphics has caused the game to begin to have some swapping problems on 64mb machines if you have all the quality options up high or are running other things.

I am looking into what I can do to reduce memory consumption for the next release, but in the meantime, you can turn down sound quality, geometry, or texture detail to get rid of it.

If you have less than 64mb, go buy more memory! The final game will have an option to run in less memory, but the graphics and sound quality will be a lot lower.

---

I made a simple change in the file management that I think is clearly a big win.

Instead of scanning for pak0.pk3, pak1.pk3, ... pak9.pk3 in game directories, I now scan the entire directory for all .pk3 files, and add them to the search path in alphabetical order.

This gives us the same needed functionality we have now -- overriding things in pak0.pk3 with updated data in pak1.pk3, but it also gives a signficant benefit to the user community.

There has been a lot of requests to have textures inside maps like Quake1 did, but I was not willing to do that. Having the files separate saves an immense amount of duplication, and keeps the internal architecture uniform.

Now, you can just add all your new art into a pk3 (zip) file along with your map and users can just drop that single file into their quake3/baseq3 directory as a single operation.

This will also be nice for custom models, which require several component parts: legs, torso, head, animations, levels of detail, and skins.

So, if you are strictly ADDING media (models, textures, maps, etc), then you can just drop the pk3 files in the normal directory.

If you are REPLACING data (code modules, menus, etc), you should make a new game directory and put the pk3 there. Starting up with "quake3 +set game mygame" will make all the mygame pk3 files override anything in baseq3. You could do it by just naming your pk3 file "zzzstuff.pk3", but then you wouldn't have a way to run the game without the addons.

A prudent person might choose to put ALL addons into a separate directory and leave baseq3 pristine for official additions.

---

Other stuff that has been done lately:

* don't clamp dedicated server or client times until much later -- prevents time resets under ordinary conditions
* fixed CG_ProcessSnapshots: cg.snap->serverTime
* only drift time on receipt of packets with a steady ping -- improve catchup after drops
* fixed players getting stuck together
* new pak file support
* derez mac resource file
* intro/loop option for music files
* fixed lerp-through-world when changing teams
* show ping and netgraph for spectators
* fixed timescale off by one problem
* tracked down player count wrong on server list
* info_spectator_start entity
* shader language change: clampTexCoord removed, added clampmap
* r_debugSort cvar for working on transparency sorting problems
* changed minimum cl_maxpackets to 15
* fixed can't-respawn-when-someone-is-on-your-body
* fixed dlighting over alpha tested surfaces
* z_stats lists all blocks >= given size
* fixed wasted model slots
* increased com_hunkmegs
* com_buildScript cvar to force loading all media and quit on errors
* fixed bad playerstate interpolation across teleporters
* converted local sounds to sfxHandle_t
* new fog code doesn't require subdivisions
* fixed sun positioning problem
* added fogging of triangle objects
* fixed devmap issue
* make g_log a filename instead of a 0/1
* g_logsync option to force a flush after each write must be set at time of log file creation

I have been working on our memory footprint for the past couple days. There are two types of paging that occur:

One time only paging that eventually settles down after you have run around the entire level a few times. This can be a result of having lots of data around that isn't actively used during the game, or was only used at startup.

True capacity misses, where the game is actually touching more memory during play than you have available. This is usually due to there just being too many textures and models.

Loading a large map with full resolution, 32 bit textures and mipmaps consumes almost 40 megs of texture space. The default of picmip 1 and 16 bit textures reduces that to six or seven (not all images observe picmip) in theory, but some OpenGL implementations keep a 32 bit version of the texture around even if the card only uses 16 bit (I consider this inapropriate), so it may still be twelve or more megs just for the images.

I have been able to save a couple megs off of the true capacity requirements by not rolling through some larger buffers when not needed (not smp and not dedicated server), and several megs more of one-time data mostly by moving a lot of static tables to dynamic allocation.

I can probably save another meg of true capacity and I think three or four megs of initialization data. The big thing that I might be forced to do is go to a skeletal animation system. I would hate to do that at this late a point in the project, but it would save about two megs per player model in the game.

In the process of chasing down the static memory hogs, I finally got around to starting something I have needed to do for years: learn perl.

I was scanning through a linker map file looking for large gaps in addresses, thinking to myself "this is one of those things you can probably do in three lines of perl code". I have many and varied excuses for why I have never gotten around to it before, mostly involving the fact that I have C parsing code that lets me get what I need done with only minimal headache when I do force myself to do some text file grovelling.

I decided my excuses weren't good anymore, and went out to the bookstore and grabbed the llama book. Many of you would have been amused seeing me go through the

print "Hello, $name!"

tutorial code as I did the examples in the first couple chapters. :-)

I got my task done, so now I just need to force myself to write little perl programs whenever a need comes up, until I get fluent with it.

* save 2.5 megs by reworking shader allocation
* save 1 meg by not double buffering backend if not smp
* convert all tr. arrays into pointers
* don't allocate as many snapshotentities when non dedicated
* new shader option: deformvertex move
* stackable deformvertex
* reduced lightning damage by 10%
* light emit from two sided surfaces
* reduced starting machinegun ammo in teamplay to 50 from 100

It looks like we are going to go to a skeletal model system. Jim Dose of Ritual had already started on an exporter from character studio, so we decided to just meet in the middle.

I implemented the loading and rendering support this weekend and tested it with a couple hand-inserted bones, so now we just need to write the glue between character studio and the new .md4 format.

The new format is bone based, but it is NOT hierarchial. Each vertex just has an arbitrary weighted list of the bones that influence it. Bones are just 4x3 matricies of floats.

A hierarchial skeleton has some advantages (angles instead of matricies, ability to do IK, etc), but this is a direct and simple replacement for our existing infrastructure that doesn't require any cached state per model instance.

A single .md4 file holds multiple level of detail surface sets, which all share the same bone frames.

In use, it is exactly like the existing models (interpolate between two frame numbers), it just saves a huge amount of space.

I used perl to generate my test data, and it was definately faster than having a separate msdev open and doing it in C.

I am trying to use parenthesis on all perl functions, but when I type "print", my fingers seem to have a flashback to applesoft basic fifteen years ago, and I wind up with bare quotes on prints and parens on everything else...

Does anyone know if there is an existing msdev syntax coloring file for perl? (no, I don't want to switch to a different editor!)


* md4 model loading and displaying
* removed clip models from cgame, use renderer models instead
* fixed mover pushing again
* fixed bug with culling of mirrors made of multiple faces
* fixed quad on spinning machinegun
* surfaceparm alphashadow - This causes q3map -light to check individual texture pixels on transparant surfaces for light shadowing instead of making the entire surface either cast or not cast shadows

I wrote this in answer to a question on the mac opengl programming list, but it is of general enough interest to programmers that I am repeating it here.

>You also mentioned display lists... Can you explain what some of the
>major things that should be rendered using a display list are? I see
>the importance of using them for characters (animation) and objects, but
>what about using them for the rest of the world, particles, and other
>things that are nifty.

This is not yet a big issue, although even pure software OpenGL's could perform some optimizations with display lists that aren't possible with vertex arrays. With hardware geometry acceleration, it can be an honest 4x improvement in throughput.

The important point is that once geometry acceleration becomes a primary target, practically everything will have to be rendered with display lists, or you will run into a nasty case of Amdahl's law.

In a busy Q3 battle, the triangle count may be split roughly evenly between character models and world geometry. Going from an empty scene to a pitched battle can result in a 50% performance drop