ore my next contract I decided to finish my «cycle of id». After Doom, Doom Iphone, Quake1, Quake2, Wolfenstein iPhone and Doom3 I decided to read the last codebase I did not review yet:

idTech3 the 3D engine that powers Quake III and Quake Live.

The engine is mostly an evolution of idTech2 but there are some interesting novelties. The key points can be summarized as follow:

An anonymous reader writes "id Software has a history of releasing the source code for their older games under the GPL. Coder Fabien Sanglard has been taking it upon himself to go through each of these releases, analyze the source code, and post a detailed write-up about it. He's now completed a review of the Quake 3 source code, diving into the details of idTech3. It's an interesting read — he says he was impressed in particular by the 'virtual machines system and the associated toolchain that altogether account for 30% of the code released. Under this perspective idTech3 is a mini operating system providing system calls to three processes.'"

Apart from the usual renderer, netcode and bot ai the author also reviews the virtual machine architecture, and because he reviewed the other open sourced quake, doom and wolfenstein code before he has a good grasp on the evolution of the idTech engine code. Quite a nice read.

The subreddit covers various game development aspects, including programming, design, writing, art, game jams, postmortems, and marketing. It serves as a hub for game creators to discuss and share their insights, experiences, and expertise in the industry.

---

Members Online

When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.

Jul 1, 2012 12:31 EDT · Hot!

id software frequently releases source code to their older titles under the GPL license. It’s a nice gesture of goodwill to the community and gives people an interesting look into how a high end game is created. While most people will just casually glance at the code, Fabien Sanglard digs into each and every release and gives insight into the architecture, graphics rendering, networking, and more. Yesterday he released his findings on the Quake 3 engine.

The analysis is an interesting look at how Quake 3 was written and how it functioned. The AI, for example, was brought in-house as opposed to using a third party solution. Unfortunately, that didn’t last as the in-house programmer did a poor job, so it was later outsourced to Jean-Paul Waveren, the creator of the initial two AIs.

Some of the more interesting tidbits that Sanglard identifies are:

The virtual machines system and the associated toolchain that altogether account for 30% of the code released. Under this perspective idTech3 is a mini operating system providing system calls to three processes. The elegant network system based on snapshots and memory introspection.

Overall it’s a very detailed and interesting read. In addition to his own notes, he gives pointers to other books and references. Anybody who is interested in game development owes it to themselves to spend some time reading over not only his Quake 3 assessment, but also the many other reviews he’s done.

In a fast paced environment any information that is not received on first transmission is not worth sending again because it will be too old anyway.

As a result the engine relies essentially on UDP/IP: There is no trace of TCP/IP anywhere since the "Reliable transmission" aspect introduced intolerable latency. The network stack has been augmented with two mutually exclusive layers:

• Encryption using preshared key.
• Compression with pre-computed huffman key.

But where the design really shine is on the server side where an elegant system minimize the size of each UDP datagram while compensating for the unreliablity of UDP: An history of snapshots generate deltas packets via memory introspection.

Architecture

The Client side of the network model is fairly simple: Client sends commands to the Server each frame and receive update for the gamestate. The Server side is as bit more complex since it has to propagate the Master gamestate to each Client while accounting for lost UDP packets. This mechanism features three key elements:

• A Master Gamestate that is the universal true state of things. Clients send theirs commands on the Netchannel. They are transformed in event_t which will modifiy the state of the game when they arrive on the Server.
• For each Client the server keeps the 32 last gamestate sent over the network in a cycling array: They are called snapshots. The array cycle with the famous binary mask trick I mentioned in Quake World Network (Some elegant things).
• The server also features a "dummy" gamestate with every single field set to zero. This is used to delta snapshots when there is no "previous state" available.

When the server decides to send an update to a client it uses each three elements in order to generate a message that is then carried over the NetChannel.

Trivia :

To keep so many gamestate for each players consumes a lot of memory: 8 MB for 4 players according to my measurements.

Snapshot systems

In order to understand the snapshop system, here is an example with the following conditions:

• The server is sending update to a Client1.
• The server is attempting to propagate the state of Client2 which has four 4 fields (3 ints position[X], position[Y], position[Z] and one int health).
• Communication are done over UDP/IP: Those messages gets lost quite often on the internet.

Server Frame 1:

The Server has received a few updates from every client. They have impacted the Master gamestate (in green). It is now time to propagate the state to Client1:

In order to generate a message the network module will ALWAYS do the following :

1. Copy the Master gamestate in the next Client history slot.
2. Compare it with an other snapshot.

This is what we can see in the next drawing:

1. Master gamestate is copied at index 0 in Client1 history: It is now called"Snapshot1".
2. Since this is the first udpate, there are no valid snapshot in Client1 history so the engine is going to use the "Dummy snapshot" where all fields are always ZEROed. This results in a FULL update since every single field is sent to the NetChannel.

The key point to understand here is that if no valid snapshots are available in the client history the engine will pick "dummy snapshot" to generate a delta message. This will result in a full udpate sent to the Client using 132 bits (each field is ): [1 A_on32bits 1 B_on32bits 1 B_on32bits 1 C_on32bits].

Server Frame 2:

Now let's move forward in time: this is the Server second frame. As we can see each client have sent commands and they have impacted the Master gamestate: Client2 has moved on the Y axis so pos[1] is now equal to E (in blue). Client1 has also sent commands but more important it has also acknowledged receiving the previous udpate so Snapshot1 has been marked as "ACK":

The process is the same:

1. Copy the Master gamestate in the next Client history slot: (index 1): This is Snapshot2
2. This time we have a valid snapshot in the client history (snapshot1). Compare those two snapshots

As result only a partial update ( pos[1] = E ) is sent over the network. This is the beauty of the design: The process is always the same.

Note :

Since each field is (1=changed, 0=not changed) the partial update above would uses 36 bits: [0 1 32bitsNewValue 0 0].

Server Frame 3:

Let's move forward one more step in order to see how the system deals with lost packets. This is now Frame 3. Clients have kept on sending commands to the server. Client2 has lost life and health is now equal to H. But Client1 has not acknowledged the last update. Maybe the Server's UDP got lost, maybe the ACK from the Client got lost but bottom line is that it cannot be used.

Regardless the process remains the same:

1. Copy the Master gamestate in the next Client history slot: (index 2): This is Snapshot3
2. Compare with the last valid acknowledged snapshot (snapshot1).

As a result the message sent it partial and contains a combination of old changes and new changes: (pos[1]=E and health=H). Note that snapshot1 could have been too old to be used. In this case the engine would have used the "dummy snapshot" again, resulting in a full update.

The beauty and elegance of the system resides in its simplicity. The same algorithm automatically:

• Generate partial or full update.
• Resend OLD information that were not received and NEW information in a single message.

Memory introspection with C

You may wonder how Quake3 is comparing snapshots with introspection...since C does not have introspection. The answer is that each field locations for a netField_t is preconstructed via an array and some clever preprocessing directives:

typedef struct {
    char    *name;
    int     offset;
    int     bits;
} netField_t; 
// using the stringizing operator to save typing...

# define NETF(x)

x,(int)&((entityState_t*)0)->x

netField_t  entityStateFields[] = 
{
{ NETF(pos.trTime), 32 },
{ NETF(pos.trBase[0]), 0 },
{ NETF(pos.trBase[1]), 0 },
...
}

The full code of this part can be found in snapshot.c's MSG_WriteDeltaEntity. Quake3 does not even know what it is comparing: It just blindly follow entityStateFields's index,offset and size...and sends the difference over the network.

Pre-fragmentation

Digging into the code we see that the NetChannel module slices messages in chunks of 1400 bytes (Netchan_Transmit), even though the maximum size of an UDP datagram is 65507 bytes. Doing so the engine avoid having its packets fragmented by routers while traveling over the internet since most network MTU are 1500 bytes. Avoiding router fragmentation is very important since:

• Upon entering the network the router must block the packet while it is fragmenting it.
• Upon leaving the network problems are even worse since every part of the datagram have to be waited on and then time-costly re-assembled.

Reliable and Unreliable messages

If the snapshot system compensate for UDP datagrams lost over the network, some messages and commands must be GUARANTEED to be delivered (when a player quits or when the Server needs the Client to load a new level).

This guarantee is abstracted by the NetChannel: I wrote about it a few years ago (wow my drawings have come a long way !!).

Is Quake 3 open source?

ioquake3 is a free and open-source software first person shooter engine based on the Quake 3: Arena and Quake 3: Team Arena source code.

What engine is Quake 3 on?

id Tech 3

Is Quake 3 Arena free?

Quake 3 Arena is Free!

How to play quake 3 arena in 2023?

Start your Quake 3 Arena server.

Open Quake 3 Arena..

Select Multiplayer and click Create..

Choose your preferred map as well as the game mode and click Next..

Configure the server's settings to your liking..

Set the Dedicated option to No. ​.

Click Fight..