Network messaging on WZ2100

[Cyp]

That sounds like a way of reducing bandwidth in a FPS game or a MMORPG game, rather than of reducing bandwidth in a RTS game.

Since version 2.3.8 (from before I messed with the network code) is actually implemented as a FPS game masquerading as a RTS game, I think what you are trying to do might be more achievable in 2.3.8. But probably still a lot of work. And the result might be less stable, too. Also, 2.3.8 is in plain C, not C++. Sorry that this is probably not very encouraging.

[cajadas]

That is not going to work, unfortunately. The game is not client-server, it is peer-to-peer, and each peer has a full world simulation that has to be identical down to the last digit of every relevant number. This means that each peer has to have all the information. Once one peer is missing information, its simulation of the world will differ from that of another peer, and things will go out of sync. Shots will hit on one peer, while they miss on another, and so on.

Although it presents a peer-to-peer architecture, all messages go through the host before getting to the right client, so it does resemble a client-server architecture, the difference being that every time a client sends an update, the host is responsible to send it to everyone else. So the simulation of the world may differ. I think that may not be that critical as long as the area of interest of the player presents a simulation as accurate as the host's simulation, the only one with the most accurate gamestate.

Does this make sense to you? I'm sorry but i'm felling disappointed with the fact that my work till know may have been in vain.

Anyway, can someone tell me how the network is implemented in version 2.3.8?

[cajadas]

Under Cyp's advisement, i've tried to get all messages in the queue in a std::list<NetMessage> called toBeSendQueue as following.

Code: Select all

for (uint32_t n = 0; n < num; ++n)
{
	NetMessage message = queue->getMessageForNet();
	toBeSendQueue.push_front (message);
}

NETbeginEncode(NETbroadcastQueue(), NET_SHARE_GAME_QUEUE);
	NETuint8_t(&player);
	NETuint32_t(&num);
	for (uint32_t n = 0; n < num; ++n)
	{
		queueAuto(const_cast<NetMessage &>(toBeSendQueue.back()));
      toBeSendQueue.pop_back();
		queue->popMessageForNet();
	}
NETend();

Although i haven't change anything yet on the message order i keep getting sync out messages. Does anyone know why?

[Cyp]

The "queue->popMessageForNet();" should be moved to the same loop as "queue->getMessageForNet();", otherwise it gets the same message each time.

In master, if one tank out of 1000 tanks is out of place by 1 pixel, it doesn't take very long before the game goes completely weird, with the tanks in different places on different clients, and soon, different tanks are alive on different clients, and clients even disagree on who owns which tank. See Butterfly effect. Every 1/10th of a second, a CRC hash of the entire game state is sent and compared, if there is a difference, the sync error messages are printed. If not intending all clients to have the same entire game state, the sync error check would have to be disabled.

2.3.8 is much more robust against inconsistencies (but unlike in master, inconsistencies will happen, by design).

[cajadas]

In master, if one tank out of 1000 tanks is out of place by 1 pixel, it doesn't take very long before the game goes completely weird, with the tanks in different places on different clients, and soon, different tanks are alive on different clients, and clients even disagree on who owns which tank.

I don't understand why a misplaced pixel in one tank will affect the game since the update messages don't regard information about the way units should be displayed, only the new position. Does the way a droid is drawn affect the game in a combat situation or something?

Every 1/10th of a second, a CRC hash of the entire game state is sent and compared, if there is a difference, the sync error messages are printed. If not intending all clients to have the same entire game state, the sync error check would have to be disabled.

What happens if a CRC hash checking goes wrong other than the sync error messages? Does the game have some recovery mechanism? Where should i look to disable the CRC checking system?

I am also starting to research the way the network is implemented in version 2.3.8 to see if it would be more convenient for my work.

[Cyp]

The way the droid is drawn shouldn't affect anything. I mean, the game goes weird with high probability if a droid is displaced by even a 1/128th of a tile. I didn't mean literally a pixel.

If the CRC hash checking goes wrong, it also displays a "2+2=5" icon in the game window. There are bits and pieces of game recovery left in master, but not in a very useful/functional state.

Adding "return true;" at the beginning of checkDebugSync() in lib/netplay/netplay.cpp would probably disable the check.

[cajadas]

The way the droid is drawn shouldn't affect anything. I mean, the game goes weird with high probability if a droid is displaced by even a 1/128th of a tile. I didn't mean literally a pixel.

I understood what you said, however, i still can't understand how a droid can be displaced by even a 1/128th of a tile if a position update sends the precise droid position in the map. Is It because the game does not center a droid in the position?

In another subject, i've managed to block, for example, GAME_DROIDINFO messages in NETflushGameQueues(). The thing is, i wanted the messages to be sent to the host, processed by the host, and blocked in the host to be sent to every other player. However, what's happening is that the messages don't even reach the host (are not received in NETprocessSystemMessage()).

The modified NETflushGameQueues() is implemented like this:

Code: Select all

for (uint32_t n = 0; n < num; ++n)
		{
			NetMessage message = queue->getMessageForNet();
			//debug(LOG_WZ, "Message_type_before: %s", messageTypeToString(message.type));
			
			if (message.type == GAME_DROIDINFO)
			{
				debug(LOG_WZ, "NetPlay.isHost == %d", NetPlay.isHost);
				debug(LOG_WZ, "GAME_DROIDINFO");
				queue->popMessageForNet();
			}
			else
			{
				toBeSendQueue.push_front (message);
				queue->popMessageForNet();
			}
		}
		
		num = toBeSendQueue.size();
		
		if (num <= 0)
		{
			toBeSendQueue.clear();
			continue;  // Nothing to send for this player.
		}
		
		// Decoded in NETprocessSystemMessage in netplay.c.
		NETbeginEncode(NETbroadcastQueue(), NET_SHARE_GAME_QUEUE);
			NETuint8_t(&player);
			NETuint32_t(&num);
			for (uint32_t n = 0; n < num; ++n)
			{
				NetMessage m = toBeSendQueue.back();
				toBeSendQueue.pop_back();
				queueAuto(const_cast<NetMessage &>(m));  // const_cast is safe since we are encoding, not decoding.
			}
		NETend();
		
		toBeSendQueue.clear();

By my understanding, the GAME_DROIDINFO message is sent to the host in NETflushGameQueues() but, in the host, it is redirected to the other player elsewhere. If that's the case, where does it happen? Is it in NETprocessSystemMessage()? I believe it would be the case:

Code: Select all

switch (type)
	{
		case NET_SEND_TO_PLAYER:
                ...
		if (NetPlay.isHost && sender == playerQueue.index)

But the redirected messages are NET_SHARE_GAME_QUEUE so i would have to read the messages inside. How can i do it without disturbing the game?

[Cyp]

As long as all clients do exactly the same calculations, no droid gets displaced. If there's a bug that makes some clients do something slightly different, it's a problem. When the game used some floating point, different clients would get slightly different results (due to buggy floating point support by some compilers or operating systems), so all floating point calculations had to be removed. Position updates are normally never sent, since clients normally already agree on the positions.

You can parse the message, filter it, and send a new message. I think starting from 2.3.8 (since it's implemented as a FPS game) instead of master might save a few months of work, though. Or maybe even writing a very minimal and simple game to test with.

[cajadas]

You can parse the message, filter it, and send a new message.

To parse the message, after using NETbeginDecode() on the NET_SHARE_GAME_QUEUE message, what do i need to do? another NETbeginDecode()? But to do this don't i have to get the message from some queue? Or can i decode the message other way?

I think starting from 2.3.8 (since it's implemented as a FPS game) instead of master might save a few months of work, though.

I am starting to look at 2.8.3 right now. I didn't look at it sooner because i was afraid my implementation would be affected by the recovery mechanisms this version uses in case of desync. I understand it presents a client-server architecture but it uses a different message exchange protocol. Can you explain me how it happens?

Or maybe even writing a very minimal and simple game to test with.

What do you mean?

[Cyp]

To parse the message, after using NETbeginDecode() on the NET_SHARE_GAME_QUEUE message, what do i need to do? another NETbeginDecode()? But to do this don't i have to get the message from some queue? Or can i decode the message other way?

To decode/encode an embedded message, use NETnetMessage(). If decoding, delete the message.

I am starting to look at 2.8.3 right now. I didn't look at it sooner because i was afraid my implementation would be affected by the recovery mechanisms this version uses in case of desync. I understand it presents a client-server architecture but it uses a different message exchange protocol. Can you explain me how it happens? For starters, in this implementation there are no GAME messages.

In 2.3.8, all the GAME_* messages are called NET_* instead, it doesn't have a distinction between GAME_ and NET_. Instead of trying to keep the game in synch, it sends droid positions every now and then, to keep them (very approximately) in the same place on all clients.

Or maybe even writing a very minimal and simple game to test with.

What do you mean?

I meant that it might be less work (less time debugging) to write a new (very simple) game than to work with something large (which might have bugs and weird stuff causing problems), if the aim is to show that sending part of the game state saves bandwidth compared to sending the whole game state, in some games. I don't know the exact requirements of your project, so I don't know what's appropriate.

[cajadas]

To decode/encode an embedded message, use NETnetMessage(). If decoding, delete the message.

I don't know if i'm doing it correctly but, in NETprocessSystemMessage(), i've written this:

Code: Select all

...
switch (type)
	{
		case NET_SEND_TO_PLAYER:
		{ 
...
        if (NetPlay.isHost && sender == playerQueue.index)
			{
				/// We are the host, and player is asking us to send the message to receiver.
				
				debug(LOG_WZ, "Processing message %s", messageTypeToString(message->type));
				NETnetMessage(&message);	
				NETuint8_t(&player);
				NETuint32_t(&num);
					
				debug(LOG_WZ, "PLAYER: %d", player);
				debug(LOG_WZ, "NUM: %d", num);
...

Thinking the debug messages would return the values in the message but they only return zeros.

Edit: Ok, i figured out a way to read the message inside the other message. I've written a new function in netplay/nettypes.cpp like this:

Code: Select all

void NETbeginDecodeOffQueue(NetMessage const *m)
{
	NETsetPacketDir(PACKET_DECODE);
	reader = MessageReader(const_cast<NetMessage*>(m));
}

This way, substituting the NETnetMessage() call for this one, i was able to read the correct values on the previous example, but now i don't know how to create a new message without putting it in a queue. Do i have to do something like what i've done to decode but using a MessageWriter?

On another subject, can you tell me how the GAME_GAME_TIME messages are processed and what happens if they are not correctly delivered?

[cajadas]

I've been researching how to create a NET_SHARE_GAME_QUEUE message without putting it in a queue and i've notice that, in NETflushGameQueues(), to group several messages inside a NET_SHARE_GAME_QUEUE message its called a queueAuto function. Can i use it in this case? How does it works?

Thanks

[cajadas]

I've done it!

I am now able to filter every message that comes to the host before redirecting them. But a question remains: What are the GAME_GAME_TIME messages for? Are they used to sync game time between every peer in the session or are they used to something more like stating the moment some update have to take place?

In another topic. How does a peer keep the information regarding every object of the game? Is there any struture that holds locally all in game object? Where can i find it?

Thanks.

[Cyp]

The GAME_GAME_TIME separates game ticks, effectively stating the moment the updates take place and synching game time.

The information is stored in a mess of various global structures.

[cajadas]

I've noticed that when i create a droid instead of being sent a GAME_DROID message, to create a new droid of a known type, it's sent a GAME_DROIDINFO. Is the GAME_DROID message type still used? Or are the GAME_DROIDINFO messages used also to announce the creation of new droids? Is this type of message used to send all kinds of updates related to droids?

Thank you.