Discusion: Network Protocol (masterserver)

Discuss the future of Warzone 2100 with us.
Kamaze
Regular
Regular
Posts: 1017
Joined: 30 Jul 2006, 15:23

Discusion: Network Protocol (masterserver)

Post by Kamaze » 20 Dec 2006, 00:03

--> http://wz2100.net/wiki/user:masterserver_protocol

I'm not really a fan of XML in the network,
but wouldn't it be better and easier to make the Client->Masterserver and Masterserver->Client communication XML based?

I know that it would increase the traffic usage heavily, so maybe compress the transfered XML via zlib then.
Something simple like:

Code: Select all

<OpenGames count="5">
	<game ip="255.255.255.0" port="1337" players="5" map="SK-Rush" version="2.1" name="My Game1" /> 
	<game ip="255.255.255.1" port="1337" players="5" map="SK-Rush" version="2.1" name="My Game2" /> 
	...
</openGames>
I think that would be solve the "unknown size" issue and make is much easier to extend it later.
(like free slots, current player names etc...)

Second: It would be easier to create/maintain a master server app too :)
Last edited by Kamaze on 21 Dec 2006, 23:33, edited 1 time in total.
We all have the same heaven, but not the same horizon.

karmazilla
Trained
Trained
Posts: 84
Joined: 26 Aug 2006, 21:05

Re: Discusion: Network Protocol

Post by karmazilla » 20 Dec 2006, 00:59

Interesting proposal.

The format isn't so much of a problem on the server side, as long at we don't use any overly exotic format (like writing the data in word-art embedded in an Excel file - yes, I have seen it happen and I am still traumatized).

Really, the problem is how easy it'll be for Dev/Per/Whoever to write the client side of this.

User avatar
kage
Regular
Regular
Posts: 751
Joined: 05 Dec 2006, 21:45

Re: Discusion: Network Protocol

Post by kage » 20 Dec 2006, 05:27

as long as it isn't used for transmission of ingame data, it really wouldn't be all that bad -- in fact, we could do something like extend the jabber protocol to provide this in addition to pre-game chat services.

User avatar
Terminator
Regular
Regular
Posts: 1070
Joined: 05 Aug 2006, 13:46
Location: Ukraine
Contact:

Re: Discusion: Network Protocol

Post by Terminator » 20 Dec 2006, 19:32

As for me netcode is a high preority task for WRP. Because  MP (multiplayer) is like a "face" of the game, which must be at list good.
Death is the only way out... sh*t Happens !

Russian-speaking Social network Group http://vk.com/warzone2100

Kamaze
Regular
Regular
Posts: 1017
Joined: 30 Jul 2006, 15:23

Re: Discusion: Network Protocol

Post by Kamaze » 20 Dec 2006, 21:37

Third: With the TinyXML lib, its really easy to create a XML object. So no fancy parsing/writing.
We all have the same heaven, but not the same horizon.

karmazilla
Trained
Trained
Posts: 84
Joined: 26 Aug 2006, 21:05

Re: Discusion: Network Protocol

Post by karmazilla » 20 Dec 2006, 22:59

From a discussion on IRC, this was suggested as a solution:

We stream XML over TCP, but start the transmition with a 32 bit integer to specify the payload size. Something like this:
[Int32][XML payload]
Very simple... and the XML messages will then contain both data and commands.

The details of the XML messages isn't specified any further, but Kamaze's initial post is a good example, IMO.

Plus, Kamaza said that this would be easy to implement, so I guess he's on it? ;)

User avatar
kage
Regular
Regular
Posts: 751
Joined: 05 Dec 2006, 21:45

Re: Discusion: Network Protocol

Post by kage » 21 Dec 2006, 04:58

karmazilla wrote: We stream XML over TCP, but start the transmition with a 32 bit integer to specify the payload size. Something like this:
[Int32][XML payload]
Very simple... and the XML messages will then contain both data and commands.
are you referring to the ingame netcode?

if so, and if kamaze or anyone else can manage to make wz playable that way (on the same comps and connections it currently can be played with) without resorting to sscanf or something similar, then i'd buy that person a few beers, or any other reasonably affordable drink of their choice.

Kamaze
Regular
Regular
Posts: 1017
Joined: 30 Jul 2006, 15:23

Re: Discusion: Network Protocol

Post by Kamaze » 21 Dec 2006, 10:32

kage wrote: are you referring to the ingame netcode?
No, just the master server communication protocol.
We all have the same heaven, but not the same horizon.

User avatar
kage
Regular
Regular
Posts: 751
Joined: 05 Dec 2006, 21:45

Re: Discusion: Network Protocol

Post by kage » 21 Dec 2006, 14:35

Kamaze wrote: No, just the master server communication protocol.
oh, okay. i was both intrigued and scared at the same time at the thought of it being in the main game code, but something like master-server stuff will benefit greatly from a switch to xml.

karmazilla
Trained
Trained
Posts: 84
Joined: 26 Aug 2006, 21:05

Re: Discusion: Network Protocol

Post by karmazilla » 21 Dec 2006, 15:26

Formalized to proposal 3:
http://wz2100.net/wiki/user:masterserve ... proposal_3
This matches what you had in mind, Kamaze?

Kamaze
Regular
Regular
Posts: 1017
Joined: 30 Jul 2006, 15:23

Re: Discusion: Network Protocol

Post by Kamaze » 21 Dec 2006, 20:19

karmazilla wrote: Formalized to proposal 3:
http://wz2100.net/wiki/user:masterserve ... proposal_3
This matches what you had in mind, Kamaze?
Right, thanks.
But we should not forget, to discuss about the XML formatting.
We all have the same heaven, but not the same horizon.

Kamaze
Regular
Regular
Posts: 1017
Joined: 30 Jul 2006, 15:23

Re: Discusion: Network Protocol

Post by Kamaze » 21 Dec 2006, 23:21

Here a small theoretical prototype for adding / deleting games:
---------------

Client -> Masterserver
- Open a new game.

The client sends the deflated XML with all necessary data (extensible) to the master server.
(I tested compressing this small XML's. It only saves ~1 or 2 bytes due it own tininess.

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<ClientMessage>
	<OpenGame 
		port="9999"
		map="Sk-Rush"
		maxplayer="4"
		techlevel="1"
		version="2.1"
		name="My Game Name"
	/>
</ClientMessage>
(177 Byte)

Masterserver -> Client
- Tell the client all goes find and give him a game ID.

The master server sends an ack and gives the client a unique ID for his game.
(For the case, that multiple games are running from the same IP)

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<ServerMessage>
	<GameAdded
		gameid="sv79dkn2t9r0"
	/>	
</ServerMessage>
(112 Byte)

Client -> Masterserver (Case: 1)
- Change game state to running.

Clients are joined now and the client logs out of the masterserver.
(Are running games interesting? And no 'Ping-Pong' event for running games)

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<ClientMessage>
	<ChangeGameState 
		gameid="sv79dkn2t9r0"
		state="running"
	/>
</ClientMessage>
(134 Byte)

Client -> Masterserver (Case: 2)
- Cancel the announced game.

Client logs out of the masterserver.
(He decided to cancel it)

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<ClientMessage>
	<ChangeGameState
		gameid="sv79dkn2t9r0"
		state="deleted"
	/>
</ClientMessage>
(134 Byte)

---------------

Values:
port int_16
map char[64]
maxplayer int_8
techlevel int_8
version char[16]
name char[128] (limit it in game or via the server -just cut it if too long) to 60 or 80 chars)
gameid char[16] (12 chars length)
state char[16]
(224 Byte)

And clients should send a:

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<ClientMessage>
	<KeepAlive
		gameid="sv79dkn2t9r0"
	/>
</ClientMessage>
(112 Byte)

each 30 or 60 seconds.
So that games with 90 secs after the last 'KeepAlive' can get deleted by the server.

Yes, this is much more text (thought xml) then really needed, but its very extensible.
To add Mod-Dependencies / Download URL's / MOTD / Password or ... later.
Last edited by Kamaze on 21 Dec 2006, 23:53, edited 1 time in total.
We all have the same heaven, but not the same horizon.

User avatar
kage
Regular
Regular
Posts: 751
Joined: 05 Dec 2006, 21:45

Re: Discusion: Network Protocol

Post by kage » 22 Dec 2006, 01:04

Kamaze wrote: The client sends the deflated XML with all necessary data (extensible) to the master server.
(I tested compressing this small XML's. It only saves ~1 or 2 bytes due it own tininess.
yeah. saving a few bytes isn't worth the processor usage (however small) that'd involved.

and in case it's not blatantly obvious, whitespace is good for showing the spec, but you'd want to generate xml messages without any formatting whatsover, as that'd save those 17 bytes in the below message compared to when it has the extra line breaks and tabs.

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<ClientMessage>
	<OpenGame 
		port="9999"
		map="Sk-Rush"
		maxplayer="4"
		techlevel="1"
		version="2.1"
		name="My Game Name"
	/>
</ClientMessage>
it might be worth allowing for an optional ip attribute -- there might be reasons that a user would want to use an intermediate "master" server, or other instances where the ip address of the actual game would be different than the ip address of that the "OpenGame" message came from, like if i had one connection for high-priority traffic (such as the actual game) while i had another for low priority stuff (such as http and master server requests) on the same machine, and if the ip address is not specified, it'd be assumed that the game would be hosted on the same ip as the one used to send the "OpenGame" request.

additionally, a map "hash" might be useful in addition to the map name, so that it'd allow the client to determine if they have the right version of a map -- would be wise to accompany this with client-side map hashing so that multiple versions of a map may be stored and used.
port int_16
map char[64]
maxplayer int_8
techlevel int_8
version char[16]
name char[128] (limit it in game or via the server -just cut it if too long) to 60 or 80 chars)
gameid char[16] (12 chars length)
state char[16]
the reason multiplayer game servers often have really long names is generally because there is no adequate system for filtering and tagging different types of servers -- even quake3-based games, which generally have very above-par server filtering options don't give the server admins enough flexibility, since there's almost never any way to specify house server rules before you enter a game. for that, we might add an optional child element to OpenGame, named something like "ServerRulesText" that holds plaintext, which allows the server admins to specify house rules that aren't enforcable by the engine, but can still be viewed before joining a game. Also, showing every single settable game option to be viewable from the player's client will greatly lessen the need for any server admins to have overly verbose game names like "~~MEH~~ clan *house of pain* - Sk-Rush HUMANS-VS-BOTS NO RUSHING jimmy is admin, CHEATERS GET ONE WARNING" -- in that example, if you also add clan information to be available as a seperate attribute or element, then you can allow the client to display or hide such information, and perhaps allowing clients to add a color based on filters would allow them to display all small map servers in red, or all techlevel=2 servers in blue, or both of those in red at the same time (user customizable). ideally, we'd want to get the actual server name down to just "*house of pain*", where everything else is a collapsable column or is displayed in the extended server info when it is selected but before it is joined.
Last edited by kage on 22 Dec 2006, 01:09, edited 1 time in total.

Chojun
Regular
Regular
Posts: 518
Joined: 25 Nov 2006, 17:49
Contact:

Re: Discusion: Network Protocol (masterserver)

Post by Chojun » 22 Dec 2006, 02:53

Last year I wrote part of a game server that used XML messages to communicate.  None of the systems that use XML that I've seen are very efficient (at all).  Great consideration must be given into what kind of a parser will be used, or if you will roll your own (strongly discouraged).  MSXML is pretty good and fairly easy to use, but the parsing provides quite a bit of overhead that will be prohibitive, especially in a network game where many, many packets are sent every second.  ZLib compression is also not the best idea since it adds compression/decompression overhead to the works.

If XML is what you want, then I'd consider using the FIPA protocol, which is what we used to implement our matchup server.  Here is the doc that you should look at:

http://www.fipa.org/specs/fipa00085/SC00085J.html

This doc implements the XML Envelope for the message that is to be sent.  There are other docs available if you would like to conform to this specification.  My matchup server basically did exactly what you are proposing here in this thread for a master server.  It is easy to implement but again, XML is not a very efficient way to handle things.  Think of all that overhead just for the data tags!  :o  FIPA gives specifications and examples for a bit efficient message protocol that you should give strong consideration to...

I have a document that outlines the message structure for the various messages that is sent to the various components (agents) that make up the whole game.  I can post it if there are any interested.
The best thing to do when your philosophies don't stand up to debate is to lock the thread and claim victory.

Kamaze
Regular
Regular
Posts: 1017
Joined: 30 Jul 2006, 15:23

Re: Discusion: Network Protocol (masterserver)

Post by Kamaze » 22 Dec 2006, 09:40

kage wrote: and in case it's not blatantly obvious, whitespace is good for showing the spec, but you'd want to generate xml messages without any formatting whatsover, as that'd save those 17 bytes in the below message compared to when it has the extra line breaks and tabs.
I'd stripped the whitespaces andnew-lines in the reference XML files ;) (not the indent tab)
additionally, a map "hash" might be useful in addition to the map name, so that it'd allow the client to determine if they have the right version of a map -- would be wise to accompany this with client-side map hashing so that multiple versions of a map may be stored and used.
Sure i forgot that.

@Chojun, this is just a suggestion for the master server. Not the general net-code.
And after the test with the compression, i think only the gamelist will be compressed. (there it makes sense)
We all have the same heaven, but not the same horizon.

Post Reply