Very Computer

Hi all,

I'm writing (or trying to write) a N-player game that communicates over
sockets, using Xlib for graphics on a UNIX platform (probably SGI). I've
done this sort of thing before, but I'm having some difficulties designing
a real-time system.

Each player needs to see every other player in the correct position every
frame, and motion must be smooth. For N>2, I suppose it's easiest to use a
star-type network with the server in the centre which rebroadcasts
messages.

It'll have to be rigorously clocked, at 25Hz for smooth animation. This
means that the game state can't be updated until all the clients know what
each other has done that clock tick, and I'm not sure if there's going to
be too much information zipping around to get 25Hz action. My plan at the
moment is:

1. clients perform one frame's worth of action, and send info about player
position, action, changed status etc. to server
2. server rebroadcasts relevant info to all clients
3. clients update their internal game state, redraw screen etc
4. server sends start next frame message to clients

Can anybody help me decide how to handle the inter-process comms (what type
of socket to use or what socket options to set) and if it's possible to
implement the above scheme as is without running into bandwidth problems.

Also, does anybody know how Doom (to name but one) manages its multiplayer
games? I suppose Quake is a better example, as it has a client/server
architecture.

Thanks for any help,

Michael

I'd try to pack as much intelligence as possible into the client, and as
little as possible into the server.  I don't think the kind of
synchronousness you're asking for is going to be possible;  it's
generally best in a game to just have each client manage its own state
and timer locally, and update its idea of where the other players are
and what they're doing as soon as the info makes it there, having the
server basically broadcast data from each client to all the others.

Quote:>Can anybody help me decide how to handle the inter-process comms (what type
>of socket to use or what socket options to set) and if it's possible to
>implement the above scheme as is without running into bandwidth problems.

Typically, if your packets contain stateful commands like "create
object" or "change mode", you can't afford to lose a packet.  OTOH, if
the contain position coordinates for objects, with automatic creation
the first time an object is referenced, and include all possible "modes"
all the time, they'll be independent enough that you can handle losing
packets.  Or you can have two connections, one TCP and one UDP, with
things like "pause game" going over the TCP and all real-time disposable
coordinates and the like going over UDP.  I'd say that all the
synchronization issues between the two connections would tend to make
this quite a nightmare.

As for bandwith, don't count on everything being synchronous the way you
exposed it;  it might be feasible bandwith-wise on a really good
connection, but the latencies will never make it on links that are more
than a few hops away, or machines that have more than minimal load.
Clients should generate their own frames and their own time counters,
servers should just broadcast everything, and maintain global info like
"playing level 3" or "paused" or "12 clients connected".  If you really
want a master clock on the server, that'll work too (over UDP, sending
clock stuff over TCP can't work b/c TCP messages can get resent and
delayed), but you'll need local clocks on the clients too, and have the
master clock update them a-la NTP.

For the socket options ... if you use TCP, you'll most likely want to
disable the Nagle algorithm (option NO_DELAY), otherwise the TCP stack
will keep things around for as much as 1/5th of a second in order to
attempt to coalesce write()s.  Once you've disabled this, be aware that
you can saturate the network with small packets, so it's up to you to
buffer up all the stuff you want to send at one time.

ObNitPick: I've been talking about "packets" in two different senses:
true network packets (UDP, IP), and data-packets used by the game.  I
know well enough that the application doesn't see network packets with
TCP, but a game will typically want to use some higher-level packets
over TCP; call those "chunks" if you want to avoid the confusion.

--
Roger Espel Llima

http://www.eleves.ens.fr:8080/home/espel/index.html

%Hi all,

%I'm writing (or trying to write) a N-player game that communicates over
%sockets, using Xlib for graphics on a UNIX platform (probably SGI). I've
%done this sort of thing before, but I'm having some difficulties designing
%a real-time system.

%Each player needs to see every other player in the correct position every
%frame, and motion must be smooth. For N>2, I suppose it's easiest to use a
%star-type network with the server in the centre which rebroadcasts
%messages.

%It'll have to be rigorously clocked, at 25Hz for smooth animation. This
%means that the game state can't be updated until all the clients know what
%each other has done that clock tick, and I'm not sure if there's going to
%be too much information zipping around to get 25Hz action. My plan at the
%moment is:

%1. clients perform one frame's worth of action, and send info about player
%position, action, changed status etc. to server
%2. server rebroadcasts relevant info to all clients
%3. clients update their internal game state, redraw screen etc
%4. server sends start next frame message to clients

%Can anybody help me decide how to handle the inter-process comms (what type
%of socket to use or what socket options to set) and if it's possible to
%implement the above scheme as is without running into bandwidth problems.

For the bandwidth considerations, your best bet is to use IP multicasting.
Then no rebroadcasting is needed between clients and server. In fact, you
can eliminate the server, and run the game purely on a peer-to-peer basis.

The only question is how to handle lost datagrams...

Probably, if you expect the load to be fairly intense, you'll just have to
tolerate lost data, and allow clients to continue to function without complete
updates.
--
Howard Chu                              Principal Member of Technical Staff

Adverti*ts proof-read for US$100 per word. Submission of your ad to my
email address constitutes your acceptance of these terms.

Quote:>For the bandwidth considerations, your best bet is to use IP multicasting.
>Then no rebroadcasting is needed between clients and server. In fact, you
>can eliminate the server, and run the game purely on a peer-to-peer basis.

Quote:>The only question is how to handle lost datagrams...

Is this some sort of disease?  I see more and more posts of this form
where some half-baked solution is given and by the end of the post it
is made crystal clear that the solution is worthless.

<on-topic>

Just use TCP/IP.  Forget UDP/IP.

--

``Not only is UNIX dead, it's starting to smell really bad.'' -- rob

Thanks for all your responses - here's my new idea plus (of course) some
more questions...

I think the server is just going to keep track of how many clients are
running and handle rarely-occurring admin events (score tables &c) - the
clients will have to be reasonably self contained to keep network traffic
down.

UDP is risky because of dropping packets, but as Roger Espel Llima said,
TCP can be used for all the must-get-through stuff and UDP for the
nice-to-have-but-not-essential stuff. I'd imagine more UDP packets get
through than not so you'd only suffer from lack of UDP communication if
you're playing over a large # of hops, which you'd expect. And UDP
multicasting is an incredibly nice way to talk to other clients without
long select() checks on sockets.

The TCP connection would probably have to be set NO_DELAY (thanks again to
Roger - I had to look this one up in the FAQ!) with implications for the
program to be mature about what it sends out across the network.

I'm developing this code on a stand-alone SG Indigo with Irix 4 with just
the loopback networking enabled so I don't see any difference between UDP
or TCP in terms of speed or reliability (at least at ~25 small
packets/second) so...

Does anybody have any 'field' experience about how much faster/less
reliable UDP is than TCP? For example, if UDP is five times faster going
between two machines on the same subnet, I could have 4 UDP packets
followed by a TCP packet for positional updates. What sort of bandwidth can
I expect?

If you have N clients connected through TCP, then each client has an array
of N-1 sockets, one for each client. I assume you'd use a SIGIO to get
reads and then select() to see which one caused the SIGIO.

So here's my Master Plan so far;

1 UDP multicast group which sends sync packets several times per second to
keep each client's real-time clock in sync and to send time-stamped
not-required information at a high rate (say 10Hz?) Forget about UDP
retransmission - the client should be able to interpolate between UDP
packets and try and keep 'jumps' hidden from the player when a correct TCP
packet arrives. Primary consideration is speed, which'll be limited mostly
by bandwidth.

1 TCP network that connects each client to every other client (or a
star-type network with a server at the centre) for essential messages and
once-per-second sync packets. I suppose a star-type arrangement is best
because that'd use N socket pairs rather than N*(N-1)/2. TCP packets will
be the 'bedrock' - any UDP packets with a timestamp greater than the
next-expected TCP packet will be stored for later action.

Is it OK to have a TCP socket set up on a separate port on the client side
that's set to listen(...,5) at the start and is only accept(....)ed when
the client gets a new-client message from the server?

Last question - are there any conventions/suggestions for picking multicast
group ip addresses?

Cheers

Michael

Quote:>If you have N clients connected through TCP, then each client has an array
>of N-1 sockets, one for each client. I assume you'd use a SIGIO to get
>reads and then select() to see which one caused the SIGIO.

Quote:>So here's my Master Plan so far;

>1 UDP multicast group which sends sync packets several times per second to
>keep each client's real-time clock in sync and to send time-stamped
>not-required information at a high rate (say 10Hz?) Forget about UDP
>retransmission - the client should be able to interpolate between UDP
>packets and try and keep 'jumps' hidden from the player when a correct TCP
>packet arrives. Primary consideration is speed, which'll be limited mostly
>by bandwidth.

Quote:>1 TCP network that connects each client to every other client (or a
>star-type network with a server at the centre) for essential messages and
>once-per-second sync packets. I suppose a star-type arrangement is best
>because that'd use N socket pairs rather than N*(N-1)/2. TCP packets will
>be the 'bedrock' - any UDP packets with a timestamp greater than the
>next-expected TCP packet will be stored for later action.

Quote:>Is it OK to have a TCP socket set up on a separate port on the client side
>that's set to listen(...,5) at the start and is only accept(....)ed when
>the client gets a new-client message from the server?

Quote:>Last question - are there any conventions/suggestions for picking multicast
>group ip addresses?

--
Roger Espel Llima

http://www.eleves.ens.fr:8080/home/espel/index.html

 mp> If you have N clients connected through TCP, then each client has
 mp> an array of N-1 sockets, one for each client. I assume you'd use
 mp> a SIGIO to get reads and then select() to see which one caused
 mp> the SIGIO.

Avoid SIGIO - it's hard to handle correctly.

If you don't have anything else to do (other than wait for a known
time period) then block in select() waiting for data.

--
Andrew.

comp.unix.programmer FAQ: see <URL: http://www.erlenstar.demon.co.uk/unix/>
                           or <URL: http://www.whitefang.com/unix/>

Quote:><on-topic>
>Just use TCP/IP.  Forget UDP/IP.

For this particular application, it's fairly obvious that you *don't* want to
use TCP; not if you want to support lots of players at once. When the network
load starts to get too much (as it rapidly will using TCP) and packets start
getting lost, and retransmits occur, and stations start backing off, your game
leaves the realm of real-time. Events that eventually do get reported to the
other stations will arrive long after they should have occurred. For the most
part, they will arrive long after they are relevant.

Even with a star configuration, N clients talking to one server, every
bit of data that one client sends will generate N-1 duplicates of itself.
The network load increases by the square of the number of nodes playing.
Using multicasts, the load is purely linear with the number of players;
the network will take a lot longer to saturate, and in the normal case there
will be little or no packet loss anyway.

If the transmitted data is carefully packaged, occasional losses will have
negligible impact. E.g., a steady stream of deltas, with periodic full-state
packets from each node will be effectively as reliable as TCP, without the
inefficiency.

re: half-baked solutions - it's not like we're here to do detailed research and
offer complete designs in response to every query that comes along. Just
because one doesn't offer the complete answer doesn't make it the wrong answer.
I *do* know the complete answer; I've been writing IP network code for 12 years
and I've written several multiplayer games, but there's no reason for me to
spell it all out. Just as you feel it's sufficient to advise "Just use TCP,
forget UDP." But in your case, your response *is* half-baked, and won't scale
in any practical manner.
--
Howard Chu                              Principal Member of Technical Staff

Adverti*ts proof-read for US$100 per word. Submission of your ad to my
email address constitutes your acceptance of these terms.