# Copyright (C) 2007 Codership Oy <[email protected]>
  
DISCLAIMER:
===========
This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY, to the extent permitted by law; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

See COPYING for license details.

ABOUT:
======
glb is a simple TCP connection balancer made with scalability and
performance in mind. It was inspired by pen, but unlike pen its
functionality is limited only to balancing generic TCP connections.

Features:
 - list of backend servers is configurable in runtime.
 - supports server "draining", i.e. does not allocate new connections to
   server, but does not kill existing ones, waiting for them to end
   gracefully.
 - can use epoll API provided by Linux version 2.6 and higher for ultimate
   routing performance.
 - glb is multithreaded, so it can utilize multiple CPU cores. Also, if your
   OS does not support epoll API, consider using several threads even on a
   single clore machine as it will lessen poll() overhead proportionally and
   can improve overall performance by a factor of 2 or more.

COMMAND LINE OPTIONS:
=====================
See output of the --help option.

RUNTIME MANAGEMENT:
===================
Runtime management can be done either through FIFO file, or network socket.
By default network socket is not opened; address and port to listen at must
be explicitly specified with -c option.

To add/modify/delete backend server:
send server specification in the form <IP address>:<port>[:weight] where
weight is an integer to the daemon. Connections are distributed proportionally
to the weight. Default weight is 1. Weight of 0 means drain the server.
Negative weight means delete the server completely (all connections to that
server are closed immediately). This works both on socket connection and on
FIFO file.

To see the stats:
send "getstat" command to the daemon. This works only on socket connection
since it implies response.

Example (here glbd is listening at 127.0.0.1:4444):
$ echo "192.168.0.1:3307:5" | nc -q 1 127.0.0.1 4444
OK
$ echo "192.168.0.2:3307:5" | nc -q 1 127.0.0.1 4444
OK
$ echo "getinfo" | nc -q 1 127.0.0.1 4444
Router:
----------------------------------------------------
   Address       : weight   usage  conns
192.168.0.1:3307 :  5.000   0.000    0
192.168.0.2:3307 :  5.000   0.000    0
----------------------------------------------------
Destinations: 2, total connections: 0 

'usage' here is some dimensionless metric of how much destination is staffed
with connections (relative to weight). Ranges from 0 (totally unused) to 1.0
(very busy). Router tries to keep 'usage' equal on all destinations.


ADDRESS CONVENTIONS:
====================
All network addresses are specified in the form IP|HOSTNAME:PORT:WEIGHT.
Depending on the context some parts can be optional, in that case they can be
omitted. For example address to listen for client connections can be specified
either as HOSTNAME:PORT or just PORT. In the latter case glbd will listen for
client connections on all interfaces. Backend servers can be specified as
HOSTNAME1,HOSTNAME2,HOSTNAME3. In that case incoming port number will be used
for PORT value and 1 will be used for WEIGHT value.


PERFORMANCE STATISTICS:
=======================
GLB allows to query raw performance statistics through control socket using
'getstat' command. The client can use these data to obtain useful information,
e.g. average number of reads per poll() call.

$ echo "getstat" | nc -q 1 127.0.0.1 4444
in: 6930 out: 102728 recv: 109658 / 45 send: 109658 / 45 conns: 0 / 4 poll: 45 / 0 / 45 elapsed: 1.03428

Statistics line consists of fields separated by spaces for ease of parsing in
scripts. A few description fields are added to assist in human reading. Value
fields are all even and go as follows:

  2  - number of bytes received on incoming interface (client requests)
  4  - number of bytes sent from incoming inteface (server responses)
  6  - number of bytes passed through recv() call
  8  - number of recv() calls
  10 - number of bytes passed through send() call (should be equal to p.6)
  12 - number of send() calls
  14 - number of created connections
  16 - number of concurrent connections
  18 - number of read-ready file descriptors returned by poll()/epoll_wait()
  20 - number of write-ready file descriptors returned by poll()/epoll_wait()
  22 - number of times poll()/epoll_wait() triggered
  24 - time elapsed since last statistics report (seconds)
  
All values except for 16 and 24 are totals accumulated since the last report.
In order to obtain some variable rate it must be divided by the elapsed time.
On 32-bit architectures the values are stored in 4-byte integers and can
overflow if too much time elapsed, so the first statistics report in the series
may need to be discarded.