Clock and Network Simulator (clknetsim)
=======================================

clknetsim is a tool designed to test programs which synchronize the system
clock, either over network or from a hardware reference clock. It simulates a
system or a number of systems connected to each other in a network and the
tested programs discipline the simulated system clocks. It can be used to
quickly test how well the programs control the system clocks in various
conditions or to test the network protocols.

The tested programs are not modified in order to be included in the simulation,
but they have some system calls redirected by a clknetsim library, which is
loaded by the LD_PRELOAD feature of the dynamic linker, to a clknetsim server,
which runs the simulation and collects several statistics about each client.
The server and the clients run on a single host, they communicate via a UNIX
domain socket. The simulation runs as fast as the host system is capable of,
with two simulated systems it is usually three or four orders of magnitude
faster than real time.

Supported programs:
- chronyd and chronyc from chrony (http://chrony.tuxfamily.org/)
- ntpd, ntpdate, sntp and ntpq from ntp (http://www.ntp.org/)
- ntpd from busybox (http://www.busybox.net/)
- ptp4l, phc2sys, pmc and nsm from linuxptp (http://linuxptp.sourceforge.net/)

Limitations:
- only Linux is supported
- the fake system calls implement only a minimal functionality required to
  keep the supported clients working
- the simulated system clock advances only on select(), poll() or usleep()
  calls, this means the client sees the CPU as infinitely fast
- adjtimex() frequency and tick changes happen immediately, the kernel has
  infinite HZ
- adjtime() and PLL updates happen in one second intervals in the simulated
  time instead of the uncorrected simulated system time, all clocks are updated
  at the same time


Usage
-----

The clknetsim server is started with two required arguments, the first one is
path to a configuration file describing the network and clocks and the second
argument is the number of simulated nodes. The simulation is started when all
clients are connected.

The clients are started under a non-root user, with preloaded clknetsim.so and
the environment variable CLKNETSIM_NODE set to the number of the client.
Optionally, the environment variable CLKNETSIM_SOCKET can be set to the path of
the UNIX domain socket which is used to connect to the server, clknetsim.sock
in current directory is used by default. The CLKNETSIM_START_DATE variable can
be used to specify in seconds since 1970 when should the simulated time start,
1262304000 by default (2010-01-01 0:00 UTC). The CLKNETSIM_CONNECT_TIMEOUT
variable sets the server connection timeout, 10 seconds by default.

The simulated network is available to the clients as one or more Ethernet
networks with IPv4 addressing. All nodes have interfaces to all networks.
Their addresses are 192.168.122+s.n, where n is the number of the node
(starting at 1) and s is the number of the network (starting at 1). The
broadcast addresses are 192.168.122+s.255.

At the end of the simulation clock and network statistics are printed.
clknetsim has options which can be used to control for how long the
simulation should run, or if the frequency, offset or network log should be
written. clknetsim -h prints a complete list of available options.

A minimal example how to start a simulation:

$ LD_PRELOAD=./clknetsim.so CLKNETSIM_NODE=1 chronyd -d -f chrony.conf &
$ LD_PRELOAD=./clknetsim.so CLKNETSIM_NODE=2 ntpd -n -c ntp.conf &
$ ./clknetsim -o log.offset -l 100000 clknetsim.conf 2

clknetsim.conf:
node2_freq = (sum (* 1e-8 (normal)))
node1_delay2 = (+ 1e-1 (* 1e-3 (exponential)))
node2_delay1 = (+ 1e-1 (* 1e-3 (exponential)))

chrony.conf:
pidfile chronyd.pid
local stratum 1
allow

ntp.conf:
pidfile ntpd.pid
server 192.168.123.1

The clknetsim.bash file contains bash functions which can create the
configuration in several network settings, start the simulation, stop the
clients and process the results. The examples subdirectory contains an example
script for each supported client. The above example can be written in a bash
script as:

CLKNETSIM_PATH=.
. ./clknetsim.bash

generate_config1 2 0.0 "(sum (* 1e-8 (normal)))" "(+ 1e-1 (* 1e-3 (exponential)))"
start_client 1 chrony "local stratum 1"
start_client 2 ntp "server 192.168.123.1"
start_server 2 -o log.offset -l 100000

cat tmp/stats


Configuration file
------------------

The configuration file is a text file containing a list of assignments, each
specified on a separate line, and comments using # as delimiter. Each node has
several variables, which configure the system clock, the reference clock and
the network delays to other nodes in the network. They can be set either to an
integer value, a floating-point value or a number generating expression written
in a Lisp-style syntax.

Variables:
- nodeX_freq = float | expr
  the system clock frequency error in terms of gained seconds per second of
  simulated time, if an expression is specified, the expression is evaluated and
  frequency updated once per simulated second (or at the rate specified with
  the -R option), the allowed range is (-0.2, 0.2), the default is 0
- nodeX_delayY = expr
  the network delay for packets sent from node X to node Y in seconds, the
  expression is evaluated for each sent packet, a negative value means the
  packet will be dropped, there is no default (packets are dropped)
- nodeX_delay_correctionY = expr
  the correction written to PTP packets (as a one-step E2E transparent clock)
  sent from node X to node Y in seconds, no correction is written by default
- nodeX_offset = float
  the initial time error of the system clock in seconds, the default is 0
- nodeX_start = float
  the time in seconds when will be the node started, the default is 0
- nodeX_refclock = expr
  the reference clock time error in seconds, the clock can be accessed by the
  client via shared memory (NTP SHM protocol) or as a PTP hardware clock (PHC)
  via the clock_gettime() function, there is no default (the clock is disabled)
- nodeX_refclock_base = nodeX
  the base of the reference clock, the default is the network time
- nodeX_step = expr
  the extra time step applied once per second (or at the rate specified with
  the -R option) in seconds, there is no default (no extra steps are applied)
- nodeX_shift_pll = integer
  kernel PLL parameter, the default is 2
- nodeX_pll_clamp = 1 | 0
  kernel PLL parameter, the default is 0
- nodeX_fll_mode2 = 1 | 0
  kernel FLL parameter, the default is 0

Functions and their parameters supported in the expressions:
  (* [expr | float] ...) - multiplication
  (+ [expr | float] ...) - addition
  (% [expr | float] ...) - modulo
  (sum [expr | float] ...)
                         - summation over consecutive evaluation of parameters
  (uniform)              - random number generator with standard uniform
                           distribution
  (normal)               - random number generator with standard normal
                           distribution
  (exponential)          - random number generator with exponential distribution
                           (lambda = 1)
  (poisson lambda)       - random number generator with poisson distribution
  (file "datafile")      - number generator reading floating-point values from
                           the specified file in an inifinite loop
  (pulse high low)       - pulse wave generator
  (sine period)          - sine wave generator
  (cosine period)        - cosine wave generator
  (triangle period)      - triangle wave generator
  (equal epsilon [expr | float] ...)
                         - returns 1.0 if the values of all parameters are
                           equal within epsilon, 0.0 otherwise
  (max [expr | float] ...)
                         - returns maximum value
  (min [expr | float] ...)
                         - returns minimum value

Variables available in network delay expressions:
  time                   - current network time
  from                   - number of the sending node
  to                     - number of the receiving node
  port                   - receiving port number
  length                 - length of the packet
  subnet                 - number of the Ethernet network in which
                           the packet was sent

Variables available in delay correction expressions:
  delay                  - delay of the packet
  length                 - length of the packet (layer 4)

An example:

# node1 is an NTP server, it has an accurate and absolutely stable clock 
node1_offset = 0
node1_freq = 0

# node2 is an NTP client, it starts with 0.1s offset and has
# 0.01ppm/s frequency wander
node2_offset = 0.1
node2_freq = (sum (* 1e-8 (normal)))

# network delays between the two nodes have 10ms mean and 100us
# jitter in both directions
node1_delay2 = (+ 9.9e-3 (* 100e-6 (exponential)))
node2_delay1 = (+ 9.9e-3 (* 100e-6 (exponential)))


Author
------

Miroslav Lichvar <[email protected]>


License
-------

Copyright (C) 2010, 2011, 2012  Miroslav Lichvar <[email protected]>

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.