-
Notifications
You must be signed in to change notification settings - Fork 10
"linets" and "tscat" Commands
Japanese document is here.（日本語版はこちら）
Imagine you have an IoT device that sends ordering text messages intermittently via a special file like /dev/tty. Or imagine the ordering messages come from the Internet, and you receive them with like cURL, netcat, MQTT, and so forth. And now you have to develop a program which operates some system by the messages. To debug the program, don't you hope that the messages always come with the completely same timing?
Today, I introduce two useful commands for that scene. One of them is a standard I/O timing recorder "linets." And the other is a timing reproducer "tscat."
"linets" -- Add Each Line Timestamp
This command receives text data from files or the standard input (stdin), and immediately sends them to the standard output (stdout). However, it inserts a timestamp into the head of each line before sending it. The timestamp each means the time when the line came to the command.
You can add timestamps to the incoming text data, like the following.
$ cat /dev/ttyS1 | linets
20200318200358 Go
20200318200301 Stop
20200318200301 Turn left
20200318200302 Go
:
:The default format of the timestamp is the calendar-time "YYYYMMDDhhmmss." in the timezone in which the host are. You can change it with the TZ environmental variable.
You can choose another one with an option. "-e" is to change the format to the UNIX-time. And "-Z" is to change it to the number of seconds from the command start. Moreover, "-3," "-6," and "-9" options is to make the timestamp more precise. In the case of "-3," three decimal digits ".nnn," which means a millisecond, follow the timestamp. Here is an example.
$ cat /dev/ttyS1 | linets -Z3
0 Go
3.501 Stop
3.999 Turn left
4.499 Go
:
:See the command help or the source file on GitHub for more detail.
"tscat" -- Timestamp Conscious cat Command
This command receives the timestamp-labeled text data and sends them by each line to the standard output (stdout) with waiting until the time the timestamp says and cutting the timestamp. Here is the simplest example.
$ date
Wed Mar 18 20:30:00 JST 2020
(Suppose that you executed the following command in 0 seconds)
$ echo '20200318203005.5 foo' | tscat
foo <== This line is displayed after 5.5 seconds.
$ The default format of the timestamp this command assumes is the calendar-time "YYYYMMDDhhmmss." in the timezone in which the host are. You can change it with the TZ environmental variable.
You can use a timestamp having a decimal point up to the nanosecond unit. And you can also use a timestamp in UNIX-time if you use the command with the "-e" option, like this.
$ date
Wed Mar 18 20:30:00 JST 2020
(Suppose that you executed the following command in 0 second)
$ echo '1584531005.5 foo' | tscat -e <== "1584531005" means "Wed Mar 18 20:30:05 JST 2020"
foo <== This line is displayed after 5.5 seconds.
$ And you should know about the "-Z" option. If you use it, this command sends the first line immediately, whatever the timestamp means. Then, the next and the following lines go out after waiting for the time from the first line's timestamp to its one, respectively. See the following example.
$ date
Wed Mar 18 20:30:00 JST 2020
$ cat <<__TEXTDATA__ | tscat -Z
> 20390101102030 1st <== These timestamps obviously mean the future.
> 20390101102031 2nd
> 20390101102040 3rd
__TEXTDATA__
1st <== displayed immediately
2nd <== displayed one second later
3rd <== displayed another nine seconds later
$ This option is useful for reproducing the timing anytime you want.
See the command help or the source file on GitHub for more detail.
In this article, I'll show you two techniques.
Now, imagine a robot in which you can control it with text messages and a command "robot_controller." You can receive the ordering message from a TTY special file "/dev/ttyS1." And you can control the robot by sending the message to the controller command as follows.
$ cat /dev/ttyS1 | <== The source of ordering messages
> robot_controller <== Some robot driving commandAt first, insert three commands between "cat" and "robot_controller." Then you can record the timing when each message has sent into the file "data_with_timing.txt." Note that "ptw," which is before the "sed", is the command to avoid the full-buffering mode. See its article for more detail.
$ cat /dev/ttyS1 |
> linets -3 | <== Label the timestamp up to millisecond unit
> tee data_with_timing.txt | <== Save the timestamp-labelled data
> ptw sed 's/^[^ ]* //' | <== Cut the timestamp field
> robot_controllerBut "robot_controller" can still work with no problems because sed command which follows "linets" removes the timestamp field.
After recording it, you can make the robot dance the completely same by passing the timing file through tscat.
$ cat data_with_timing.txt |
> tscat -Z | <== Send messages at exactly the same time
> robot_controllerNot only can you reproduce the timing accurately, but also you can change the reproducing speed.
At first, record the timing as same as the previous example. But the option for linets is slightly different, Add the "-z" option as well to record the number of seconds which is from the command starting instead of the calendar-time.
$ cat /dev/ttyS1 |
> linets -z3 | <== Label the number of secs. from command booting
> tee data_with_timing.txt |
> ptw sed 's/^[^ ]* //' |
> robot_controllerWhen you reproduce the dance, you can make the robot dance at half speed with the following AWK command. That is because the awk doubles the number of seconds.
$ cat data_with_timing.txt |
> ptw awk '{print $1*2,substr($0,strlen($1)+2)}' | <== Double each first field
> tscat -Z |
> robot_controllerBy increasing the number, you can observe the motion of the robot more carefully.
linets can label the timestamp up to the nanosecond unit. But the accuracy depends on your computer's performance. In the 2020s, almost no computer can measure the exact time to reach that point.
And tscat can also read the timestamp up to the nanosecond unit. But the accuracy depends on several factors, such as computer performance, scheduling algorithm of the kernel, the number of total processes on the OS, process priority the process has. A UNIX system call "nanosleep()," which the command uses for the reproducing, almost always oversleeps. That is because most UNIX OSes are designed as multitask OS, and the running turn of a process doesn't always come soon after the sleeping time. It heavily depends on its process priority.