miniseed2dmc [options] host:port files|directories ...
miniseed2dmc transfers selected miniSEED data records from the local computer to a remote Data Management Center. Prior to using this program coordination with DMC is required and the user must have been assigned a host and port indicating where the data should be sent.
miniSEED to be transferred can be identified 3 different ways:
1 - files on the command line 2 - directories on the command line which will be searched 3 - files containing lists of files and/or directories
If directories are specified on the command line or in list files all files they contain are assumed to be input files and all sub-directories will be searched (the level of recursion can be limited with the -r option).
List files are identified by prefixing the file name with '@' on the command line or by using the -l option.
A state file is maintained by miniseed2dmc to track the progress of data transfer. This tracking means that the client can be shut down and then resume the transfer when the client is restarted. More importantly it allows the client to determine when all records from a given data set have been transferred preventing them from being transferred again erroneously. By default the state file is written to a file named, creatively, 'statefile' in the working directory (see the -w option). The default state file location may be overridden using the -S option.
To track the miniSEED data transferred miniseed2dmc writes SYNC files representing the data coverage. The SYNC files are written to the working directory (see the -w option). The SYNC file names contain the connection start and end times as a record of when the data was sent. A separate SYNC file listing is written for each time the program is executed.
-V
Print the program version and exit.
-h
Print the program help/usage and exit.
-v
Be more verbose. This flag can be used multiple times ("-v -v" or "-vv") for more verbosity.
-p
Pretend, process input files as usual and write the state file and SYNC file but do not connect or send data to the submission server. Useful for client side testing, the transfer statistics will not be accurate. After running the program in this mode the state file (and probably the SYNC file) created must be removed prior to actually transferring the data.
-r level
Specify the maximum number of directories to recurse into, default is no limits.
-fn
Embed a relative path and filename in the data stream packets sent to the server, this information can be used by the remote server to re-create the local file names and organization.
-E
Exit the program on connection errors. By default miniseed2dmc will continuously try to connect to the specified server until all data has been sent.
-q
Be quiet, do not print the default diagnostic messages or transmission summary.
-NS
Do not write a SYNC file after sending data, by default a SYNC file representing the data set sent is written for each session. All SYNC files are written to the "work directory" (the -w option).
-ACK
Request and require acknowledgements from the server for each miniSEED record sent, this guarantees that each record sent was written to the filesystem by the remote server. This should not be necessary since TCP performs this function for the network layer, leaving only a very small potential that a server crash will lose data sent by the client. It will also significantly slow down the transfer rate.
-mr maxrate
Specify a maximum transmission rate in bits/second. The suffixes K, M and G are recognized for the maxrate value, e.g. '100M' is understood to be 100 megabits/second. The transmission rate is not limited by default.
-I
Print the transfer rate at a specified interval (the -It option) during transmission.
-It interval
Specify interval in seconds at which to print transfer rate statistics during transmission, default is 30 seconds. This option implies the -I option.
-w workdir
The working directory is where the (default) state file and SYNC listing will be written, default is the current working directory.
-S statefile
A state file is written to track the status of the transmission. It is recommended to use a unique state file for each separate data set. By default a file named "statefile" will be written to the workdir. If the specified value is not an absolute path it is relative to the current working directory (not workdir).
-l listfile
The listfile is a file containing a list of files and/or directories containing miniSEED to be sent. This is an alternative to prefixing an input file with the '@' which identifies it as a list file.
-s selectfile
Limit processing to miniSEED records that match a selection in the specified file. The selection file contains parameters to match the network, station, location, channel, quality and time range for input records. As a special case, specifying "-" will result in selection lines being read from stdin. For more details see the SELECTION FILE section below.
host:port
The required host and port arguments specify the server where the miniSEED records should be sent.
A selection file is used to match input data records based on network, station, location and channel information. Optionally a quality and time range may also be specified for more refined selection. The non-time fields may use the '*' wildcard to match multiple characters and the '?' wildcard to match single characters. Character sets may also be used, for example '[ENZ]' will match either E, N or Z. After a '#' character is read the remaining portion of the line will be ignored.
Example selection file entires (the first four fields are required)
#net sta loc chan qual start end IU ANMO * BH? II * * * Q IU COLA 00 LH[ENZ] R IU COLA 00 LHZ * 2008,100,10,00,00 2008,100,10,30,00
For the below examples the host and port are specified as host:port, in real usage these must be a real host name and port.
Example 1
The most simple example is sending a couple of files containing miniSEED:
> miniseed2dmc host:port 080101.mseed 080102.mseed
In this case the state file and SYNC files will be written to the current directory.
Example 2
If there is a directory called 'data' that contains only files of miniSEED they can all be sent using:
> miniseed2dmc host:port data
The program automatically distinguishes between files and directories.
Example 3
A list file named 'mseeds.txt' contains:
---- mseeds.txt /data/080101.mseed /data/080102.mseed ----
This file can be used with miniseed2dmc using the follow sytax (both are equivalent):
> miniseed2dmc host:port @mseeds.txt > miniseed2dmc host:port -l mseeds.txt
Example 4
A recommended organization of data is to have a directory for each data set and keep all the miniSEED files in a sub-directory. For example, a monthly data set can be kept in /archive/Jan2008/ with all the January 2008 miniSEED files kept in /archive/Jan2008/data/. miniseed2dmc could then we executed in the following manner:
> miniseed2dmc host:port -w /archive/Jan2008/ /archive/Jan2008/data/
This will result in all the miniSEED data in /archive/Jan2008/data/ being transferred and the related state and SYNC file(s) being written to /archive/Jan2008/.
This program is intended to transfer static data sets, it is not designed for transfer of real-time streaming data.
Chad Trabant EarthScope Data Services
(man page 2017/01/17)