Skip to content

das3_csv

Jump to bottom
C. Piker edited this page Jul 26, 2024 · 1 revision

das3_csv - Transforms das v2 and v3 streams to "Comma" Separated Values

Name

das3_csv - Transform das streams to a delimited text format

Synopsis

das3_csv [options] < INFILE

Description

das3_csv is a filter. It reads a das2 or das3 stream on standard input and writes a delimited text stream suitable for use in common spreadsheet programs to standard output.

To preserve some of the structure of a das stream while providing compatible output, each row starts with the dataset ID. Dataset IDs are positive integers and all rows for the same dataset are tagged with the same value. ID zero is a special value which indicates global stream information. For data streams which only output a single dataset, these IDs may be safely ignored and thus -i may be used to disable them.

The second column (or first if using -i) has one of the following strings:

      "header"   - The row contains dataset header information
      "values"   - The row contains data values
      "property" - The row contians an object property

In general, streams may contain any number of datasets, thus the output may contain any number of header rows. Header rows can be disabled via the -n option below. Note that das streams push new object definitions onto the stream as they are encountered so for multi-dataset streams, new headers may be encountered after data values start.

Within a dataset, the number of header columns and the number of data columns are always the same. "Property" rows do not attempt to match the number of columns as the surounding datasets, however object properties are not emitted by default. Thus for single-dataset streams, with default options, the output of das3_csv is RFC-4180 compliant.

Defaults

  • All object properties are dropped, except for 'label' if available.

  • The field delimiter character is a ';' (semicolon).

  • Input UTF-8 values are output as-is, without conversions

  • 32-bit floating point values are written with 6 significant digits in the mantissa and 2 digits in the exponent.

  • 64-bit floating point values are written with 16 significant digits in the mantissa and 2 digits in the exponent.

  • Time values are written as ISO-8601 timestamps with microsecond resolution, i.e. the pattern YYYY-MM-DDTHH:mm:SS.ssssss

  • All output values are rounded normally instead of truncating fractions.

  • All output text is encoded as UTF-8.

Options

-h,--help

Show this help text

-l LEVEL,--log=LEVEL

Set the logging level, where LEVEL is one of 'debug', 'info', 'warning', 'error' in order of decreasing verbosity. All log messages go to the standard error channel, the default is 'info'.

-p,--props

Output object property rows. Each property row is tagged with a 1st column containing the string "property".

-n,--no-headers

Do not output column headers. This makes for an under-documented output file, but is useful in some cases. Using this option overrides -p if both are given.

-i,--no-id

Do not output logical dataset IDs in the first column. Das streams can define multiple datasets but if a data source is known to generate only a single dataset in each stream, then the ID column may be omitted without loss of clarity.

-d DELIM

Change the default text delimiter from ';' (semicolon) to some other ASCII 7-bit character.

-r DIGITS

Set the number of significant digits for general output. The minimum resolution is 2 significant digits.

-s SUBSEC

Set the sub-second resolution. Output N digits of sub-second resolution. The minimum value is 0, thus time values are always output to at least seconds resolution.

Addendum

Limitations: Data are always output so that one spreadsheet row corresponds to one increment of the major array index for each dataset. Thus high rank datasets are flattened, which may make them somewhat awkward to work with. In those cases its recommended to transform streams using das2_ascii or das3_cdf

See Also:

Source: das3_csv.c

Author: C. Piker

Clone this wiki locally