HTTP transport and configuration string

Release overview

Major API improvements, including support for InfluxDB Line Protocol over HTTP.

Breaking change: With the introduction of the HTTP transport it is now mandatory to select the protocol to be used by the client.

What is new

Support for the ILP protocol over HTTP transport, as well as the new config string syntax

• The new HTTP sender has better error reporting. In particular, it returns an error if the QuestDB server failed to write ILP messages for any reason.
• HTTP sender handles network errors and retries writes automatically.
• HTTP sender reuses connections to avoid open connection overhead for each flush() call.
• The new fromConf() method provides a convenient way for constructing a Sender via a configuration string.

Migration

v2 code example:

const sender = new Sender();
await sender.connect({ port: 9009, host: 'localhost' });
sender.table('tablename').intColumn('id', 0).atNow();
await sender.flush();
await sender.close();

Migrated v3 code:

const sender = Sender.fromConfig('http::addr=localhost:9000');
await sender.table('tablename').intColumn('id', 0).atNow();
await sender.flush();
await sender.close();

Note, that the migrated code uses the HTTP sender instead of the TCP one.

Simpler authentication

Previous versions of the client required the entire JWK for authentication.

const sender = new Sender({
    jwk: {
        kid: '<username>',
        d: '<private key>'
        x: '<public key x>',
        y: '<public key y>',
        kty: 'EC',
        crv: 'P-256'
    }
});

Starting with v2.1.0 the client supports a simpler way of authentication, where it is enough to specify only the username and the user's private key.

const sender = new Sender({
    auth: {
        keyId: '<username>',
        token: '<private key>'
    }
});

Safer timestamp API

An optional time unit parameter can be passed to timestamp-accepting methods:

  const bday = Date.parse('1856-07-10');
  sender
    .table('inventors')
    .symbol('born', 'Austrian Empire')
    .timestampColumn('birthday', bday, 'ms') // notice 'ms' here
    .intColumn('id', 0)
    .stringColumn('name', 'Nicola Tesla')
    .at(Date.now(), 'ms'); // notice 'ms' here

The time unit defaults to 'us' (native QuestDB resolution) in both timestampColumn() and at().

Supported values are:

• 'ns' - nanoseconds
• 'us' - microseconds
• 'ms' - milliseconds

Note: This release contains a breaking change.
at() does not accept string type anymore, and number/BigInt values are interpreted as microseconds instead of nanoseconds.

BigInt support for timestamp values

The client's API will accept BigInt types wherever a timestamp needs to be passed.
The API change is backwards compatible.

Sender option 'copyBuffer' defaults to 'true'

• sender option 'copyBuffer' defaults to 'true'
• type checks for sender options
• worker threads example

TCP keepalive

TCP keepalive has been enabled for the QuestDB socket connection to avoid disconnects caused by client inactivity.
Should work in most cases; if the connection goes through a firewall this solution might not be good enough.
In that case users could add empty messages on application layer probably to keep the connection alive.
The long term solution is definitely to improve the client further and implement keepalive on ILP protocol level (or whatever will be replacing it in the future).

Custom logging

A new option to pass a custom logging function to the client.
If no logger specified the client will write log messages to console, default logging level is info.

Add 'copyBuffer' option

This release introduces a new option called 'copyBuffer', the option is not set by default.
If the option is set the client will create a new buffer for each flush() call with the data belongs to the returned promise.
This prevents creating duplicate rows if await is missed when calling flush() or if the calls to flush() are not serialised.

Example:

const sender = new Sender({ copyBuffer: true });

First release

Node.js client for QuestDB.

Connect to QuestDB and ingest data via ILP.
Supports TLS encryption and ILP authentication.