Skip to content

Commit 1a075e5

Browse files
mshustovmshustov
authored
Merge pull request #1621 from ClickHouse/chjs-parquet
Add Parquet and file streaming docs for CHJS
2 parents d4f3422 + ac72777 commit 1a075e5

File tree

1 file changed

+75
-55
lines changed
  • docs/en/integrations/language-clients

1 file changed

+75
-55
lines changed

‎docs/en/integrations/language-clients/js.md

Lines changed: 75 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -47,9 +47,9 @@ npm i @clickhouse/client-web
4747

4848
## Compatibility with ClickHouse
4949

50-
| Client version | ClickHouse |
51-
|----------------|-------------|
52-
| 0.2.0 | 22.8 - 23.7 |
50+
| Client version | ClickHouse |
51+
|----------------|--------------|
52+
| 0.2.6 | 22.8 - 23.10 |
5353

5454
## ClickHouse Client API
5555

@@ -331,32 +331,6 @@ interface InsertParams<T> {
331331
}
332332
```
333333

334-
#### Web version limitations
335-
336-
Currently, inserts in `@clickhouse/client-web` only work with `Array<T>` and `JSON*` formats.
337-
Inserting streams is not supported in the web version yet due to poor browser compatibility.
338-
339-
This is a subject to change in the future.
340-
341-
```ts
342-
interface InsertParams<T> {
343-
// Table name to insert the data into
344-
table: string
345-
// A dataset to insert.
346-
values: ReadonlyArray<T>
347-
// Format of the dataset to insert.
348-
format?: DataFormat
349-
// ClickHouse settings that can be applied on statement level.
350-
clickhouse_settings?: ClickHouseSettings
351-
// Parameters for query binding.
352-
query_params?: Record<string, unknown>
353-
// AbortSignal instance to cancel an insert in progress.
354-
abort_signal?: AbortSignal
355-
// query_id override; if not specified, a random identifier will be generated automatically.
356-
query_id?: string
357-
}
358-
```
359-
360334
:::important
361335
A request canceled with `abort_signal` does not guarantee that data insertion did not take place.
362336
:::
@@ -429,6 +403,35 @@ await client.insert({
429403

430404
If you have a custom INSERT statement that is difficult to model with this method, consider using [command](#command-method)
431405

406+
#### Web version limitations
407+
408+
Currently, inserts in `@clickhouse/client-web` only work with `Array<T>` and `JSON*` formats.
409+
Inserting streams is not supported in the web version yet due to poor browser compatibility.
410+
411+
Consequently, the `InsertParams` interface for the web version looks slightly different from the Node.js version,
412+
as `values` are limited to the `ReadonlyArray<T>` type only:
413+
414+
```ts
415+
interface InsertParams<T> {
416+
// Table name to insert the data into
417+
table: string
418+
// A dataset to insert.
419+
values: ReadonlyArray<T>
420+
// Format of the dataset to insert.
421+
format?: DataFormat
422+
// ClickHouse settings that can be applied on statement level.
423+
clickhouse_settings?: ClickHouseSettings
424+
// Parameters for query binding.
425+
query_params?: Record<string, unknown>
426+
// AbortSignal instance to cancel an insert in progress.
427+
abort_signal?: AbortSignal
428+
// query_id override; if not specified, a random identifier will be generated automatically.
429+
query_id?: string
430+
}
431+
```
432+
433+
This is a subject to change in the future.
434+
432435
### Command method
433436

434437
It can be used for statements that do not have any output, when the format clause is not applicable, or when you are not
@@ -589,6 +592,18 @@ Closes all the open connections and releases resources. No-op in the web version
589592
await client.close()
590593
```
591594

595+
## Streaming files (Node.js only)
596+
597+
There are several file streaming examples with popular data formats (NDJSON, CSV, Parquet) in the client repository.
598+
599+
- [Streaming from an NDJSON file](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/insert_file_stream_ndjson.ts)
600+
- [Streaming from a CSV file](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/insert_file_stream_csv.ts)
601+
- [Streaming from a Parquet file](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/insert_file_stream_parquet.ts)
602+
- [Streaming into a Parquet file](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/select_parquet_as_file.ts)
603+
604+
Streaming other formats into a file should be similar to Parquet,
605+
the only difference will be in the format used for `query` call (`JSONEachRow`, `CSV`, etc.) and the output file name.
606+
592607
## Supported Data formats
593608

594609
The client handles data formats as JSON or text.
@@ -599,32 +614,37 @@ and deserialize data during the communication over the wire.
599614
Data provided in the text formats (`CSV`, `TabSeparated` and `CustomSeparated` families) are sent over the wire without
600615
additional transformations.
601616

602-
| Format | Input (array) | Input (stream) | Input (object) | Output (JSON) | Output (text) |
603-
|--------------------------------------------|---------------|----------------|----------------|---------------|---------------|
604-
| JSON | ❌ | ❌ | ✔️ | ✔️ | ✔️ |
605-
| JSONObjectEachRow | ❌ | ❌ | ✔️ | ✔️ | ✔️ |
606-
| JSONStrings | ❌ | ❌ | ✔️ | ✔️ | ✔️ |
607-
| JSONCompact | ❌ | ❌ | ✔️ | ✔️ | ✔️ |
608-
| JSONCompactStrings | ❌ | ❌ | ❌ | ✔️ | ✔️ |
609-
| JSONColumnsWithMetadata | ❌ | ❌ | ✔️ | ✔️ | ✔️ |
610-
| JSONEachRow | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
611-
| JSONStringsEachRow | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
612-
| JSONCompactEachRow | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
613-
| JSONCompactStringsEachRow | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
614-
| JSONCompactEachRowWithNames | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
615-
| JSONCompactEachRowWithNamesAndTypes | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
616-
| JSONCompactStringsEachRowWithNames | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
617-
| JSONCompactStringsEachRowWithNamesAndTypes | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
618-
| CSV | ❌ | ✔️ | ❌ | ❌ | ✔️ |
619-
| CSVWithNames | ❌ | ✔️ | ❌ | ❌ | ✔️ |
620-
| CSVWithNamesAndTypes | ❌ | ✔️ | ❌ | ❌ | ✔️ |
621-
| TabSeparated | ❌ | ✔️ | ❌ | ❌ | ✔️ |
622-
| TabSeparatedRaw | ❌ | ✔️ | ❌ | ❌ | ✔️ |
623-
| TabSeparatedWithNames | ❌ | ✔️ | ❌ | ❌ | ✔️ |
624-
| TabSeparatedWithNamesAndTypes | ❌ | ✔️ | ❌ | ❌ | ✔️ |
625-
| CustomSeparated | ❌ | ✔️ | ❌ | ❌ | ✔️ |
626-
| CustomSeparatedWithNames | ❌ | ✔️ | ❌ | ❌ | ✔️ |
627-
| CustomSeparatedWithNamesAndTypes | ❌ | ✔️ | ❌ | ❌ | ✔️ |
617+
| Format | Input (array) | Input (stream) | Input (object) | Output (JSON) | Output (text) |
618+
|--------------------------------------------|---------------|----------------|----------------|---------------|----------------|
619+
| JSON | ❌ | ❌ | ✔️ | ✔️ | ✔️ |
620+
| JSONObjectEachRow | ❌ | ❌ | ✔️ | ✔️ | ✔️ |
621+
| JSONStrings | ❌ | ❌ | ✔️ | ✔️ | ✔️ |
622+
| JSONCompact | ❌ | ❌ | ✔️ | ✔️ | ✔️ |
623+
| JSONCompactStrings | ❌ | ❌ | ❌ | ✔️ | ✔️ |
624+
| JSONColumnsWithMetadata | ❌ | ❌ | ✔️ | ✔️ | ✔️ |
625+
| JSONEachRow | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
626+
| JSONStringsEachRow | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
627+
| JSONCompactEachRow | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
628+
| JSONCompactStringsEachRow | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
629+
| JSONCompactEachRowWithNames | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
630+
| JSONCompactEachRowWithNamesAndTypes | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
631+
| JSONCompactStringsEachRowWithNames | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
632+
| JSONCompactStringsEachRowWithNamesAndTypes | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
633+
| CSV | ❌ | ✔️ | ❌ | ❌ | ✔️ |
634+
| CSVWithNames | ❌ | ✔️ | ❌ | ❌ | ✔️ |
635+
| CSVWithNamesAndTypes | ❌ | ✔️ | ❌ | ❌ | ✔️ |
636+
| TabSeparated | ❌ | ✔️ | ❌ | ❌ | ✔️ |
637+
| TabSeparatedRaw | ❌ | ✔️ | ❌ | ❌ | ✔️ |
638+
| TabSeparatedWithNames | ❌ | ✔️ | ❌ | ❌ | ✔️ |
639+
| TabSeparatedWithNamesAndTypes | ❌ | ✔️ | ❌ | ❌ | ✔️ |
640+
| CustomSeparated | ❌ | ✔️ | ❌ | ❌ | ✔️ |
641+
| CustomSeparatedWithNames | ❌ | ✔️ | ❌ | ❌ | ✔️ |
642+
| CustomSeparatedWithNamesAndTypes | ❌ | ✔️ | ❌ | ❌ | ✔️ |
643+
| Parquet | ❌ | ✔️ | ❌ | ❌ | ✔️❗- see below |
644+
645+
For Parquet, the main use case for selects likely will be writing the resulting stream into a file.
646+
See [the example](https://github.com/ClickHouse/clickhouse-js/blob/main/examples/node/select_parquet_as_file.ts)
647+
in the client repository.
628648

629649
The entire list of ClickHouse input and output formats is available
630650
[here](https://clickhouse.com/docs/en/interfaces/formats).

0 commit comments

Comments
 (0)