Skip to content

Commit

Permalink
Add docs
Browse files Browse the repository at this point in the history
  • Loading branch information
ljupcovangelski committed May 30, 2023
1 parent 4df45fc commit f9133ea
Show file tree
Hide file tree
Showing 3 changed files with 174 additions and 0 deletions.
169 changes: 169 additions & 0 deletions docs/docs/api/endpoints/streams.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
---
title: Streams
sidebar_label: Streams
---

Refer to our [stream](getting-started/glossary.md#stream) definition
for more information.

## List

`POST /streams.list`

The list of created streams is returned.

**Sample response**

```json5
[
{
"name": "customapp",
"topic": "customapp"
},
{
"name": "events",
"topic": "events"
}
]
```

At the moment the name of the output topic is the same as the name of the stream itself.

## Create

`POST /streams.create`

**Sample request**

```json5
{
"name": "customapp",
"topics": [{
"name": "messages",
"fields": [{
"name": "source",
"newName": "from"
},
{
"name": "senderId",
"nawName": "sender"
}
]
},
{
"name": "channels",
"fields": [{
"name": "connectionState",
"newName": "state"
}
]
}
],
"joins": [{
"name": "newSource",
"field1": "source",
"field2": "source"
}],
"aggregations": [],
"key": "messages.source"
}
```

The name of the created stream is returned, alongside with the output topic.

**Sample response**

```json5
{
"name":"customapp",
"outputTopic":"customapp"
}
```

## Info

`POST /streams.info`

**Sample request**

```json5
{
"name": "customapp"
}
```

Information about the stream is returned.

**Sample response**

```json5
{
"stream": {
"name": "CUSTOMAPP",
"writeQueries": [{
"queryString": "CREATE STREAM CUSTOMAPP WITH (KAFKA_TOPIC='CUSTOMAPP', PARTITIONS=10, REPLICAS=1) AS SELECT\n A.SOURCE A_SOURCE,\n A.SENDERID SENDERID,\n B.CONNECTIONSTATE CONNECTIONSTATE\nFROM MESSAGES A\nINNER JOIN CHANNELS B WITHIN 365 DAYS ON ((B.SOURCE = A.SOURCE))\nEMIT CHANGES;",
"sinks": [
"CUSTOMAPP"
],
"sinkKafkaTopics": [
"CUSTOMAPP"
],
"id": "CSAS_CUSTOMAPP_79",
"statusCount": {
"RUNNING": 1
},
"queryType": "PERSISTENT",
"state": "RUNNING"
}],
"fields": [{
"name": "A_SOURCE",
"schema": {
"type": "STRING"
},
"type": "KEY"
},
{
"name": "SENDERID",
"schema": {
"type": "STRING"
}
},
{
"name": "CONNECTIONSTATE",
"schema": {
"type": "STRING"
}
}
],
"type": "STREAM",
"keyFormat": "KAFKA",
"valueFormat": "AVRO",
"topic": "CUSTOMAPP",
"partitions": 10,
"replication": 1,
"statement": "CREATE STREAM CUSTOMAPP WITH (KAFKA_TOPIC='CUSTOMAPP', PARTITIONS=10, REPLICAS=1) AS SELECT\n A.SOURCE A_SOURCE,\n A.SENDERID SENDERID,\n B.CONNECTIONSTATE CONNECTIONSTATE\nFROM MESSAGES A\nINNER JOIN CHANNELS B WITHIN 365 DAYS ON ((B.SOURCE = A.SOURCE))\nEMIT CHANGES;"
}
}
```

## Delete

`POST /streams.delete`

**Sample request**

```json5
{
"name": "customapp"
}
```

The name of the deleted stream is returned.

**Sample response**

```json5
{
"name": "customapp"
}
```
4 changes: 4 additions & 0 deletions docs/docs/getting-started/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,10 @@ conversations.
A source represents a system that generates messaging data that a user wants to
process with Airy Core.

## Stream

The whole Airy platform is based on Kafka and real-time streaming of messages. In the context of `streams` feature that Airy supports, a `stream` is the process of joining two or multiple Kafka topics, combining the data and creating an outout topic where the result of the streaming operation will be stored. It is based on KSQL.

### Provider

Source providers are API platforms that allow Airy Core to connect to one or
Expand Down
1 change: 1 addition & 0 deletions docs/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ module.exports = {
'api/endpoints/client-config',
'api/endpoints/components',
'api/endpoints/cluster',
'api/endpoints/streams',
],
},
'api/httpClient',
Expand Down

0 comments on commit f9133ea

Please sign in to comment.