Skip to content

Commit

Permalink
[Feature] add seatunnel monitor (apache#2782)
Browse files Browse the repository at this point in the history 
Co-authored-by: shown <[email protected]>
Co-authored-by: aias00 <[email protected]>
Co-authored-by: Zhang Yuxuan <[email protected]>
  • Loading branch information
@zhangshenghang @yuluo-yx
@Aias00 @zuobiao-zhou
4 people authored Oct 29, 2024
1 parent afe68ba commit 1267fca
Show file tree
Hide file tree
Showing 4 changed files with 687 additions and 1 deletion.
99 changes: 99 additions & 0 deletions home/docs/help/seatunnel.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
---
id: seatunnel
title: Monitoring： SeaTunnel
sidebar_label: SeaTunnel
keywords: [ Open Source Monitoring System, Monitor SeaTunnel ]
---

> Collect monitoring metrics for SeaTunnel.
### Configuration Parameters

| Parameter Name | Parameter Help Description |
|-------------------|-----------------------------------------------------------------------------------------------------------------|
| Target Host | The monitored endpoint's IPV4, IPV6, or domain name. Note ⚠️ no protocol header (e.g., https://, http://). |
| Task Name | The name that identifies this monitoring task, which needs to be unique. |
| Port | The monitoring port opened by SeaTunnel, default value: 5801. |
| SSL | Whether SSL is enabled for connecting to SeaTunnel. |
| Query Timeout | Set the timeout for unresponsive queries, in milliseconds (ms), default 6000 ms. |
| Collection Interval| The interval time for periodic data collection, in seconds; the minimum interval that can be set is 30 seconds. |
| Binding Tags | Used for categorizing and managing monitoring resources. |
| Description Notes | Additional identification and description notes for this monitoring; users can add notes here. |

### Collected Metrics

#### Metric Set: Cluster Overview

| Metric Name | Metric Unit | Metric Help Description |
|-----------------------|-------------|----------------------------|
| projectVersion | None | Project version |
| gitCommitAbbrev | None | Git commit hash |
| totalSlot | None | Total number of slots |
| unassignedSlot | None | Number of unassigned slots |
| runningJobs | None | Number of running tasks |
| finishedJobs | None | Number of completed tasks |
| failedJobs | None | Number of failed tasks |
| cancelledJobs | None | Number of cancelled tasks |
| workers | None | Number of workers |

#### Metric Set: Thread Information

| Metric Name | Metric Unit | Metric Help Description |
|------------------|-------------|-------------------------|
| threadName | None | Thread name |
| threadId | None | Thread ID |
| threadState | None | Thread state |
| stackTrace | None | Stack trace information |

#### Metric Set: Node Monitoring

| Metric Name | Metric Unit | Metric Help Description |
|---------------------------------|-------------|------------------------------|
| isMaster | None | Whether it is a master node |
| host | None | IP address |
| port | None | Port |
| processors | None | Number of processors |
| physical.memory.total | None | Total physical memory |
| physical.memory.free | None | Available physical memory |
| swap.space.total | None | Total swap space |
| swap.space.free | None | Available swap space |
| heap.memory.used | None | Used heap memory |
| heap.memory.free | None | Available heap memory |
| heap.memory.total | None | Total heap memory |
| heap.memory.max | None | Maximum heap memory |
| heap.memory.used/total | None | Heap memory usage rate |
| heap.memory.used/max | None | Maximum heap memory usage rate|
| minor.gc.count | None | Minor garbage collection count|
| minor.gc.time | None | Minor garbage collection time |
| major.gc.count | None | Major garbage collection count|
| major.gc.time | None | Major garbage collection time |
| load.process | None | Process load |
| load.system | None | System load |
| load.systemAverage | None | Average system load |
| thread.count | None | Number of threads |
| thread.peakCount | None | Peak thread count |
| cluster.timeDiff | None | Cluster time difference |
| event.q.size | None | Event queue size |
| executor.q.async.size | None | Asynchronous execution queue size |
| executor.q.client.size | None | Client execution queue size |
| executor.q.client.query.size | None | Client query queue size |
| executor.q.client.blocking.size | None | Client blocking queue size |
| executor.q.query.size | None | Query queue size |
| executor.q.scheduled.size | None | Scheduled execution queue size |
| executor.q.io.size | None | IO queue size |
| executor.q.system.size | None | System execution queue size |
| executor.q.operations.size | None | Operations queue size |
| executor.q.priorityOperation.size| None | Priority operations queue size |
| operations.completed.count | None | Completed operations count |
| executor.q.mapLoad.size | None | Map load queue size |
| executor.q.mapLoadAllKeys.size | None | Map load all keys size |
| executor.q.cluster.size | None | Cluster execution queue size |
| executor.q.response.size | None | Response queue size |
| operations.running.count | None | Number of running operations |
| operations.pending.invocations.percentage | None | Percentage of pending invocations |
| operations.pending.invocations.count | None | Number of pending invocations |
| proxy.count | None | Number of proxies |
| clientEndpoint.count | None | Number of client endpoints |
| connection.active.count | None | Number of active connections |
| client.connection.count | None | Number of client connections |
| connection.count | None | Total number of connections |
99 changes: 99 additions & 0 deletions home/i18n/zh-cn/docusaurus-plugin-content-docs/current/help/seatunnel.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
---
id: seatunnel
title: 监控：SeaTunnel
sidebar_label: SeaTunnel
keywords: [ 开源监控系统, 监控SeaTunnel ]
---

> 对 SeaTunnel 指标进行采集监控。
### 配置参数

| 参数名称 | 参数帮助描述 |
|---------|------------------------------------------------------|
| 目标Host | 被监控的对端IPV4，IPV6或域名。注意⚠️不带协议头(eg: https://, http://)。 |
| 任务名称 | 标识此监控的名称，名称需要保证唯一性。 |
| 端口 | SeaTunnel开放的监控端口，默认值：5801。 |
| 启用HTTPS | 是否启用HTTPS。 |
| 查询超时时间 | 设置查询未响应数据时的超时时间，单位ms毫秒，默认6000毫秒。 |
| 采集间隔 | 监控周期性采集数据间隔时间，单位秒，可设置的最小间隔为30秒 |
| 绑定标签 | 用于对监控资源进行分类管理。 |
| 描述备注 | 更多标识和描述此监控的备注信息，用户可以在这里备注信息。 |

### 采集指标

#### 指标集合：集群概览

| 指标名称 | 指标单位 | 指标帮助描述 |
|-----------------------|------|----------|
| projectVersion | 无 | 项目版本 |
| gitCommitAbbrev | 无 | Git提交编号 |
| totalSlot | 无 | 总槽位数 |
| unassignedSlot | 无 | 未分配槽位数 |
| runningJobs | 无 | 运行中任务数 |
| finishedJobs | 无 | 已完成任务数 |
| failedJobs | 无 | 失败任务数 |
| cancelledJobs | 无 | 取消任务数 |
| workers | 无 | Worker数 |

#### 指标集合：线程信息

| 指标名称 | 指标单位 | 指标帮助描述 |
|-------------|------|--------|
| threadName | 无 | 线程名称 |
| threadId | 无 | 线程ID |
| threadState | 无 | 线程状态 |
| stackTrace | 无 | 堆栈信息 |

#### 指标集合：节点监控

| 指标名称 | 指标单位 | 指标帮助描述 |
|------------|------|------------|
| isMaster | 无 | 是否主节点 |
| host | 无 | IP地址 |
| port | 无 | 端口 |
| processors | 无 | 处理器数 |
| physical.memory.total | 无 | 物理内存总量 |
| physical.memory.free | 无 | 物理内存可用 |
| swap.space.total | 无 | 交换空间总量 |
| swap.space.free | 无 | 交换空间可用 |
| heap.memory.used | 无 | 堆内存已用 |
| heap.memory.free | 无 | 堆内存可用 |
| heap.memory.total | 无 | 堆内存总量 |
| heap.memory.max | 无 | 堆内存最大 |
| heap.memory.used/total | 无 | 堆内存使用率 |
| heap.memory.used/max | 无 | 堆内存使用率最大 |
| minor.gc.count | 无 | 垃圾回收次数 |
| minor.gc.time | 无 | 垃圾回收时间 |
| major.gc.count | 无 | Major垃圾回收次数 |
| major.gc.time | 无 | Major垃圾回收时间 |
| load.process | 无 | 进程负载 |
| load.system | 无 | 系统负载 |
| load.systemAverage | 无 | 系统平均负载 |
| thread.count | 无 | 线程数 |
| thread.peakCount | 无 | 线程峰值 |
| cluster.timeDiff | 无 | 集群时间差 |
| event.q.size | 无 | 事件队列大小 |
| executor.q.async.size | 无 | 异步执行队列大小 |
| executor.q.client.size | 无 | 客户端执行队列大小 |
| executor.q.client.query.size | 无 | 客户端查询队列大小 |
| executor.q.client.blocking.size | 无 | 客户端阻塞队列大小 |
| executor.q.query.size | 无 | 查询队列大小 |
| executor.q.scheduled.size | 无 | 定时执行队列大小 |
| executor.q.io.size | 无 | IO队列大小 |
| executor.q.system.size | 无 | 系统执行队列大小 |
| executor.q.operations.size | 无 | 操作队列大小 |
| executor.q.priorityOperation.size | 无 | 优先操作队列大小 |
| operations.completed.count | 无 | 完成操作次数 |
| executor.q.mapLoad.size | 无 | 映射加载队列大小 |
| executor.q.mapLoadAllKeys.size | 无 | 映射加载所有键大小 |
| executor.q.cluster.size | 无 | 集群执行队列大小 |
| executor.q.response.size | 无 | 响应队列大小 |
| operations.running.count | 无 | 正在运行的操作数量 |
| operations.pending.invocations.percentage | 无 | 待处理调用百分比 |
| operations.pending.invocations.count | 无 | 待处理调用数量 |
| proxy.count | 无 | 代理数量 |
| clientEndpoint.count | 无 | 客户端端点数量 |
| connection.active.count | 无 | 活动连接数 |
| client.connection.count | 无 | 客户端连接数 |
| connection.count | 无 | 总连接数 |
3 changes: 2 additions & 1 deletion home/sidebars.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,8 @@
"help/elasticsearch",
"help/influxdb",
"help/presto",
"help/hugegraph"
"help/hugegraph",
"help/seatunnel"
]
},
{
Expand Down
Loading

0 comments on commit 1267fca

Please sign in to comment.