Skip to content

Commit

Permalink
use memory tracker in a containered env (#2541)
Browse files Browse the repository at this point in the history
* use memory tracker in a containered env

* comment fix

* Update docs-2.0-zh/5.configurations-and-logs/1.configurations/3.graph-config.md

* Update docs-2.0-zh/5.configurations-and-logs/1.configurations/4.storage-config.md

---------

Co-authored-by: cooper-lzy <[email protected]>
  • Loading branch information
abby-cyber and cooper-lzy authored Apr 19, 2024
1 parent 11794b4 commit d0430aa
Show file tree
Hide file tree
Showing 4 changed files with 64 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,23 @@ For all parameters and their current values, see [Configurations](1.configuratio

## Memory tracker configurations

!!! Note

Memory Tracker is a memory management tool designed to monitor and limit memory usage. For large-scale queries, Memory Tracker can prevent Out Of Memory (OOM) issues. If you're using Memory Tracker in a containerized environment, you need to add the relevant configurations to the configuration file of the Graph service.

1. Create the directory `/sys/fs/cgroup/graphd/`, and then add and configure the `memory.max` file under the directory.
2. Add the following configurations to `etc/nebula-graphd.conf`.

```bash
--containerized=true
--cgroup_v2_controllers=/sys/fs/cgroup/graphd/cgroup.controllers
--cgroup_v2_memory_stat_path=/sys/fs/cgroup/graphd/memory.stat
--cgroup_v2_memory_max_path=/sys/fs/cgroup/graphd/memory.max
--cgroup_v2_memory_current_path=/sys/fs/cgroup/graphd/memory.current
```

For more details, see [Memory Tracker: Memory Management Practice in NebulaGraph Database](https://www.nebula-graph.io/posts/memory-tracker-practices).

| Name | Predefined value | Description |Whether supports runtime dynamic modifications|
| :------------------- | :------------------------ | :------------------------------------------ |:------------------|
|`memory_tracker_limit_ratio` |`0.8` | The value of this parameter can be set to `(0, 1]`, `2`, and `3`.<br/>**Caution: When setting this parameter, ensure that the value of `system_memory_high_watermark_ratio` is not set to `1`, otherwise the value of this parameter will not take effect.**<br/>`(0, 1]`: The percentage of available memory. Formula: `Percentage of available memory = Available memory / (Total memory - Reserved memory)`.<br/>When an ongoing query results in memory usage exceeding the configured limit, the query fails and subsequently the memory is released. <br/> **Note**: For the hybrid deployment of a cluster with cloud-based and on-premises nodes, the value of `memory_tracker_limit_ratio` should be set to a **lower** value. For example, when the graphd is expected to occupy only 50% of memory, the value can be set to less than `0.5`.<br/>`2`: Dynamic Self Adaptive mode. MemoryTracker dynamically adjusts the available memory based on the system's current available memory. <br/>**Note**: This feature is experimental. As memory usage cannot be monitored in real time in dynamic adaptive mode, an OOM error may still occur to handle large memory allocations. <br/>`3`: Disable MemoryTracker. MemoryTracker only logs memory usage and does not interfere with executions even if the limit is exceeded.| Yes|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,22 @@ For more information, see [RocksDB official documentation](https://rocksdb.org/)
## Memory Tracker configurations
For details about Memory Tracker, see [Memory Tracker: Memory Management Practice in NebulaGraph Database](https://www.nebula-graph.io/posts/memory-tracker-practices)
!!! Note
Memory Tracker is a memory management tool designed to monitor and limit memory usage. For large-scale queries, Memory Tracker can prevent Out Of Memory (OOM) issues. If you're using Memory Tracker in a containerized environment, you need to add the relevant configurations to the configuration file of the Storage service.
1. Create the directory `/sys/fs/cgroup/storaged/`, and then add and configure the `memory.max` file under the directory.
2. Add the following configurations to `etc/nebula-storaged.conf`.
```bash
--containerized=true
--cgroup_v2_controllers=/sys/fs/cgroup/graphd/cgroup.controllers
--cgroup_v2_memory_stat_path=/sys/fs/cgroup/graphd/memory.stat
--cgroup_v2_memory_max_path=/sys/fs/cgroup/graphd/memory.max
--cgroup_v2_memory_current_path=/sys/fs/cgroup/graphd/memory.current
```
For more details, see [Memory Tracker: Memory Management Practice in NebulaGraph Database](https://www.nebula-graph.io/posts/memory-tracker-practices).
| Name | Predefined value | Description |Whether supports runtime dynamic modifications|
| :------------------- | :------------------------ | :------------------------------------------ |:------------------|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -128,8 +128,22 @@ Graph 服务提供了两份初始配置文件`nebula-graphd.conf.default`和`neb

## memory tracker 配置

有关 Memory Tracker 的详细信息,请参见[图数据库 NebulaGraph 的内存管理实践之 Memory Tracker](https://discuss.nebula-graph.com.cn/t/topic/13128)
!!! note

Memory Tracker 是一个内存管理工具,用于监控和限制内存使用。对于大型查询,Memory Tracker 可以防止 OOM 问题。如果在容器环境中使用 Memory Tracker,需要在 Graph 服务的配置文件中添加相关配置。操作如下:

1. 创建目录`/sys/fs/cgroup/graphd/`,添加并配置`memory.max`。
2. 在`etc/nebula-graphd.conf`添加以下配置。
```bash
--containerized=true
--cgroup_v2_controllers=/sys/fs/cgroup/graphd/cgroup.controllers
--cgroup_v2_memory_stat_path=/sys/fs/cgroup/graphd/memory.stat
--cgroup_v2_memory_max_path=/sys/fs/cgroup/graphd/memory.max
--cgroup_v2_memory_current_path=/sys/fs/cgroup/graphd/memory.current
```

详细信息,请参见[容器/cgroup 模式](https://discuss.nebula-graph.com.cn/t/topic/13128)。
| 名称 | 预设值 | 说明 |是否支持运行时动态修改|
| :------------------- | :------------------------ | :------------------------------------------ |---------------------|
|`memory_tracker_limit_ratio` |`0.8` |取值可设置为:`(0, 1]`、`2`、`3`。<br/>**警惕:设置该参数时请确保`system_memory_high_watermark_ratio`的值不为`1`,否则该参数的值不生效。**<br/>`(0, 1]`:可用内存的百分比。计算公式:可用内存的百分比 = 可用内存 / (总内存 - 保留内存)。<br/>当正在进行的查询导致内存使用超过配置的内存限制时,该查询会失败,并在失败后释放内存。<br/> **注意**:对于云上和本地节点混合部署的集群,需要根据实际情况**调小**该参数。例如,当预期 Graphd 只占用 50% 的内存时,该参数的值可设置为小于`0.5`。<br/>`2`:动态自适应模式(Dynamic Self Adaptive),Memory Tracker 会根据系统当前的可用内存,动态调整可用内存。<br/>**注意**:此功能为实验性功能,由于动态自适应不能做到实时监控操作系统内存使用情况,在一些大内存分配的场景,还是会存在 OOM 可能。<br/>`3`:关掉 Memory Tracker,Memory Tracker 将只记录内存使用情况,即使超过限额,也不会干预执行。|支持|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,22 @@ rocksdb options 配置的格式为`{"<option_name>":"<option_value>"}`,多个
## memory tracker 配置
有关 Memory Tracker 的详细信息,请参见[图数据库 NebulaGraph 的内存管理实践之 Memory Tracker](https://discuss.nebula-graph.com.cn/t/topic/13128)。
!!! note
Memory Tracker 是一个内存管理工具,用于监控和限制内存使用。对于大型查询,Memory Tracker 可以防止 OOM 问题。如果在容器环境中使用 Memory Tracker,需要在 Storage 服务的配置文件中添加相关配置。操作如下:
1. 创建目录`/sys/fs/cgroup/storaged/`,添加并配置`memory.max`。
2. 在`etc/nebula-storaged.conf`添加以下配置。
```bash
--containerized=true
--cgroup_v2_controllers=/sys/fs/cgroup/graphd/cgroup.controllers
--cgroup_v2_memory_stat_path=/sys/fs/cgroup/graphd/memory.stat
--cgroup_v2_memory_max_path=/sys/fs/cgroup/graphd/memory.max
--cgroup_v2_memory_current_path=/sys/fs/cgroup/graphd/memory.current
```
详细信息,请参见[图数据库 NebulaGraph 的内存管理实践之 Memory Tracker](https://discuss.nebula-graph.com.cn/t/topic/13128)。
| 名称 | 预设值 | 说明 |是否支持运行时动态修改|
| :------------------- | :------------------------ | :------------------------------------------ |:----------------------- |
Expand Down

0 comments on commit d0430aa

Please sign in to comment.