Skip to content
This repository has been archived by the owner on Jun 16, 2023. It is now read-only.

jstorm 获取系统(linux)获取可用内存应该使用/proc/meminfo中的MemAvailable #427

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

jstorm 获取系统(linux)获取可用内存应该使用/proc/meminfo中的MemAvailable #427

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6eb7e18
getMemAvailable
superwood Feb 9, 2017
3e515df
using_available_mem_and_forward_compatibility_free_mem
superwood Feb 10, 2017
5e720d5
using_String_index_of
superwood Feb 10, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion jstorm-core/src/main/java/com/alibaba/jstorm/daemon/supervisor/Heartbeat.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ private List<Integer> calculatorAvailablePorts() {
if (cpuUsage <= 0.0 || !ConfigExtension.isSupervisorEnableAutoAdjustSlots(conf)) {
return defaultPortList;
}
long freeMemory = JStormUtils.getFreePhysicalMem() * 1024L;
long freeMemory = JStormUtils.getAvailablePhysicalMem() * 1024L;

long reserveMemory = ConfigExtension.getStormMachineReserveMem(conf);

Expand Down
4 changes: 4 additions & 0 deletions jstorm-core/src/main/java/com/alibaba/jstorm/utils/JStormUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -518,6 +518,10 @@ public static Long getFreePhysicalMem() {
return LinuxResource.getFreePhysicalMem();
}

public static Long getAvailablePhysicalMem() {
return LinuxResource.getAvailablePhysicalMem();
}

public static int getNumProcessors() {
return LinuxResource.getProcessNum();
}
Expand Down
14 changes: 13 additions & 1 deletion jstorm-core/src/main/java/com/alibaba/jstorm/utils/LinuxResource.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,19 @@ public static Long getFreePhysicalMem() {
}
return 0L;
}

public static Long getAvailablePhysicalMem() {
if (!OSInfo.isLinux()) {
return 0L;
}
try {
List<String> lines = IOUtils.readLines(new FileInputStream(PROCFS_MEMINFO));
String free = lines.get(2).split("\\s+")[1];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里的逻辑，我觉得应该反过来，默认使用mem free，当存在MemAvailable的时候才用这个。
另外，默认的情况下，需要考虑mem free + cache

return Long.valueOf(free);
} catch (Exception ignored) {
LOG.warn("failed to get total free memory.");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@superwood , 切换到available memory 没有什么问题，

但这里有个兼容性的问题， 如果os 不带“MemAvailable”时，需要额外增加一个逻辑，回退到getFreeMemory

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@longdafeng 没问题，可以在代码里面做一个兼容。 稍后重新提交下。

}
return 0L;
}
/**
* calcute the disk usage at current filesystem
* @return
Expand Down