OTEL Collector metrics data is not visible at skywalking portal

[vishnu-k1]

Hi Team,

I'm using custom otel collector with Host Metrics Receiver for which collected data's are not showing up at portal, can someone help on this.

Regards,
Vishnu K.

[wu-sheng]

SkyWalking requires you to write MAL to process the metrics and config UI dashboard to visualize them.

• https://skywalking.apache.org/docs/main/latest/en/concepts-and-designs/mal/
• https://skywalking.apache.org/docs/main/latest/en/ui/readme/

[vishnu-k1]

is there any guide available to write MAL

[vishnu-k1]

{"resourceMetrics":[{"resource":{},"scopeMetrics":[{"scope":{"name":"otelcol/hostmetricsreceiver/cpu","version":"1.0.0"},"metrics":[{"name":"system.cpu.time","description":"Total seconds each logical CPU spent on each mode.","unit":"s","sum":{"dataPoints":[{"attributes":[{"key":"cpu","value":{"stringValue":"cpu0"}},{"key":"state","value":{"stringValue":"user"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":76695.92},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu0"}},{"key":"state","value":{"stringValue":"system"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":13229.05},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu0"}},{"key":"state","value":{"stringValue":"idle"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":1641203.69},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu0"}},{"key":"state","value":{"stringValue":"interrupt"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":3412.51},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu0"}},{"key":"state","value":{"stringValue":"nice"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":420.23},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu0"}},{"key":"state","value":{"stringValue":"softirq"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":1512.88},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu0"}},{"key":"state","value":{"stringValue":"steal"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":0},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu0"}},{"key":"state","value":{"stringValue":"wait"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":1583.52},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu1"}},{"key":"state","value":{"stringValue":"user"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":76457.01},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu1"}},{"key":"state","value":{"stringValue":"system"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":13267.49},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu1"}},{"key":"state","value":{"stringValue":"idle"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":1641217.07},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu1"}},{"key":"state","value":{"stringValue":"interrupt"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":3458.51},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu1"}},{"key":"state","value":{"stringValue":"nice"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":421.81},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu1"}},{"key":"state","value":{"stringValue":"softirq"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":1568.65},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu1"}},{"key":"state","value":{"stringValue":"steal"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":0},{"attributes":[{"key":"cpu","value":{"stringValue":"cpu1"}},{"key":"state","value":{"stringValue":"wait"}}],"startTimeUnixNano":"1718108515000000000","timeUnixNano":"1719849598230218084","asDouble":1630.04}],"aggregationTemporality":2,"isMonotonic":true}}]}],"schemaUrl":"https://opentelemetry.io/schemas/1.9.0"}]}

[vishnu-k1]

this is the json output of metrics being collected

[vishnu-k1]

here is the error when I use mal config under otel rules

filter: "{ tags -> tags.job_name == 'mta-metrics' }" # The OpenTelemetry job name
expSuffix: service(['node_identifier_host_name'] , Layer.GENERAL)
metricPrefix: mta-metrics
metricsRules:

• name: system.cpu.time
  exp: system.cpu.time

2024-07-01 15:03:31,707 - org.apache.skywalking.oap.meter.analyzer.dsl.Expression - 92 [main] ERROR [] - failed to run "(system.disk.io).service(['node_identifier_host_name'] , Layer.OS_LINUX)"
groovy.lang.MissingPropertyException: No such property: disk for class: org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:67) ~[groovy-4.0.15.jar:4.0.15]
at org.codehaus.groovy.vmplugin.v8.IndyGuardsFiltersAndSignatures.unwrap(IndyGuardsFiltersAndSignatures.java:163) ~[groovy-4.0.15.jar:4.0.15]
at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321) ~[groovy-4.0.15.jar:4.0.15]
at Script1.run(Script1.groovy:1) ~[?:?]
at org.apache.skywalking.oap.meter.analyzer.dsl.Expression.run(Expression.java:81) ~[meter-analyzer-10.0.1.jar:10.0.1]

[wu-sheng]

I have given you the docs, also there are a lot of examples you could refer to, https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-vm-monitoring/

Note, MAL is powered by Groovy script. You need to make sure you at least write them in the correct grammaer.

[wu-sheng]

For your write, at least, system.cpu.time should be system_cpu_time.

A CPU monitoring from OTEL Linux exporter, https://github.com/apache/skywalking/blob/master/oap-server/server-starter/src/main/resources/otel-rules/vm.yaml