forked from alibaba/metrics
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
411 changed files
with
48,570 additions
and
24 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,23 +1,10 @@ | ||
# Compiled class file | ||
*.class | ||
|
||
# Log file | ||
*.log | ||
|
||
# BlueJ files | ||
*.ctxt | ||
|
||
# Mobile Tools for Java (J2ME) | ||
.mtj.tmp/ | ||
|
||
# Package Files # | ||
*.jar | ||
*.war | ||
*.nar | ||
*.ear | ||
*.zip | ||
*.tar.gz | ||
*.rar | ||
|
||
# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml | ||
hs_err_pid* | ||
target/ | ||
.settings/ | ||
.project | ||
.classpath | ||
/bin/ | ||
.idea | ||
*.iml | ||
.DS_Store | ||
.flattened-pom.xml | ||
*.orig |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
Alibaba Metrics | ||
Copyright 2016-2017 Aibaba Inc. | ||
|
||
This product includes software developed by Alibaba Inc. | ||
|
||
This product includes software developed by Coda Hale and Yammer, Inc. | ||
|
||
This product includes code derived from the JSR-166 project (ThreadLocalRandom, Striped64, | ||
LongAdder), which was released with the following comments: | ||
|
||
Written by Doug Lea with assistance from members of JCP JSR-166 | ||
Expert Group and released to the public domain, as explained at | ||
http://creativecommons.org/publicdomain/zero/1.0/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,108 @@ | ||
Dubbo Metrics简介 | ||
=== | ||
Dubbo Metrics是一套标准度量库, 用于提供对从操作系统, 虚拟机, 容器,到应用的全方位, 多维度, 实时, 准确的度量服务。 | ||
|
||
Metrics命名规范 | ||
=== | ||
|
||
我们把一个metric的名字抽象为MetricName, 它由两部分组成, key和tag。 | ||
|
||
### key的概念 | ||
|
||
key是一个由`.`分隔的字符串, 它描述了这个metric的基本含义, 例如`system.cpu.user`描述了用户cpu的使用率这一指标。 | ||
|
||
### tag的概念 | ||
|
||
metrics的本质是在某个时间点上, 某个key的某个value的一个快照, 它是一个随时间变化的时间序列数据。 而metrics如果只考虑单机上查看这一场景的话, 其实光有key就已经够了, 因为只需要把每个时间点的数据展示出来就好了。 | ||
|
||
那为什么会有tag呢? 因为我们的metrics数据需要支持按应用, 单元, 机房等多维度进行聚合, 所以要把这些数据存到专门的时间序列数据库。 为了更好的支持在时间序列数据库中进行动态聚合操作, 我们引入了tag的概念。 | ||
|
||
tag由两部分组成, tagKey和tagValue, 形式为`{tagKey=tagValue}`。一个tag可以理解为对一组数据全集的一个完整划分, tagKey表示划分方式, tagValue表示划分后的值。 | ||
|
||
例如交易下单的场景, 我们需要知道用户是从pc下单, 还是手机下单。 那么按下单来源划分的话, 可以把所有交易下单划分来自pc和来自手机(暂不考虑其他场景), 所以tagKey就是source, tagValue就是pc或者mobile, 记为`{source=pc}`和`{source=mobile}`. | ||
|
||
> 注意, tag和tag之间应当是完全正交的, 也就是没有任何的交叉关系。 | ||
tag的存在使得时间序列数据库中的数据聚合变得非常灵活。例如, 我们把订单创建按照来源进行划分之后, 当指定了`{source=*}`这一查询条件时, 就可以方便的聚合出, 某个时间段内来自于pc端的创建量, 和来自手机端的创建量, 这一点类似于传统数据库的`group by`操作。 当不指定`source`这一tag的时候, 我们能聚合出总的创建量。 | ||
|
||
### 命名规范 | ||
|
||
1. key和tag只支持:`[a-z][A-Z][0-9][-_./]`, 不能有空格, 大小写敏感, key原则上不包含大写。 | ||
2. 格式为`department.app_name.category[.sub_category]*`, `category`和`sub_category`里面如果有多个单词,用下划线'_'连接, 不要用'.'连接 | ||
3. 需要动态聚合的维度, 放在tag里面, 同时在tagKey也在key中体现。 不需要聚合的维度, 放在key里面。 | ||
4. 不要使用太多的tag, 一般而言4-5个已经足够 | ||
|
||
#### 举例 | ||
|
||
| key | tag | | ||
|------|--------| | ||
| department.dubbo.consumer.qps | | | ||
| department.dubbo.consumer.service.qps | service=org.apache.xxxServer | | ||
|
||
|
||
### <a href="#metric-level" />Metric的等级 | ||
|
||
构建metric的时候,可以传入一个等级,用于表征指标的重要程度, 默认是`NORMAL`。 | ||
目前有5个级别,按照重要性程度从高到低排列依次为: | ||
|
||
``` | ||
CRITICAL > MAJOR > NORMAL > MINOR > TRIVIAL | ||
``` | ||
|
||
为了保证数据的实时性, 请不要过多的创建CRITICAL级别的指标。 | ||
|
||
Metric等级的一个重要用途是控制落盘的频率, 默认情况下频率为: | ||
|
||
| 等级 | 落盘频率 | | ||
|---- | --- | | ||
|CRTICAL| 5s| | ||
|MAJOR| 10s| | ||
| NORMAL| 15s| | ||
|MINOR |30s| | ||
|TRIVIAL |1min| | ||
|
||
|
||
## 统一的数据格式 | ||
|
||
为了实时的了解应用程序的状态,所有的指标应当实时的得到更新,并且能够通过某种接口暴露出来,这里我们采用了HTTP的方式进行暴露。 | ||
|
||
为了保证数据的连续性,内存态的指标数据最终形态还是持久化。持久化的最佳方案是日志,因为日志天然可以异构系统中进行共享。 | ||
|
||
我们期望的数据格式是`结构化`, `自描述`,`可扩展`的。因此metrics采用了如下的格式: | ||
|
||
![image](http://git.cn-hangzhou.oss.aliyun-inc.com/uploads/middleware-container/ali-metrics/3a7116b4ea0a58d3bad7d345a4ad4d06/image.png) | ||
|
||
* metric key:存储当前指标的key | ||
* metric tag:存储当前指标的tag | ||
* metric value:存储当前指标的值 | ||
* timestamp:存储当前的时间戳 | ||
* metadata:存储当前指标的一些元信息,元信息并不对这个指标本身造成影响,而用来帮助读取这些数据的程序理解该指标的特性 | ||
|
||
不管是内存态的实时数据,还是持久化的数据,都应该通过统一的格式来展示。 | ||
|
||
这里,dubbo-metrics选用了`JSON`作为其结构化数据的格式,一条持久化后的metrics数据展示如下: | ||
|
||
``` | ||
{"metric":"department.dubbo.read.count","metricType":"COUNTER","tags":{"appName":"sample"},"timestamp":1470298287916,"value":1167126} | ||
``` | ||
|
||
### dubbo-metrics Java SDK | ||
|
||
基于上述的度量场景,dubbo-metrics提供了Java层面的SDK,包含了以下4种基础度量场景 | ||
|
||
* Counter: 用于对累加型数据进行度量 | ||
* Gauge:用于对瞬态型数据进行度量 | ||
* Meter:用于对变化速率型指标进行度量 | ||
* Histogram:用于对分布型数据进行度量 | ||
|
||
并且在此基础上,封装了更加简便的API,来方便对典型的业务场景进行埋点统计 | ||
|
||
* Timer: 用于统计某个接口的调用速率(qps)和rt分布 | ||
* Compass:用于统计某个接口的调用速率,rt分布,成功次数,错误码 | ||
|
||
### 使用方法和Demo | ||
|
||
具体的使用方法,[metrics-demo](http://gitlab.alibaba-inc.com/middleware-container/dubbo-metrics/wikis/demo) | ||
|
||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
<?xml version="1.0" encoding="UTF-8"?> | ||
<project xmlns="http://maven.apache.org/POM/4.0.0" | ||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
<modelVersion>4.0.0</modelVersion> | ||
<parent> | ||
<artifactId>dubbo-metrics-all</artifactId> | ||
<groupId>com.alibaba.middleware</groupId> | ||
<version>${metrics.version}</version> | ||
<relativePath>../pom.xml</relativePath> | ||
</parent> | ||
|
||
<artifactId>metrics-annotation-springaop</artifactId> | ||
<name>${project.artifactId}</name> | ||
|
||
|
||
<dependencies> | ||
<dependency> | ||
<groupId>com.alibaba.middleware</groupId> | ||
<artifactId>metrics-annotation</artifactId> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.springframework</groupId> | ||
<artifactId>spring-context</artifactId> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.springframework</groupId> | ||
<artifactId>spring-aop</artifactId> | ||
</dependency> | ||
<dependency> | ||
<groupId>cglib</groupId> | ||
<artifactId>cglib-nodep</artifactId> | ||
</dependency> | ||
<!-- test dependencies --> | ||
<dependency> | ||
<groupId>com.alibaba.middleware</groupId> | ||
<artifactId>metrics-core-impl</artifactId> | ||
<scope>test</scope> | ||
</dependency> | ||
<dependency> | ||
<groupId>junit</groupId> | ||
<artifactId>junit</artifactId> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.springframework</groupId> | ||
<artifactId>spring-test</artifactId> | ||
</dependency> | ||
</dependencies> | ||
|
||
</project> |
48 changes: 48 additions & 0 deletions
48
...n-springaop/src/main/java/com/alibaba/metrics/annotation/GaugeRegistry4SpringContext.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
package com.alibaba.metrics.annotation; | ||
|
||
import java.lang.reflect.Method; | ||
import java.util.Map; | ||
|
||
import com.alibaba.metrics.MetricManager; | ||
|
||
import org.springframework.aop.support.AopUtils; | ||
import org.springframework.context.ApplicationContext; | ||
import org.springframework.context.ApplicationListener; | ||
import org.springframework.context.event.ContextRefreshedEvent; | ||
|
||
/** | ||
* 扫描spring容器中的所有bean,识别标记有{@linkplain com.alibaba.metrics.annotation.EnableGauge}注解的方法,并将其注册至 | ||
* {@linkplain MetricManager} | ||
* | ||
* 注意:被注解的方法必须为public的方法 | ||
* | ||
*/ | ||
public class GaugeRegistry4SpringContext implements ApplicationListener<ContextRefreshedEvent> { | ||
@Override | ||
public void onApplicationEvent(ContextRefreshedEvent contextRefreshedEvent) { | ||
ApplicationContext applicationContext = contextRefreshedEvent.getApplicationContext(); | ||
//仅在root application context启动后执行 | ||
if (applicationContext.getParent() != null) { | ||
return; | ||
} | ||
Map<String, Object> beansOfType = applicationContext.getBeansOfType(Object.class); | ||
for (Object bean : beansOfType.values()) { | ||
if (bean == null) { | ||
return; | ||
} | ||
Class<?> targetClass = AopUtils.getTargetClass(bean); | ||
Method[] methods = targetClass.getMethods(); | ||
if (methods == null || methods.length == 0) { | ||
return; | ||
} | ||
for (Method method : methods) { | ||
EnableGauge aliGaugeAnnotation = method.getAnnotation(EnableGauge.class); | ||
if (aliGaugeAnnotation == null) { | ||
continue; | ||
} | ||
GaugeRegistry.registerGauge(aliGaugeAnnotation, bean, method); | ||
} | ||
} | ||
} | ||
|
||
} |
44 changes: 44 additions & 0 deletions
44
...-springaop/src/main/java/com/alibaba/metrics/annotation/MetricsAnnotationInterceptor.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
package com.alibaba.metrics.annotation; | ||
|
||
|
||
import org.aspectj.lang.ProceedingJoinPoint; | ||
import org.aspectj.lang.annotation.Around; | ||
import org.aspectj.lang.annotation.Aspect; | ||
import org.springframework.beans.factory.annotation.Autowired; | ||
import org.springframework.context.annotation.Bean; | ||
import org.springframework.context.annotation.Configuration; | ||
import org.springframework.context.annotation.EnableAspectJAutoProxy; | ||
|
||
/** | ||
* 除Gauge类型度量指标注解外的其他注解对应的拦截器 | ||
* | ||
*/ | ||
@Aspect | ||
@EnableAspectJAutoProxy | ||
@Configuration | ||
public class MetricsAnnotationInterceptor { | ||
|
||
@Autowired | ||
private MetricsAroundAdvice aroundAdvice; | ||
|
||
@Around("@annotation(com.alibaba.metrics.annotation.EnableCounter)" | ||
+ "|| @annotation(com.alibaba.metrics.annotation.EnableMeter)" | ||
+ "|| @annotation(com.alibaba.metrics.annotation.EnableHistogram)" | ||
+ "|| @annotation(com.alibaba.metrics.annotation.EnableTimer)" | ||
+ "|| @annotation(com.alibaba.metrics.annotation.EnableFastCompass)" | ||
+ "|| @annotation(com.alibaba.metrics.annotation.EnableCompass)") | ||
public Object wrapWithTryCatch(ProceedingJoinPoint joinPoint) throws Throwable { | ||
return this.aroundAdvice.wrapWithTryCatch(joinPoint); | ||
} | ||
|
||
|
||
@Bean | ||
public MetricsAroundAdvice cache() { | ||
return new MetricsAroundAdvice(); | ||
} | ||
|
||
@Bean | ||
public GaugeRegistry4SpringContext gaugeRegistry4SpringContext() { | ||
return new GaugeRegistry4SpringContext(); | ||
} | ||
} |
48 changes: 48 additions & 0 deletions
48
...tation-springaop/src/test/java/com/alibaba/metrics/annotation/test/EnableCompassTest.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
package com.alibaba.metrics.annotation.test; | ||
|
||
import com.alibaba.metrics.Compass; | ||
import com.alibaba.metrics.MetricManager; | ||
import com.alibaba.metrics.MetricName; | ||
import com.alibaba.metrics.annotation.MetricsAnnotationInterceptor; | ||
|
||
import junit.framework.TestCase; | ||
import org.junit.Test; | ||
import org.junit.runner.RunWith; | ||
import org.springframework.beans.factory.annotation.Autowired; | ||
import org.springframework.test.context.ContextConfiguration; | ||
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; | ||
|
||
@RunWith(SpringJUnit4ClassRunner.class) | ||
@ContextConfiguration(classes = {MetricsAnnotationInterceptor.class, TestConfig.class}) | ||
public class EnableCompassTest { | ||
|
||
@Autowired | ||
private MetricsAnnotationTestService dubboMetricsTestService; | ||
|
||
@Test | ||
public void test() { | ||
this.dubboMetricsTestService.testCompass1(); | ||
|
||
Compass compass = MetricManager.getCompass("test", | ||
MetricName.build("ascp.upcp-scitem.metrics-annotation.compass.test1") | ||
.tagged("purpose", "test")); | ||
|
||
long successCount = compass.getSuccessCount(); | ||
TestCase.assertEquals(1, successCount); | ||
long count = compass.getCount(); | ||
TestCase.assertEquals(1, count); | ||
|
||
try { | ||
this.dubboMetricsTestService.testCompass1(); | ||
this.dubboMetricsTestService.testCompass2(); | ||
TestCase.fail(); | ||
} catch (Exception e) { | ||
//ignore | ||
} | ||
|
||
successCount = compass.getSuccessCount(); | ||
TestCase.assertEquals(2, successCount); | ||
count = compass.getCount(); | ||
TestCase.assertEquals(3, count); | ||
} | ||
} |
Oops, something went wrong.