Skip to content

Commit

Permalink
init commit
Browse files Browse the repository at this point in the history
  • Loading branch information
nzomkxia committed Jan 24, 2019
1 parent 1cba452 commit b87c4a5
Show file tree
Hide file tree
Showing 411 changed files with 48,570 additions and 24 deletions.
33 changes: 10 additions & 23 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,23 +1,10 @@
# Compiled class file
*.class

# Log file
*.log

# BlueJ files
*.ctxt

# Mobile Tools for Java (J2ME)
.mtj.tmp/

# Package Files #
*.jar
*.war
*.nar
*.ear
*.zip
*.tar.gz
*.rar

# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
hs_err_pid*
target/
.settings/
.project
.classpath
/bin/
.idea
*.iml
.DS_Store
.flattened-pom.xml
*.orig
5 changes: 4 additions & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Apache License
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/

Expand Down Expand Up @@ -199,3 +199,6 @@
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

This product contains a modified portion of 'dropwizard metrics', a java implement Capturing JVM- and application-level metrics. also
under a "Apache License 2.0" license, see https://github.com/dropwizard/metrics/blob/4.1-development/LICENSE
13 changes: 13 additions & 0 deletions NOTICE
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Alibaba Metrics
Copyright 2016-2017 Aibaba Inc.

This product includes software developed by Alibaba Inc.

This product includes software developed by Coda Hale and Yammer, Inc.

This product includes code derived from the JSR-166 project (ThreadLocalRandom, Striped64,
LongAdder), which was released with the following comments:

Written by Doug Lea with assistance from members of JCP JSR-166
Expert Group and released to the public domain, as explained at
http://creativecommons.org/publicdomain/zero/1.0/
108 changes: 108 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
Dubbo Metrics简介
===
Dubbo Metrics是一套标准度量库, 用于提供对从操作系统, 虚拟机, 容器,到应用的全方位, 多维度, 实时, 准确的度量服务。

Metrics命名规范
===

我们把一个metric的名字抽象为MetricName, 它由两部分组成, key和tag。

### key的概念

key是一个由`.`分隔的字符串, 它描述了这个metric的基本含义, 例如`system.cpu.user`描述了用户cpu的使用率这一指标。

### tag的概念

metrics的本质是在某个时间点上, 某个key的某个value的一个快照, 它是一个随时间变化的时间序列数据。 而metrics如果只考虑单机上查看这一场景的话, 其实光有key就已经够了, 因为只需要把每个时间点的数据展示出来就好了。

那为什么会有tag呢? 因为我们的metrics数据需要支持按应用, 单元, 机房等多维度进行聚合, 所以要把这些数据存到专门的时间序列数据库。 为了更好的支持在时间序列数据库中进行动态聚合操作, 我们引入了tag的概念。

tag由两部分组成, tagKey和tagValue, 形式为`{tagKey=tagValue}`。一个tag可以理解为对一组数据全集的一个完整划分, tagKey表示划分方式, tagValue表示划分后的值。

例如交易下单的场景, 我们需要知道用户是从pc下单, 还是手机下单。 那么按下单来源划分的话, 可以把所有交易下单划分来自pc和来自手机(暂不考虑其他场景), 所以tagKey就是source, tagValue就是pc或者mobile, 记为`{source=pc}``{source=mobile}`.

> 注意, tag和tag之间应当是完全正交的, 也就是没有任何的交叉关系。
tag的存在使得时间序列数据库中的数据聚合变得非常灵活。例如, 我们把订单创建按照来源进行划分之后, 当指定了`{source=*}`这一查询条件时, 就可以方便的聚合出, 某个时间段内来自于pc端的创建量, 和来自手机端的创建量, 这一点类似于传统数据库的`group by`操作。 当不指定`source`这一tag的时候, 我们能聚合出总的创建量。

### 命名规范

1. key和tag只支持:`[a-z][A-Z][0-9][-_./]`, 不能有空格, 大小写敏感, key原则上不包含大写。
2. 格式为`department.app_name.category[.sub_category]*`, `category``sub_category`里面如果有多个单词,用下划线'_'连接, 不要用'.'连接
3. 需要动态聚合的维度, 放在tag里面, 同时在tagKey也在key中体现。 不需要聚合的维度, 放在key里面。
4. 不要使用太多的tag, 一般而言4-5个已经足够

#### 举例

| key | tag |
|------|--------|
| department.dubbo.consumer.qps | |
| department.dubbo.consumer.service.qps | service=org.apache.xxxServer |


### <a href="#metric-level" />Metric的等级

构建metric的时候,可以传入一个等级,用于表征指标的重要程度, 默认是`NORMAL`
目前有5个级别,按照重要性程度从高到低排列依次为:

```
CRITICAL > MAJOR > NORMAL > MINOR > TRIVIAL
```

为了保证数据的实时性, 请不要过多的创建CRITICAL级别的指标。

Metric等级的一个重要用途是控制落盘的频率, 默认情况下频率为:

| 等级 | 落盘频率 |
|---- | --- |
|CRTICAL| 5s|
|MAJOR| 10s|
| NORMAL| 15s|
|MINOR |30s|
|TRIVIAL |1min|


## 统一的数据格式

为了实时的了解应用程序的状态,所有的指标应当实时的得到更新,并且能够通过某种接口暴露出来,这里我们采用了HTTP的方式进行暴露。

为了保证数据的连续性,内存态的指标数据最终形态还是持久化。持久化的最佳方案是日志,因为日志天然可以异构系统中进行共享。

我们期望的数据格式是`结构化`, `自描述``可扩展`的。因此metrics采用了如下的格式:

![image](http://git.cn-hangzhou.oss.aliyun-inc.com/uploads/middleware-container/ali-metrics/3a7116b4ea0a58d3bad7d345a4ad4d06/image.png)

* metric key:存储当前指标的key
* metric tag:存储当前指标的tag
* metric value:存储当前指标的值
* timestamp:存储当前的时间戳
* metadata:存储当前指标的一些元信息,元信息并不对这个指标本身造成影响,而用来帮助读取这些数据的程序理解该指标的特性

不管是内存态的实时数据,还是持久化的数据,都应该通过统一的格式来展示。

这里,dubbo-metrics选用了`JSON`作为其结构化数据的格式,一条持久化后的metrics数据展示如下:

```
{"metric":"department.dubbo.read.count","metricType":"COUNTER","tags":{"appName":"sample"},"timestamp":1470298287916,"value":1167126}
```

### dubbo-metrics Java SDK

基于上述的度量场景,dubbo-metrics提供了Java层面的SDK,包含了以下4种基础度量场景

* Counter: 用于对累加型数据进行度量
* Gauge:用于对瞬态型数据进行度量
* Meter:用于对变化速率型指标进行度量
* Histogram:用于对分布型数据进行度量

并且在此基础上,封装了更加简便的API,来方便对典型的业务场景进行埋点统计

* Timer: 用于统计某个接口的调用速率(qps)和rt分布
* Compass:用于统计某个接口的调用速率,rt分布,成功次数,错误码

### 使用方法和Demo

具体的使用方法,[metrics-demo](http://gitlab.alibaba-inc.com/middleware-container/dubbo-metrics/wikis/demo)



50 changes: 50 additions & 0 deletions metrics-annotation-springaop/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<artifactId>dubbo-metrics-all</artifactId>
<groupId>com.alibaba.middleware</groupId>
<version>${metrics.version}</version>
<relativePath>../pom.xml</relativePath>
</parent>

<artifactId>metrics-annotation-springaop</artifactId>
<name>${project.artifactId}</name>


<dependencies>
<dependency>
<groupId>com.alibaba.middleware</groupId>
<artifactId>metrics-annotation</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-aop</artifactId>
</dependency>
<dependency>
<groupId>cglib</groupId>
<artifactId>cglib-nodep</artifactId>
</dependency>
<!-- test dependencies -->
<dependency>
<groupId>com.alibaba.middleware</groupId>
<artifactId>metrics-core-impl</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
</dependency>
</dependencies>

</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
package com.alibaba.metrics.annotation;

import java.lang.reflect.Method;
import java.util.Map;

import com.alibaba.metrics.MetricManager;

import org.springframework.aop.support.AopUtils;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationListener;
import org.springframework.context.event.ContextRefreshedEvent;

/**
* 扫描spring容器中的所有bean,识别标记有{@linkplain com.alibaba.metrics.annotation.EnableGauge}注解的方法,并将其注册至
* {@linkplain MetricManager}
*
* 注意:被注解的方法必须为public的方法
*
*/
public class GaugeRegistry4SpringContext implements ApplicationListener<ContextRefreshedEvent> {
@Override
public void onApplicationEvent(ContextRefreshedEvent contextRefreshedEvent) {
ApplicationContext applicationContext = contextRefreshedEvent.getApplicationContext();
//仅在root application context启动后执行
if (applicationContext.getParent() != null) {
return;
}
Map<String, Object> beansOfType = applicationContext.getBeansOfType(Object.class);
for (Object bean : beansOfType.values()) {
if (bean == null) {
return;
}
Class<?> targetClass = AopUtils.getTargetClass(bean);
Method[] methods = targetClass.getMethods();
if (methods == null || methods.length == 0) {
return;
}
for (Method method : methods) {
EnableGauge aliGaugeAnnotation = method.getAnnotation(EnableGauge.class);
if (aliGaugeAnnotation == null) {
continue;
}
GaugeRegistry.registerGauge(aliGaugeAnnotation, bean, method);
}
}
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
package com.alibaba.metrics.annotation;


import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.EnableAspectJAutoProxy;

/**
* 除Gauge类型度量指标注解外的其他注解对应的拦截器
*
*/
@Aspect
@EnableAspectJAutoProxy
@Configuration
public class MetricsAnnotationInterceptor {

@Autowired
private MetricsAroundAdvice aroundAdvice;

@Around("@annotation(com.alibaba.metrics.annotation.EnableCounter)"
+ "|| @annotation(com.alibaba.metrics.annotation.EnableMeter)"
+ "|| @annotation(com.alibaba.metrics.annotation.EnableHistogram)"
+ "|| @annotation(com.alibaba.metrics.annotation.EnableTimer)"
+ "|| @annotation(com.alibaba.metrics.annotation.EnableFastCompass)"
+ "|| @annotation(com.alibaba.metrics.annotation.EnableCompass)")
public Object wrapWithTryCatch(ProceedingJoinPoint joinPoint) throws Throwable {
return this.aroundAdvice.wrapWithTryCatch(joinPoint);
}


@Bean
public MetricsAroundAdvice cache() {
return new MetricsAroundAdvice();
}

@Bean
public GaugeRegistry4SpringContext gaugeRegistry4SpringContext() {
return new GaugeRegistry4SpringContext();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
package com.alibaba.metrics.annotation.test;

import com.alibaba.metrics.Compass;
import com.alibaba.metrics.MetricManager;
import com.alibaba.metrics.MetricName;
import com.alibaba.metrics.annotation.MetricsAnnotationInterceptor;

import junit.framework.TestCase;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = {MetricsAnnotationInterceptor.class, TestConfig.class})
public class EnableCompassTest {

@Autowired
private MetricsAnnotationTestService dubboMetricsTestService;

@Test
public void test() {
this.dubboMetricsTestService.testCompass1();

Compass compass = MetricManager.getCompass("test",
MetricName.build("ascp.upcp-scitem.metrics-annotation.compass.test1")
.tagged("purpose", "test"));

long successCount = compass.getSuccessCount();
TestCase.assertEquals(1, successCount);
long count = compass.getCount();
TestCase.assertEquals(1, count);

try {
this.dubboMetricsTestService.testCompass1();
this.dubboMetricsTestService.testCompass2();
TestCase.fail();
} catch (Exception e) {
//ignore
}

successCount = compass.getSuccessCount();
TestCase.assertEquals(2, successCount);
count = compass.getCount();
TestCase.assertEquals(3, count);
}
}
Loading

0 comments on commit b87c4a5

Please sign in to comment.