Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Agents auto-retry connect to heartbeat server alternative addresses #368

Open
wants to merge 41 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
480f325
Agents auto-retry connect to heartbeat server alternative addresses, …
Jan 25, 2018
0934f87
Implementation string matcher monitoring.
Mar 20, 2018
06c064f
Merge github.com:open-falcon/falcon-plus
Mar 20, 2018
89f3bd3
Improve string matcher, filter matched strings and save them into DB.
Mar 22, 2018
2a04408
Append github.com/jmoiron/sqlx into govendor.
Mar 22, 2018
4590a80
Append vendor/ into git ignore list.
Mar 23, 2018
feac80a
Fix package sqlx dependencies.
Mar 23, 2018
706aa3e
Set mysql user falcon password falcon instaed of root by default.
Mar 23, 2018
cdc06c6
map ValueRaw in JSON.
Mar 23, 2018
c6d727e
Fix forward ValueRaw in gateway.
Mar 23, 2018
6395ed5
Merge branch 'master' of github.com:shuge/falcon-plus
Mar 23, 2018
77b1a9c
Fix default DB password.
Mar 30, 2018
f9854b5
Add counterType=STRMATCH for function match(pattern, period).
Mar 30, 2018
a6d3021
add Delete hard-coded metric="str.match" in judge.
Mar 30, 2018
6233402
Fix counterType=g.STRMATCH metric in graph.
Mar 30, 2018
b9b81b6
Return invalid metric detail in RPC call response.
Jun 6, 2018
0ae1769
Delete vendor/github.com/ from VC.
Jun 11, 2018
d2d37e8
Fix db falcon_portal password in configuration.
Jun 14, 2018
0e36eac
Append upgrade note for support multiple-metrics extend expression ve…
Jun 14, 2018
bb53982
Fix db falcon_portal password in configuration.
Jun 14, 2018
dd49f4a
Fix db falcon_portal password in configuration.
Jun 14, 2018
2829f10
Intro new model EExpression;
Jun 14, 2018
05f8cbe
Intro new model EExpression;
Jun 14, 2018
a507ad7
Add new RPC method for forwarding EMetric data.
Jun 14, 2018
5c6f9a1
Add new RPC method for forwarding EMetric data.
Jun 14, 2018
012e303
Intro new EMetric and EExpression models and support multiple-metrics…
Jun 14, 2018
262805a
Merge branch 'master' of github.com:shuge/falcon-plus
Jun 14, 2018
43182a9
Use the term EExp instead of EExpression.
Jun 24, 2018
72af697
Support multiple-metrics extend expression.
Jun 25, 2018
92d2ff1
Support multiple-metrics extend expression.
Jun 25, 2018
5c0ee67
Support multiple-metrics extend expression.
Jun 25, 2018
302abb3
Support multiple-metrics extend expression.
Jun 25, 2018
2a04fbd
Mute log.
Jun 26, 2018
7fcfa2a
Fix type.
Jul 10, 2018
b71100c
Use falcon as db default username, password and dbn.
Jul 11, 2018
c05eae3
Disable stringMatcher by default.
Jul 11, 2018
1e1b986
Add MMEE mysql DB schemas.
Jul 11, 2018
43bf957
Delete 7_portal-db-schema-extra.sql.
Jul 11, 2018
963e064
Support redis password.
Jul 11, 2018
549d4f1
Support redis password.
Jul 11, 2018
ced89e1
Fix parse EExp.
Jul 11, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,19 @@ out/
*.swo
*.tar.gz
docs/_site
vendor/

# binary file
falcon-agent
falcon-aggregator
falcon-alarm
falcon-api
falcon-gateway
falcon-graph
falcon-hbs
falcon-judge
falcon-nodata
falcon-transfer
open-falcon

git.go
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,12 @@ mysql -h 127.0.0.1 -u root -p < 4_graph-db-schema.sql
mysql -h 127.0.0.1 -u root -p < 5_alarms-db-schema.sql
```

Optional feature: matched string persistence in MySQL DB

mysql -h 127.0.0.1 -u root -p < scripts/mysql/db_schema/6_history-db-schema.sql



**NOTE: if you are upgrading from v0.1 to current version v0.2.0,then**. [More upgrading instruction](http://www.jianshu.com/p/6fb2c2b4d030)

mysql -h 127.0.0.1 -u root -p < 5_alarms-db-schema.sql
Expand Down
125 changes: 125 additions & 0 deletions common/model/eexp.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
package model

import (
"encoding/json"
"log"
"math"
"strconv"
)

type Filter struct {
Func string `json:"func"`
Key string `json:"key"`
Ago uint64 `json:"ago"` // for func count
Hits uint64 `json:"hits"` // for func count
Limit uint64 `json:"limit"` // for func all
Operator string `json:"operator"`
RightValue interface{} `json:"rightValue"`
}

type EExpResponse struct {
EExps []EExp `json:"eexps"`
}

type EExp struct {
ID int `json:"id"`
Key string `json:"key"` // join(sorted(conditionKeys), ",")
Filters map[string]Filter `json:"filters"`
Priority int `json:"priority"`
MaxStep int `json:"maxStep"`
Note string `json:"note"`
}

func (c *Filter) String() string {
out, _ := json.Marshal(c)
return string(out)
}

func (ee *EExp) String() string {
out, _ := json.Marshal(ee)
return string(out)
}

func opResultFloat64(leftValue float64, operator string, rightValue float64) (isTriggered bool) {
switch operator {
case "=", "==":
isTriggered = math.Abs(leftValue-rightValue) < 0.0001
case "!=":
isTriggered = math.Abs(leftValue-rightValue) > 0.0001
case "<":
isTriggered = leftValue < rightValue
case "<=":
isTriggered = leftValue <= rightValue
case ">":
isTriggered = leftValue > rightValue
case ">=":
isTriggered = leftValue >= rightValue
}
return
}

func opResultString(leftValue string, operator string, rightValue string) (isTriggered bool) {
switch operator {
case "=", "==":
isTriggered = leftValue == rightValue
case "!=":
isTriggered = leftValue != rightValue
}
return
}

func (ee *EExp) HitFilters(m *map[string]interface{}) bool {
var err error
for k, filter := range ee.Filters {
valueI, ok := (*m)[k]
if !ok {
return false
}

switch filter.RightValue.(type) {
case float64:
{

var leftValue float64
leftValueStr, ok := valueI.(string)
if ok {
leftValue, err = strconv.ParseFloat(leftValueStr, 64)
if err != nil {
log.Println("strconv.ParseFloat failed", err)
return false
}
} else {
leftValue, ok = valueI.(float64)
if !ok {
log.Println("parse in float64 failed")
return false
}
}

rightValue := filter.RightValue.(float64)

if !opResultFloat64(leftValue, filter.Operator, rightValue) {
return false
}

}
case string:
{
leftValue, ok := valueI.(string)
if !ok {
log.Println("parse in string failed")
return false
}

rightValue := filter.RightValue.(string)

if !opResultString(leftValue, filter.Operator, rightValue) {
return false
}

}
}

}
return true
}
6 changes: 6 additions & 0 deletions common/model/ejudge.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package model

type EHistoryData struct {
Timestamp int64 `json:"timestamp"`
Filters map[string]interface{} `json:"filters"`
}
18 changes: 18 additions & 0 deletions common/model/emetric.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
package model

type EMetric struct {
Endpoint string `json:"endpoint"`
Key string `json:"key"`
Filters map[string]interface{} `json:"values"`
Timestamp int64 `json:"timestamp"`
}

func NewEMetric() *EMetric {
e := EMetric{}
e.Filters = map[string]interface{}{}
return &e
}

func (e *EMetric) PK() string {
return e.Key
}
92 changes: 81 additions & 11 deletions common/model/event.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
package model

import (
"encoding/json"
"fmt"

"github.com/open-falcon/falcon-plus/common/utils"
Expand All @@ -25,9 +26,10 @@ type Event struct {
Id string `json:"id"`
Strategy *Strategy `json:"strategy"`
Expression *Expression `json:"expression"`
EExp *EExp `json:"eexp"`
Status string `json:"status"` // OK or PROBLEM
Endpoint string `json:"endpoint"`
LeftValue float64 `json:"leftValue"`
LeftValue interface{} `json:"leftValue"`
CurrentStep int `json:"currentStep"`
EventTime int64 `json:"eventTime"`
PushedTags map[string]string `json:"pushedTags"`
Expand All @@ -38,13 +40,23 @@ func (this *Event) FormattedTime() string {
}

func (this *Event) String() string {
var leftValue interface{}
switch this.LeftValue.(type) {
case float64:
leftValue = utils.ReadableFloat(this.LeftValue.(float64))
default:
leftValue = this.LeftValue

}

return fmt.Sprintf(
"<Endpoint:%s, Status:%s, Strategy:%v, Expression:%v, LeftValue:%s, CurrentStep:%d, PushedTags:%v, TS:%s>",
"<Endpoint:%s, Status:%s, Strategy:%v, Expression:%v, EExp:%v LeftValue:%v, CurrentStep:%d, PushedTags:%v, TS:%s>",
this.Endpoint,
this.Status,
this.Strategy,
this.Expression,
utils.ReadableFloat(this.LeftValue),
this.EExp,
leftValue,
this.CurrentStep,
this.PushedTags,
this.FormattedTime(),
Expand All @@ -59,6 +71,13 @@ func (this *Event) ExpressionId() int {
return 0
}

func (this *Event) EExpID() int {
if this.Expression != nil {
return this.EExp.ID
}
return 0
}

func (this *Event) StrategyId() int {
if this.Strategy != nil {
return this.Strategy.Id
Expand Down Expand Up @@ -88,56 +107,107 @@ func (this *Event) ActionId() int {
return this.Expression.ActionId
}

return this.Strategy.Tpl.ActionId
if this.Strategy != nil {
return this.Strategy.Tpl.ActionId
}

return -1

}

func (this *Event) Priority() int {
if this.Strategy != nil {
return this.Strategy.Priority
}
return this.Expression.Priority
if this.Expression != nil {
return this.Expression.Priority
}

if this.EExp != nil {
return this.EExp.Priority
}
return -1
}

func (this *Event) Note() string {
if this.Strategy != nil {
return this.Strategy.Note
}
return this.Expression.Note

if this.Expression != nil {
return this.Expression.Note

}
if this.EExp != nil {
return this.EExp.Note
}
return ""
}

func (this *Event) Metric() string {
if this.Strategy != nil {
return this.Strategy.Metric
}
return this.Expression.Metric
if this.Expression != nil {
return this.Expression.Metric
}
if this.EExp != nil {
return this.EExp.Key
}
return ""
}

func (this *Event) RightValue() float64 {
if this.Strategy != nil {
return this.Strategy.RightValue
}
return this.Expression.RightValue

if this.Expression != nil {
return this.Expression.RightValue
}

return 0.0
}

func (this *Event) Operator() string {
if this.Strategy != nil {
return this.Strategy.Operator
}
return this.Expression.Operator

if this.Expression != nil {
return this.Expression.Operator
}
return ""
}

func (this *Event) Func() string {
if this.Strategy != nil {
return this.Strategy.Func
}
return this.Expression.Func

if this.Expression != nil {
return this.Expression.Func
}

if this.EExp != nil {
out, _ := json.Marshal(this.EExp.Filters)
return string(out)
}
return ""
}

func (this *Event) MaxStep() int {
if this.Strategy != nil {
return this.Strategy.MaxStep
}
return this.Expression.MaxStep

if this.Expression != nil {
return this.Expression.MaxStep
}
if this.EExp != nil {
return this.EExp.MaxStep
}
return 1
}

func (this *Event) Counter() string {
Expand Down
5 changes: 4 additions & 1 deletion common/model/judge.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,18 @@ type JudgeItem struct {
Endpoint string `json:"endpoint"`
Metric string `json:"metric"`
Value float64 `json:"value"`
ValueRaw string `json:"valueRaw"`
Timestamp int64 `json:"timestamp"`
JudgeType string `json:"judgeType"`
Tags map[string]string `json:"tags"`
}

func (this *JudgeItem) String() string {
return fmt.Sprintf("<Endpoint:%s, Metric:%s, Value:%f, Timestamp:%d, JudgeType:%s Tags:%v>",
return fmt.Sprintf("<Endpoint:%s, Metric:%s, Value:%.2f ValueRaw:%s, Timestamp:%d, JudgeType:%s Tags:%v>",
this.Endpoint,
this.Metric,
this.Value,
this.ValueRaw,
this.Timestamp,
this.JudgeType,
this.Tags)
Expand All @@ -46,4 +48,5 @@ func (this *JudgeItem) PrimaryKey() string {
type HistoryData struct {
Timestamp int64 `json:"timestamp"`
Value float64 `json:"value"`
ValueRaw string `json:"valueRaw"`
}
Loading