merge for the release of v1.6.0 (#195)

* add license checker (#175) * add release note for v1.5.0 (#178) Co-authored-by: Xin.Zh <[email protected]> * Imp: cache in reflection (#179) * benchmark result * use cache in findField * encode benchmark * call field() once * remove version * fix import sync * cache in registerPOJO * add json bench result * prune unneccessary rtype.Field(index) * cache comment * rename cache * switch to if * remove return value name * findFieldWithCache * remove if check when fieldStruct is nil Co-authored-by: 望哥 <[email protected]> * update dependency * rename serialize arg name * Create .asf.yaml * 优化hessian解码string性能，提升54% * optimize code. * optimize code. * fix code review. * optimize codes. * optimize cods. * optimize code. * update license * go.sum * ci go version * testify -> 1.4.0 * testcase * travis.yml * decode value before reflect find * setvalue * decode nilPtr to nilPtr * fix get attachment lost nil key * manually import package * add ToMapStringString unit test * rename test function name with issue * setmap * support for decode emoji. * refactor code * add unit test. * add unit tests. * refactor tests. * Update travis/main.sh (#200) - Remove duplicate key 'webhooks' - Key 'matrix' is an alias for `jobs`, using `jobs` - Specify the os and dist explicitly * Mod: modify * Code format (#199) * .gitignore * code clean * code clean * remove length check * Fix: comments * Fix: format package * Fix #181: float32 accuracy issue (#196) * Fix #181: float32 accuracy issue * Fix go fmt failure * Add the unit test case for Issue181 * Add encFloat32 in double.go to encode float32 type - Call encFloat32 to encode float32 while encoding - Add unit test case to test float32 encoding * Improve encFloat32 of double.go * Fix git fmt failure * add release note for v1.6.0 (#202) * add release note for v1.5.1 * add release note for v1.5.1 * add notice * update notice * =fix release note for v1.6.0 Co-authored-by: Joe Zou <[email protected]> Co-authored-by: Xin.Zh <[email protected]> Co-authored-by: huiren <[email protected]> Co-authored-by: Huang YunKun <[email protected]> Co-authored-by: zonghaishang <[email protected]> Co-authored-by: fangyincheng <[email protected]> Co-authored-by: champly <[email protected]> Co-authored-by: wilson chen <[email protected]> Co-authored-by: fangyincheng <[email protected]> Co-authored-by: gaoxinge <[email protected]>
apache · Jun 15, 2020 · f8682e1 · f8682e1
1 parent e026b32
commit f8682e1
Show file tree

Hide file tree

Showing 27 changed files with 739 additions and 80 deletions.
diff --git a/.asf.yaml b/.asf.yaml
@@ -0,0 +1,5 @@
+notifications:
+    commits:      [email protected]
+    issues:       [email protected]
+    pullrequests: [email protected]
+    jira_options: link label link label
diff --git a/.gitignore b/.gitignore
@@ -1,2 +1,3 @@
 .idea
+vendor
 coverage.txt
diff --git a/.travis.yml b/.travis.yml
@@ -1,22 +1,30 @@
 language: go
 
-matrix:
+jobs:
   include:
     - language: java
       jdk: openjdk8
 
 go:
   - "1.12"
 
+os: linux
+
+dist: xenial
+
 script:
   - mvn clean package -f test_hessian/pom.xml
   - mvn clean package -f test_dubbo/pom.xml
   - go fmt && [[ -z `git status -s` ]]
+  - sh before_validate_license.sh
+  - chmod u+x /tmp/tools/license/license-header-checker
+  - /tmp/tools/license/license-header-checker -v -a -r -i vendor  /tmp/tools/license/license.txt . go  && [[ -z `git status -s` ]]
   - GO111MODULE=on && go mod vendor && go test -race -v && go test -bench . -race -coverprofile=coverage.txt
 
 after_success:
   - bash <(curl -s https://codecov.io/bash)
 
 notifications:
-  webhooks: https://oapi.dingtalk.com/robot/send?access_token=27a5eb4510c8cf913b67a72832549b123a8c44655483d20443515604669de0ae
-  webhooks: https://oapi.dingtalk.com/robot/send?access_token=8250008579ed1defda3a44fb8608a38d81a55700fdfb15466315a90a7dd2045f
+  webhooks:
+    - https://oapi.dingtalk.com/robot/send?access_token=27a5eb4510c8cf913b67a72832549b123a8c44655483d20443515604669de0ae
+    - https://oapi.dingtalk.com/robot/send?access_token=8250008579ed1defda3a44fb8608a38d81a55700fdfb15466315a90a7dd2045f
diff --git a/CHANGE.md b/CHANGE.md
@@ -1,5 +1,18 @@
 # Release Notes
 
+## v1.6.0
+
+### New Features
+- ignore non-exist fields when decoding. [#201](https://github.com/apache/dubbo-go-hessian2/pull/201)
+
+### Enhancement
+- add cache in reflection to improve performance. [#179](https://github.com/apache/dubbo-go-hessian2/pull/179)
+- string decode performance improvement. [#188](https://github.com/apache/dubbo-go-hessian2/pull/188)
+
+### Bugfixes
+- fix attachment lost for nil value. [#191](https://github.com/apache/dubbo-go-hessian2/pull/191)
+- fix float32 accuracy issue. [#196](https://github.com/apache/dubbo-go-hessian2/pull/196)
+
 ## v1.5.0
 
 ### New Features

diff --git a/LICENSE b/LICENSE
@@ -174,3 +174,28 @@
       of your accepting any such warranty or additional liability.
 
    END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
diff --git a/README.md b/README.md
@@ -8,8 +8,15 @@
 
 ---
 
+> **Notice: When decoding, the java version of hessian will default skip and ignore non-exist fields.**
+> **From the version of v1.6.0 , dubbo-go-hessian2 will skip non-exist fields too, while that before v1.6.0 will return errors.**
+
 It's a golang hessian library used by [Apache/dubbo-go](https://github.com/apache/dubbo-go).
 
+There is a big performance improvement, and some bugs fix for v1.6.0, 
+thanks to [micln](https://github.com/micln), [pantianying](https://github.com/pantianying), [zonghaishang](https://github.com/zonghaishang),
+ [willson-chen](https://github.com/willson-chen), [champly](https://github.com/champly).
+
 ## Feature List
 
 * [All JDK Exceptions](https://github.com/apache/dubbo-go-hessian2/issues/59)

diff --git a/before_validate_license.sh b/before_validate_license.sh
@@ -0,0 +1,26 @@
+#
+#  Licensed to the Apache Software Foundation (ASF) under one or more
+#  contributor license agreements.  See the NOTICE file distributed with
+#  this work for additional information regarding copyright ownership.
+#  The ASF licenses this file to You under the Apache License, Version 2.0
+#  (the "License"); you may not use this file except in compliance with
+#  the License.  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+remoteLicenseCheckerPath="https://github.com/dubbogo/resources/raw/master/tools/license"
+remoteLicenseCheckerName="license-header-checker"
+remoteLicenseCheckerURL="${remoteLicenseCheckerPath}/${remoteLicenseCheckerName}"
+remoteLicenseName="license.txt"
+remoteLicenseURL="${remoteLicenseCheckerPath}/${remoteLicenseName}"
+
+licensePath="/tmp/tools/license"
+mkdir -p ${licensePath}
+wget -P "${licensePath}" ${remoteLicenseCheckerURL}
+wget -P "${licensePath}" ${remoteLicenseURL}
diff --git a/binary_test.go b/binary_test.go
@@ -20,8 +20,6 @@ package hessian
 import (
 	"bytes"
 	"fmt"
-
-	// "fmt"
 	"testing"
 )
 

diff --git a/decode.go b/decode.go
@@ -50,15 +50,54 @@ func NewDecoder(b []byte) *Decoder {
 	return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), typeRefs: &TypeRefs{records: map[string]bool{}}}
 }
 
+// NewDecoder generate a decoder instance
+func NewDecoderSize(b []byte, size int) *Decoder {
+	return &Decoder{reader: bufio.NewReaderSize(bytes.NewReader(b), size), typeRefs: &TypeRefs{records: map[string]bool{}}}
+}
+
 // NewDecoder generate a decoder instance with skip
 func NewDecoderWithSkip(b []byte) *Decoder {
 	return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), typeRefs: &TypeRefs{records: map[string]bool{}}, isSkip: true}
 }
 
+// NewCheapDecoderWithSkip generate a decoder instance with skip,
+// only for cache pool, before decode Reset should be called.
+// For example, with pooling use, will effectively improve performance
+//
+//	var hessianPool = &sync.Pool{
+//		New: func() interface{} {
+//			return hessian.NewCheapDecoderWithSkip([]byte{})
+//		},
+//	}
+//
+//	decoder := hessianPool.Get().(*hessian.Decoder)
+//	fill decode data
+//	decoder.Reset(data[:])
+//  decode anything ...
+//	hessianPool.Put(decoder)
+func NewCheapDecoderWithSkip(b []byte) *Decoder {
+	return &Decoder{reader: bufio.NewReader(bytes.NewReader(b)), isSkip: true}
+}
+
 /////////////////////////////////////////
 // utilities
 /////////////////////////////////////////
 
+func (d *Decoder) Reset(b []byte) *Decoder {
+	// reuse reader buf, avoid allocate
+	d.reader.Reset(bytes.NewReader(b))
+	d.typeRefs = &TypeRefs{records: map[string]bool{}}
+
+	if d.refs != nil {
+		d.refs = nil
+	}
+	if d.classInfoList != nil {
+		d.classInfoList = nil
+	}
+
+	return d
+}
+
 // peek a byte
 func (d *Decoder) peekByte() byte {
 	return d.peek(1)[0]
@@ -150,6 +189,8 @@ func (d *Decoder) Decode() (interface{}, error) {
 	return EnsureInterface(d.DecodeValue())
 }
 
+func (d *Decoder) Buffered() int { return d.reader.Buffered() }
+
 // DecodeValue parse hessian data, the return value maybe a reflection value when it's a map, list, object, or ref.
 func (d *Decoder) DecodeValue() (interface{}, error) {
 	var (

diff --git a/decode_test.go b/decode_test.go
@@ -33,6 +33,7 @@ import (
 
 const (
 	hessianJar = "test_hessian/target/test_hessian-1.0.0.jar"
+	testString = "hello, world! 你好，世界！"
 )
 
 func isFileExist(file string) bool {

diff --git a/double.go b/double.go
@@ -20,6 +20,7 @@ package hessian
 import (
 	"encoding/binary"
 	"math"
+	"strconv"
 )
 
 import (
@@ -62,6 +63,38 @@ END:
 		byte(bits>>32), byte(bits>>24), byte(bits>>16), byte(bits>>8), byte(bits))
 }
 
+func encFloat32(b []byte, v float32) []byte {
+	fv := float32(int32(v))
+	if fv == v {
+		iv := int32(v)
+		switch iv {
+		case 0:
+			return encByte(b, BC_DOUBLE_ZERO)
+		case 1:
+			return encByte(b, BC_DOUBLE_ONE)
+		}
+		if iv >= -0x80 && iv < 0x80 {
+			return encByte(b, BC_DOUBLE_BYTE, byte(iv))
+		} else if iv >= -0x8000 && iv < 0x8000 {
+			return encByte(b, BC_DOUBLE_SHORT, byte(iv>>8), byte(iv))
+		}
+
+		goto END
+	}
+
+END:
+	if float32(int32(v*1000)) == v*1000 {
+		iv := int32(v * 1000)
+		return encByte(b, BC_DOUBLE_MILL, byte(iv>>24), byte(iv>>16), byte(iv>>8), byte(iv))
+	} else {
+		str := strconv.FormatFloat(float64(v), 'f', -1, 32)
+		d, _ := strconv.ParseFloat(str, 64)
+		bits := math.Float64bits(d)
+		return encByte(b, BC_DOUBLE, byte(bits>>56), byte(bits>>48), byte(bits>>40),
+			byte(bits>>32), byte(bits>>24), byte(bits>>16), byte(bits>>8), byte(bits))
+	}
+}
+
 /////////////////////////////////////////
 // Double
 /////////////////////////////////////////

diff --git a/double_test.go b/double_test.go
@@ -42,6 +42,33 @@ func TestEncDouble(t *testing.T) {
 	t.Logf("decode(%v) = %v, %v\n", v, res, err)
 }
 
+func TestIssue181(t *testing.T) {
+	var (
+		v   float32
+		err error
+		e   *Encoder
+		d   *Decoder
+		res interface{}
+	)
+
+	e = NewEncoder()
+	v = 99.8
+	e.Encode(v)
+	if len(e.Buffer()) == 0 {
+		t.Fail()
+	}
+
+	// res would be '99.800003' without patches in PR #196
+	d = NewDecoder(e.Buffer())
+	res, err = d.Decode()
+	f := res.(float64)
+	if float32(f) != v {
+		t.Errorf("decode(%v) = %v, %v\n", v, res, err)
+		return
+	}
+	t.Logf("decode(%v) = %v\n", v, res)
+}
+
 func TestDouble(t *testing.T) {
 	testDecodeFramework(t, "replyDouble_0_0", 0.0)
 	testDecodeFramework(t, "replyDouble_0_001", 0.001)

diff --git a/encode.go b/encode.go
@@ -112,7 +112,7 @@ func (e *Encoder) Encode(v interface{}) error {
 		}
 
 	case float32:
-		e.buffer = encFloat(e.buffer, float64(val))
+		e.buffer = encFloat32(e.buffer, val)
 
 	case float64:
 		e.buffer = encFloat(e.buffer, val)

diff --git a/go.mod b/go.mod
@@ -1,7 +1,7 @@
 module github.com/apache/dubbo-go-hessian2
 
 require (
-	github.com/dubbogo/gost v1.5.1
-	github.com/pkg/errors v0.8.1
+	github.com/dubbogo/gost v1.9.0
+	github.com/pkg/errors v0.9.1
 	github.com/stretchr/testify v1.4.0
 )
diff --git a/go.sum b/go.sum
@@ -1,9 +1,11 @@
 github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/dubbogo/gost v1.5.1 h1:oG5dzaWf1KYynBaBoUIOkgT+YD0niHV6xxI0Odq7hDg=
-github.com/dubbogo/gost v1.5.1/go.mod h1:pPTjVyoJan3aPxBPNUX0ADkXjPibLo+/Ib0/fADXSG8=
+github.com/dubbogo/gost v1.9.0 h1:UT+dWwvLyJiDotxJERO75jB3Yxgsdy10KztR5ycxRAk=
+github.com/dubbogo/gost v1.9.0/go.mod h1:pPTjVyoJan3aPxBPNUX0ADkXjPibLo+/Ib0/fADXSG8=
 github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
 github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
+github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=

diff --git a/int_test.go b/int_test.go
@@ -18,10 +18,13 @@
 package hessian
 
 import (
-	"github.com/stretchr/testify/assert"
 	"testing"
 )
 
+import (
+	"github.com/stretchr/testify/assert"
+)
+
 func TestEncInt32Len1B(t *testing.T) {
 	var (
 		v   int32