Merge Zipkin v3 to the main branch. #3594

wu-sheng · 2023-11-15T09:00:19Z

We are planning to run a 3.0.0-rc0 release as we have finished all features covered in v3.\

The key features are as follows

Newly support for all versions of Elasticsearch 8 and OpenSearch 1.x. And keep supports for Elasticsearch 6 and 7.
Rapid support for the latest JDK releases, and fix for CVEs of dependencies.
Refactored storage structures and query logic for Elasticsearch to improve performance. Elasticsearch. Reduce 20% disk volume and 40% query latency on the 2 million spans case.
SQL Databases(MySQL and PostgreSQL) storage options are ready and stable for the product ready in the medium-scale deployment, or larger-scale when picking SQL Database mutations, such as Amazon Aurora.
Introduce cluster mode deployment in Zipkin v3. It reduces the memory requirements for the caches of service names, tags, and local/remote endpoints. It doesn’t require an extra deployment of Elastic Curator for Zipkin data time-to-live(TTL) in the Elasticsearch storage. SQL Databases(MySQL and PostgreSQL with their mutations) storage options support TTL for the first time.
Support for Apache BanyanDB (Alpha stage) as a new storage option, which is expected to offer better compression rates, lesser memory usage, and less CPU consumption compared to Elasticsearch data capacity(large-scale deployment) scenarios.
Automatic conversion of OpenTelemetry trace format into Zipkin via gRPC and storing it as Zipkin native spans, eliminating the need for an OpenTelemetry Collector relay.
A more powerful standalone memory mode, replacing the previous simple memory structure with H2 database, and supporting H2 DB Server for lightweight deployment.
Enable self-observability of the Zipkin server, trace querying, and span reporting by binding them to different HTTP IP addresses and ports. It provides more flexibility in a complex and security-sensitive environment.

* Support query dependency

)

…3586)

jcchavezs · 2023-11-15T09:18:29Z

docker/Dockerfile

@@ -79,7 +79,7 @@ LABEL org.opencontainers.image.description="Zipkin slim distribution on OpenJDK

 COPY --from=install --chown=${USER} /install/zipkin-slim/ /zipkin/

-EXPOSE 9411
+EXPOSE 9411 9412


Curious about this.

I will delete it.

jcchavezs · 2023-11-15T09:18:48Z

build-bin/maven/maven_release

@@ -40,4 +41,4 @@ if [ "$commit_local_release_branch" != "$commit_remote_release_branch" ]; then
 fi

 # Prepare and push release commits and the version tag (N.N.N), which triggers deployment.
-./mvnw --batch-mode -nsu -DreleaseVersion=${release_version} -Denforcer.fail=false -Darguments="-DskipTests -Denforcer.fail=false" release:prepare
+./mvnw --batch-mode -nsu -DreleaseVersion=${release_version} -Denforcer.fail=false -Darguments="-DskipTests -Denforcer.fail=false -Dcheckstyle.skip=true" release:prepare


Curious about this, are we skipping the check style?

At present, zipkin does not include checkstyle. However, the newly introduced skywalking will search for and utilize the checkstyle configuration from the current project. So, I am currently disabling skywalking's checkstyle execution by deactivating it.

jcchavezs · 2023-11-15T09:21:40Z

docker/README.md

@@ -38,8 +38,8 @@ We also provide [example compose files](examples/README.md) that integrate colle
 such as Kafka or Elasticsearch.

 ## Configuration
-Configuration is via environment variables, defined by [zipkin-server](https://github.com/openzipkin/zipkin/blob/master/zipkin-server/README.md). Notably, you'll want to look at the `STORAGE_TYPE` environment variables, which
-include "cassandra", "mysql" and "elasticsearch".
+Configuration is via environment variables, defined by [zipkin-server](https://github.com/openzipkin/zipkin/blob/master/zipkin-server/README.md). Notably, you'll want to look at the `ZIPKIN_STORAGE` environment variables, which


While I don't question the naming change I want to make sure we support BOTH to ease migration.

@mrproliu We could change this back. I think.

Got it, I will make these environments both work.

jcchavezs · 2023-11-15T09:21:54Z

docker/README.md

@@ -69,7 +69,7 @@ For example, to increase heap size, set `JAVA_OPTS` as shown in our [docker-comp

 For example, to add debug logging, set `command` as shown in our [docker-compose](examples/docker-compose.yml) file:
 ```yaml
-    command: --logging.level.zipkin2=DEBUG
+    command: -Dlog.level=DEBUG


Can we support previous option too?

This seems a little hard. log4j V.S. log4js. Unless we hard coded some hacking codes.

jcchavezs · 2023-11-15T09:25:05Z

Changes look OK to me but I will wait for others to review.

The only thing that catched my eye strongly was the env vars renaming. While this makes sense in some cases that I could identify, we must make sure migration isn't that much of a pain to users (that will have to drop their existing database anyways) and make sure old zipkin config remains working. One of the most important principles in entire zipkin ecosystem was to not to break sites even on major version changes, instead deprecate them in a major and remove them in the next major.

basvanbeek · 2023-11-15T09:33:19Z

So let's not rush to land this in main branch. I'd rather do release candidates with some real effort of reaching out to community and site owners (including a serious mention about this on our website).

When we've had sufficient positive feedback cycles, concerns dealt with, enough broadcasting to reach our users we can talk about planning an official release and moving to main branch.

basvanbeek

Let's give this more time and more communication around this effort. We need actual users to try and respond before an official release.

This also means providing notice and broadcasting about this effort on website, gitter, google groups, etc. As well as providing RC builds to be downloadable from site, docker hub, etc.

wu-sheng · 2023-11-15T09:36:57Z

So let's not rush to land this in main branch. I'd rather do release candidates with some real effort of reaching out to community and site owners (including a serious mention about this on our website).

So, is it fine to release one from a branch? If this is OK, we can work on that way.

basvanbeek · 2023-11-15T09:41:44Z

So let's not rush to land this in main branch. I'd rather do release candidates with some real effort of reaching out to community and site owners (including a serious mention about this on our website).

So, is it fine to release one from a branch? If this is OK, we can work on that way.

IMHO it is fine to do release candidates / tech previews from a branch. Not sure how much work it will be to make it available. I just don't feel it's right to merge to main branch prior to the broader community approval.

I want this effort to be correct and supported by the broader community. The impact is too large to swiftly go through without giving more time and consideration.

wu-sheng · 2023-11-15T09:45:08Z

IMHO it is fine to do release candidates / tech previews from a branch. Not sure how much work it will be to make it available.

OK, let me do this. (Release RCs from the branch)

jcchavezs · 2023-11-15T10:13:54Z

Once you can release an image we could test the automation in https://github.com/openzipkin/zipkin-php-example as smoke test first. José Carlos Chávez ons. 15. nov. 2023 kl. 10:45 skrev 吴晟 Wu Sheng ***@***.***>:

…

Closed #3594 <#3594>. — Reply to this email directly, view it on GitHub <#3594 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXOYAUZOWYXLVMZGJ6AR7DYESFLPAVCNFSM6AAAAAA7MEH5RSVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJQHE3DKMJYGQYTIMY> . You are receiving this because your review was requested.Message ID: ***@***.***>

mrproliu added 22 commits September 7, 2023 08:40

Initializing to use SkyWalking to replace Zipkin Server in v3 (#3564)

47e6428

Support multiple receivers in v3 server (#3567)

904ee42

Fix the v3 server CI tests (#3569)

ffbffd0

Support cluster plugins for Cluster aware and entity scan (#3571)

8e02e6e

Adding gRPC server for cluster mode (#3572)

3fc0a7e

Adaptation enhancements for various features in v3 (#3573)

a6f354c

Adapt the endpoints in the server (#3574)

1c74d42

Fix annotation scan not working in v3 server (#3575)

e3fb5c6

Adaptation of Dependency Query Service in Server (#3576)

f4d1a36

* Support query dependency

Support update the default log level when startup (#3577)

4028c15

Update the mysql driver for more friendlier license (#3578)

fa681ee

Support search enable configuration (#3579)

b34a84e

Support Cassandra Storage in v3 (#3570)

04feb90

Reduce the execute duration of CI (#3581)

d5d4901

Support collect OTLP traces in v3 server (#3580)

3c23205

Enhance the performance and storage spze of the cassandra storage (#3584

44b8c06

)

Support collect tracing through gRPC (#3585)

726e377

Sync skywalking upstream version and reduce Cassandra storage size (#…

29a392f

…3586)

Adapt docker images for v3 server (#3587)

1840acf

Fix license header and add header checker in GHA (#3592)

a9df0ef

Update documentation and remove unesessary configuration (#3591)

062d493

Update release version and update CI GHA (#3593)

39025e3

wu-sheng added this to the 3.0.0 milestone Nov 15, 2023

wu-sheng requested a review from jcchavezs November 15, 2023 09:00

jcchavezs reviewed Nov 15, 2023

View reviewed changes

jcchavezs requested a review from llinder November 15, 2023 09:25

jcchavezs requested review from zeagord, jeqo, basvanbeek and shakuzen November 15, 2023 09:25

basvanbeek requested changes Nov 15, 2023

View reviewed changes

wu-sheng closed this Nov 15, 2023

codefromthecrypt mentioned this pull request Dec 1, 2023

Zipkin v3: the more maintenance friendly version #3605

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge Zipkin v3 to the main branch. #3594

Merge Zipkin v3 to the main branch. #3594

wu-sheng commented Nov 15, 2023

jcchavezs Nov 15, 2023

mrproliu Nov 15, 2023

jcchavezs Nov 15, 2023

mrproliu Nov 15, 2023

jcchavezs Nov 15, 2023

wu-sheng Nov 15, 2023

mrproliu Nov 15, 2023

jcchavezs Nov 15, 2023

wu-sheng Nov 15, 2023

jcchavezs commented Nov 15, 2023 •

edited

Loading

basvanbeek commented Nov 15, 2023

basvanbeek left a comment

wu-sheng commented Nov 15, 2023

basvanbeek commented Nov 15, 2023

wu-sheng commented Nov 15, 2023

jcchavezs commented Nov 15, 2023 via email

Merge Zipkin v3 to the main branch. #3594

Merge Zipkin v3 to the main branch. #3594

Conversation

wu-sheng commented Nov 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jcchavezs commented Nov 15, 2023 • edited Loading

basvanbeek commented Nov 15, 2023

basvanbeek left a comment

Choose a reason for hiding this comment

wu-sheng commented Nov 15, 2023

basvanbeek commented Nov 15, 2023

wu-sheng commented Nov 15, 2023

jcchavezs commented Nov 15, 2023 via email

jcchavezs commented Nov 15, 2023 •

edited

Loading