Merge remote-tracking branch 'origin/main' into pulsar-standalone

apache · Apr 2, 2024 · eb65686 · eb65686
2 parents fe1a683 + 96cdbf2
commit eb65686
Show file tree

Hide file tree

Showing 54 changed files with 162,312 additions and 182 deletions.
diff --git a/contribute/release-note-guide.md b/contribute/release-note-guide.md
@@ -49,7 +49,15 @@ Here are 2 approaches:
 Using "git log"
 
 ```bash
-git log --reverse  --oneline v2.11.3..v2.11.4 | colrm 1 12 | sed 's/\] \[/][/' | perl -p -e 's/^\s+//' | sort
+PREVIOUS_VERSION=3.0.3
+VERSION_WITHOUT_RC=3.0.4
+git log --reverse  --oneline v$PREVIOUS_VERSION..v$VERSION_WITHOUT_RC | colrm 1 12 | sed 's/\] \[/][/' | perl -p -e 's/^\s+//' | awk -F ']' '{
+    if ($1 ~ /^\[/) {
+        print $1 "]" $2, $0
+    } else {
+        print "[zzz]", $0
+    }
+}' | sort | cut -d ' ' -f2- | sed 's/\(#\([0-9]\+\)\)/[#\2](https:\/\/github.com\/apache\/pulsar\/pull\/\2)/g' | sed 's/^/- /'
 ```
 
 Alternatively using "gh pr list"

diff --git a/contribute/release-process.md b/contribute/release-process.md
@@ -41,7 +41,7 @@ Before you start the next release steps, make sure you have installed these soft
 Also, you need to **clean up the bookkeeper's local compiled** to make sure the bookkeeper dependency is fetched from the Maven repository, details to see [this mailing list thread](https://lists.apache.org/thread/gsbh95b2d9xtcg5fmtxpm9k9q6w68gd2).
 
 
-## Set environment variables to be used across the commands
+## Set environment variables to be used across the commands {#env-vars}
 
 Set version
 ```shell
@@ -520,18 +520,49 @@ The vote should be open for at least 72 hours (3 days). Votes from Pulsar PMC me
 
 If the release is approved here with 3 +1 binding votes, you can then proceed to the next step. Otherwise, you should repeat the previous steps and prepare another release candidate to vote.
 
+
+## Summarize the voting for the release
+
+Once the vote has been passed, you will need to send a result vote to [[email protected]](mailto:[email protected]) on the voting thread.
+
+Message:
+
+```shell
+tee >(pbcopy) <<EOF
+Hello all,
+
+The vote to release Apache Pulsar version ${VERSION_WITHOUT_RC} based on ${VERSION_RC} is now closed.
+
+The vote PASSED with X binding "+1", Y non-binding "+1" and 0 "-1" votes:
+
+"+1" Binding votes:
+
+  - <name>
+
+"+1" Non-Binding votes:
+
+  - <name>
+
+I'll continue with the release process and the release announcement will follow shortly.
+
+Thanks,
+<your name>
+EOF
+```
+
+
 ## Promote the release
 
-For commands below, you need to set the environment variables VERSION_RC, VERSION_WITHOUT_RC and APACHE_USER.
-Please check the previous step for doing that.
+For commands below, you need to set the environment variables `VERSION_RC`, `VERSION_WITHOUT_RC`, `UPSTREAM_REMOTE` and `APACHE_USER`.
+Please check the [environment variables step](#env-vars) for doing that.
 
 ### Publish the final tag
 
 Create and push the final Git tag:
 
 ```shell
-git tag -u $APACHE_USER@apache.org v$VERSION_WITHOUT_RC -m "Release v$VERSION_WITHOUT_RC"
-git push origin v$VERSION_WITHOUT_RC
+git tag -u $APACHE_USER@apache.org v$VERSION_WITHOUT_RC v$VERSION_RC^{} -m "Release v$VERSION_WITHOUT_RC"
+git push $UPSTREAM_REMOTE v$VERSION_WITHOUT_RC
 ```
 
 ### Create release notes in GitHub
@@ -541,6 +572,9 @@ Then, you can [create a GitHub release](https://docs.github.com/en/repositories/
 ```shell
 # open this URL and create release notes by clicking "Create release from tag"
 echo https://github.com/apache/pulsar/releases/tag/v${VERSION_WITHOUT_RC}
+
+# cherry-picked changes template
+echo "[Cherry-picked changes](https://github.com/apache/pulsar/pulls?q=is%3Apr+is%3Amerged+label%3Arelease%2F${VERSION_WITHOUT_RC}+label%3Acherry-picked%2F${VERSION_BRANCH}+sort%3Acreated-asc)"
 ```
 
 1. Open the above URL in a browser and create release notes by clicking "Create release from tag".
@@ -601,14 +635,6 @@ regctl image copy apachepulsar/pulsar:$VERSION_WITHOUT_RC apachepulsar/pulsar:la
 regctl image copy apachepulsar/pulsar-all:$VERSION_WITHOUT_RC apachepulsar/pulsar-all:latest
 ```
 
-### Update project version
-After the release process, you should bump the project version and append it with `-SNAPSHOT`.
-```
-./src/set-project-version.sh x.x.x-SNAPSHOT
-git add -u
-git commit -m "Bump version to next snapshot version"
-```
-
 ### Release Helm Chart
 
 :::caution
@@ -700,14 +726,24 @@ First, build swagger files from apache/pulsar repo at the released tag:
 
 ```shell
 mvn -ntp install -Pcore-modules,swagger,-main -DskipTests -DskipSourceReleaseAssembly=true -Dspotbugs.skip=true -Dlicense.skip=true
+PULSAR_PATH=$(pwd)
 ```
 
 Now, run the following script from the main branch of apache/pulsar-site repo:
 
 ```shell
 cd tools/pytools
 poetry install
-poetry run bin/rest-apidoc-generator.py --master-path=/path/to/pulsar-2.X.Y --version=2.X.Y
+poetry run bin/rest-apidoc-generator.py --master-path=$PULSAR_PATH --version=$VERSION_WITHOUT_RC
+```
+
+```shell
+# commit files
+# move to pulsar-site root
+cd ../..
+git add -u
+git add static/swagger/$VERSION_WITHOUT_RC
+git commit -m "update rest-apidoc for $VERSION_WITHOUT_RC"
 ```
 
 Read more on the manual of [pytools](https://github.com/apache/pulsar-site/tree/main/tools/pytools/README.md).
@@ -725,7 +761,7 @@ After publish Java libraries, run the following script from the main branch of a
 ```shell
 cd tools/pytools
 poetry install
-poetry run bin/java-apidoc-generator.py 2.X.0
+poetry run bin/java-apidoc-generator.py $VERSION_WITHOUT_RC
 ```
 
 Once the docs are generated, you can add them and submit them in a PR. The expected doc output is:
@@ -750,7 +786,7 @@ You can generate references of config and command-line tool by running the follo
 # build Pulsar distributions under /path/to/pulsar-2.X.0
 cd tools/pytools
 poetry install
-poetry run bin/reference-doc-generator.py --master-path=/path/to/pulsar-2.X.0 --version=2.X.0
+poetry run bin/reference-doc-generator.py --master-path=$PULSAR_PATH --version=$VERSION_WITHOUT_RC
 ```
 
 Once the docs are generated, you can add them and submit them in a PR. The expected doc output is `static/reference/2.X.x`
@@ -784,11 +820,12 @@ Otherwise, you should update the dropdown version list in this file: <https://gi
 
 Once the release artifacts are available in the Apache Mirrors and the website is updated, you need to announce the release. You should email to [email protected], [email protected], and [email protected]. Here is a sample content:
 
-```
+```shell
+tee >(pbcopy) <<EOF
 To: [email protected], [email protected], [email protected]
-Subject: [ANNOUNCE] Apache Pulsar 2.X.0 released
+Subject: [ANNOUNCE] Apache Pulsar $VERSION_WITHOUT_RC released
 
-The Apache Pulsar team is proud to announce Apache Pulsar version 2.X.0.
+The Apache Pulsar team is proud to announce Apache Pulsar version $VERSION_WITHOUT_RC.
 
 Pulsar is a highly scalable, low latency messaging platform running on
 commodity hardware. It provides simple pub-sub semantics over topics,
@@ -800,13 +837,14 @@ For Pulsar release details and downloads, visit:
 https://pulsar.apache.org/download
 
 Release Notes are at:
-https://pulsar.apache.org/release-notes
+https://pulsar.apache.org/release-notes/versioned/pulsar-$VERSION_WITHOUT_RC/
 
 We would like to thank the contributors that made the release possible.
 
 Regards,
 
 The Pulsar Team
+EOF
 ```
 
 Send the email in plain text mode since the [email protected] mailing list will reject messages with text/html content.
@@ -828,23 +866,29 @@ Remove the old releases (if any). You only need the latest release there, and ol
 svn ls https://dist.apache.org/repos/dist/release/pulsar
 
 # Delete each release (except for the last one)
-svn rm https://dist.apache.org/repos/dist/release/pulsar/pulsar-2.Y.0
+svn rm https://dist.apache.org/repos/dist/release/pulsar/pulsar-3.X.X
 ```
 
-## Move master branch to next version
-
-:::caution
-
-This step is for feature releases only.
+## Move to next version in pom.xml
 
-:::
+### Feature releases (master branch)
 
 You need to move the master version to the next iteration `Y` (`X + 1`).
 
 ```shell
 git checkout master
-./src/set-project-version.sh 2.Y.0-SNAPSHOT
-git commit -a -s -m "[cleanup][build] Bumped version to 2.Y.0-SNAPSHOT'
+./src/set-project-version.sh 3.Y.0-SNAPSHOT
+git commit -a -s -m "[cleanup][build] Bumped version to 3.Y.0-SNAPSHOT'
 ```
 
 Since this needs to be merged into `master`, you need to follow the regular process and create a Pull Request on GitHub.
+
+### For maintenance branches
+
+After the release process, you should bump the project version and append it with `-SNAPSHOT`.
+
+```shell
+./src/set-project-version.sh x.x.x-SNAPSHOT
+git add -u
+git commit -m "Bump version to next snapshot version"
+```
diff --git a/data/release-cpp.js b/data/release-cpp.js
diff --git a/data/release-java.js b/data/release-java.js
diff --git a/data/release-pulsar.js b/data/release-pulsar.js
@@ -1,12 +1,21 @@
 module.exports = [
+  {
+    "author": "lhotari",
+    "tagName": "v3.2.2",
+    "publishedAt": "2024-04-02T06:08:51Z",
+    "vtag": "3.2.x",
+    "releaseNotes": "/release-notes/versioned/pulsar-3.2.2/",
+    "doc": "/docs/3.2.x",
+    "version": "v3.2.x"
+  },
   {
     "author": "Technoboy-",
     "tagName": "v3.2.1",
     "publishedAt": "2024-03-08T21:11:40Z",
     "vtag": "3.2.x",
     "releaseNotes": "/release-notes/versioned/pulsar-3.2.1/",
     "doc": "/docs/3.2.x",
-    "version": "v3.2.x"
+    "version": ""
   },
     {
     "author": "Technoboy-",
@@ -53,14 +62,23 @@ module.exports = [
         "doc": "/docs/3.1.x",
         "version": ""
     },
+    {
+      "author": "lhotari",
+      "tagName": "v3.0.4",
+      "publishedAt": "2024-04-02T04:56:15Z",
+      "vtag": "3.0.x",
+      "releaseNotes": "/release-notes/versioned/pulsar-3.0.4/",
+      "doc": "/docs/3.0.x",
+      "version": "v3.0.x"
+    },
     {
       "author": "heesung-sn",
       "tagName": "v3.0.3",
       "publishedAt": "2024-03-09T16:53:02Z",
       "vtag": "3.0.x",
       "releaseNotes": "/release-notes/versioned/pulsar-3.0.3/",
       "doc": "/docs/3.0.x",
-      "version": "v3.0.x"
+      "version": ""
     },
     {
       "author": "poorbarcode",

diff --git a/docs/administration-stats.md b/docs/administration-stats.md
@@ -22,8 +22,8 @@ All stats below are **reset** to 0 upon broker restart or topic unloading, **exc
 | msgRateOut                                       | The sum of all local and replication consumers' dispatch rates (message per second).                                                                                                                                                            |
 | msgThroughputOut                                 | The sum of all local and replication consumers' dispatch rates (byte per second).                                                                                                                                                               |
 | averageMsgSize                                   | The average size (bytes) of messages published within the last interval.                                                                                                                                                                        |
-| storageSize*                                     | The sum of the ledgers' storage size **in BookKeeper** for a topic (in bytes). <br/><br/>**Note**: the `total storage size of a topic` = `storageSize` + `offloadedStorageSize`.                                                                |
-| offloadedStorageSize*                            | The sum of the storage size **in tiered storage** for a topic (in bytes).<br/><br/>**Note**: the `total storage size of a topic` = `storageSize` + `offloadedStorageSize`.                                                                      |
+| storageSize*                                     | The sum of the ledgers' storage size **in BookKeeper** and **in tiered storage** for a topic (in bytes).                                                                                                                                        |
+| offloadedStorageSize*                            | The sum of the storage size **in tiered storage** for a topic (in bytes).<br/><br/>**Note**: the `total storage size of a topic` = `storageSize`, includes `offloadedStorageSize`.                                                              |
 | earliestMsgPublishTimeInBacklogs*                | The publish time of the earliest message in the backlog (in milliseconds).                                                                                                                                                                      |
 | bytesInCounter                                   | The total bytes published to the topic.                                                                                                                                                                                                         |
 | msgInCounter                                     | The total messages published to the topic.                                                                                                                                                                                                      |

diff --git a/docs/client-libraries-java-setup.md b/docs/client-libraries-java-setup.md
@@ -42,10 +42,53 @@ If you use Gradle, add the following information to the `build.gradle` file.
 def pulsarVersion = '@pulsar:version@'
 
 dependencies {
-    compile group: 'org.apache.pulsar', name: 'pulsar-client', version: pulsarVersion
+	implementation "org.apache.pulsar:pulsar-client:${pulsarVersion}"
 }
 ```
 
+### Pulsar BOM
+
+While the above dependencies are sufficient to obtain the Pulsar Java client, it is recommended to also use the [Pulsar BOM](https://github.com/apache/pulsar/blob/master/pip/pip-326.md) to ensure that all Pulsar dependencies (both direct and transitive) are at the same expected version.
+In order to use the BOM, the previous directions are modified slightly as follows:
+
+#### Maven
+
+If you use Maven, add the following information to the `pom.xml` file.
+
+```xml
+<!-- in your <properties> block -->
+<pulsar.version>@pulsar:version@</pulsar.version>
+
+<!-- in your <dependencyManagement>/<dependencies> block -->
+<dependency>
+  <groupId>org.apache.pulsar</groupId>
+  <artifactId>pulsar-bom</artifactId>
+  <version>${pulsar.version}</version>
+  <type>pom</type>
+  <scope>import</scope>
+</dependency>
+
+<!-- in your <dependencies> block -->
+<dependency>
+  <groupId>org.apache.pulsar</groupId>
+  <artifactId>pulsar-client</artifactId>
+</dependency>
+```
+
+#### Gradle
+
+If you use Gradle, add the following information to the `build.gradle` file.
+
+```groovy
+def pulsarVersion = '@pulsar:version@'
+
+dependencies {
+  implementation enforcedPlatform("org.apache.pulsar:pulsar-bom:${pulsarVersion}")
+  implementation 'org.apache.pulsar:pulsar-client'
+}
+```
+Note that the version is number for the `pulsar-client` dependency is now omitted as the Pulsar BOM dictates which version is used.
+
 ## Step 2: Connect to Pulsar cluster
 
 To connect to Pulsar using client libraries, you need to specify a [Pulsar protocol](developing-binary-protocol.md) URL.
@@ -66,4 +109,4 @@ If you use [mTLS](security-tls-authentication.md) authentication, add `+ssl` in
 
 ```http
 pulsar+ssl://pulsar.us-west.example.com:6651
-```
+```