Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated release scripts and instructions #62

Merged
merged 3 commits into from
Nov 7, 2024
+28 −59
Merged

Updated release scripts and instructions #62

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c3892e8
Updated release scripts and instructions
olivrlee Nov 6, 2024
8062c0b
Add in -Prelease flag to handle the release script versioning
olivrlee Nov 6, 2024
7ae2eeb
Update looker.md
olivrlee Nov 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion build.gradle.kts
View file
Original file line number Diff line number Diff line change
@@ -138,7 +138,7 @@ tasks.validateBeforeBuildingReleaseArtifacts {

val String.v: String get() = rootProject.extra["$this.version"] as String

val buildVersion = "calcite".v + releaseParams.snapshotSuffix
val buildVersion = if (project.gradle.startParameter.projectProperties["release"] == "true") { "calcite".v } else {"calcite".v + releaseParams.snapshotSuffix}

println("Building Apache Calcite $buildVersion")

4 changes: 2 additions & 2 deletions looker-release.sh
View file
Original file line number Diff line number Diff line change
@@ -87,8 +87,8 @@ calcite.version=$NEXT_VERSION
echo -e " This becomes the new tip of the looker branch." >&2
echo -e "- Prior fixups, if any..." >&2
echo -e "- All commits from upstream..." >&2
echo -e "\nIf everything looks good, you can publish to Nexus with this command:\n" >&2
echo -e " ./gradlew -Prelease -PskipSign publishAllPublicationsToLookerNexusRepository\n" >&2
echo -e "\nIf everything looks good, you can publish to Artifact Registry with this command:\n" >&2
echo -e " ./looker-upload-artifact-registry.sh\n" >&2
echo -e "And you can push the release tag and force-push the looker branch to looker-open-source with these commands:\n" >&2
echo -e " git push [email protected]:looker-open-source/calcite.git $NEXT_TAG"
echo -e " git push -f [email protected]:looker-open-source/calcite.git $LOOKER_COMMIT:looker"
18 changes: 9 additions & 9 deletions looker-snapshot.sh
View file
Original file line number Diff line number Diff line change
@@ -19,7 +19,7 @@
# $1 is the sub-project directory
# $2 is the artifact ID
# $3 is the version
function snapshot_upload {
function artifact_registry_snapshot_upload {
mvn deploy:deploy-file \
-DgroupId=org.apache.calcite \
-DartifactId="$2" \
@@ -28,16 +28,16 @@ function snapshot_upload {
-Dfile="./$1/build/libs/$2-$3.jar" \
-DgeneratePom=false \
-DpomFile="./$1/build/publications/$1/pom-default.xml" \
-DrepositoryId=nexus \
-Durl=https://nexusrepo.looker.com/repository/maven-snapshots/
-DrepositoryId=artifact-registry \
-Durl=https://us-maven.pkg.dev/prow-build-looker/looker-maven-snapshots
}

./gradlew build -x :redis:test && ./gradlew jar && ./gradlew generatePom && (
./gradlew build -x test && ./gradlew jar && ./gradlew generatePom && (
VERSION="$(sed -n 's/^calcite\.version=\([^ ]*\).*/\1/p' gradle.properties)-SNAPSHOT"
snapshot_upload core calcite-core "$VERSION"
snapshot_upload babel calcite-babel "$VERSION"
snapshot_upload linq4j calcite-linq4j "$VERSION"
snapshot_upload testkit calcite-testkit "$VERSION"
artifact_registry_snapshot_upload core calcite-core "$VERSION"
artifact_registry_snapshot_upload babel calcite-babel "$VERSION"
artifact_registry_snapshot_upload linq4j calcite-linq4j "$VERSION"
artifact_registry_snapshot_upload testkit calcite-testkit "$VERSION"
echo
echo "Done uploading version ${VERSION} to Looker Nexus Snapshots!"
echo "Done uploading version ${VERSION} to Looker Artifact Registry Snapshots!"
)
14 changes: 8 additions & 6 deletions looker-upload-artifact-registry.sh
View file
Original file line number Diff line number Diff line change
@@ -19,7 +19,9 @@
# $1 is the sub-project directory
# $2 is the artifact ID
# $3 is the version
function artifact_registry_upload {

# Note: Redeployment of the same versioned artifact (same name, repeated uploads) is prohibited.
function artifact_registry_release_upload {
mvn deploy:deploy-file \
-DgroupId=org.apache.calcite \
-DartifactId="$2" \
@@ -32,12 +34,12 @@ function artifact_registry_upload {
-Durl=https://us-maven.pkg.dev/prow-build-looker/looker-maven-private
}

./gradlew build -x :redis:test && ./gradlew jar && ./gradlew generatePom && (
./gradlew build -x :redis:test -Prelease=true && ./gradlew jar && ./gradlew generatePom && (
VERSION="$(sed -n 's/^calcite\.version=\([^ ]*\).*/\1/p' gradle.properties)"
artifact_registry_upload core calcite-core "$VERSION"
artifact_registry_upload babel calcite-babel "$VERSION"
artifact_registry_upload linq4j calcite-linq4j "$VERSION"
artifact_registry_upload testkit calcite-testkit "$VERSION"
artifact_registry_release_upload core calcite-core "$VERSION"
artifact_registry_release_upload babel calcite-babel "$VERSION"
artifact_registry_release_upload linq4j calcite-linq4j "$VERSION"
artifact_registry_release_upload testkit calcite-testkit "$VERSION"
echo
echo "Done uploading version ${VERSION} to Looker Artifact Registry!"
)
49 changes: 8 additions & 41 deletions looker.md
View file
Original file line number Diff line number Diff line change
@@ -27,8 +27,8 @@ Do not merge to Calcite's master branch.
*Read the instructions* in the `looker-release.sh` script,
but do not run that script until you're ready to publish a production release.

Looker has poor infrastructure for testing with local builds of Avatica.
The easiest way is to upload a snapshot version to Looker's Nexus repository and use it.
Looker has poor infrastructure for testing with local builds of Calcite.
The easiest way is to upload a snapshot version to Looker's Artifact Registry repository and use it.
To upload a snapshot version, simply run `./looker-snapshot.sh`,
which runs `./gradlew build` and, if successful,
uploads the resulting snapshot artifacts to the repo
@@ -39,48 +39,16 @@ and you're ready to build.

## Release

Define Looker's Nexus repository in your `~/.gradle/init.gradle.kts`
file:

```kotlin
allprojects {
plugins.withId("maven-publish") {
configure<PublishingExtension> {
repositories {
maven {
name = "lookerNexus"
val baseUrl = "https://nexusrepo.looker.com"
val releasesUrl = "$baseUrl/repository/maven-releases"
val snapshotsUrl = "$baseUrl/repository/maven-snapshots"
val release = !project.version.toString().endsWith("-SNAPSHOT")
// val release = project.hasProperty("release")
url = uri(if (release) releasesUrl else snapshotsUrl)
credentials {
username = "xxx"
password = "xxx"
}
}
}
}
}
}
```

In the above fragment, replace the values of the `username` and
`password` properties with the secret credentials.

*NOTE* This fragment *must* be in a file outside of your git sandbox.
If the file were in the git sandbox, it would be too easy to
accidentally commit the secret credentials and expose them on a
public site.

*Read the instructions* in the `looker-release.sh` script, then run it.
The script will only make local changes.
You'll have a chance to review them before pushing anything to Nexus or GitHub.
You'll have a chance to review them before pushing anything to Artifact Registry or GitHub.
Upload to Artifact Registry by running `looker-upload-artifact-registry.sh` after checking the release looks right.

Each release will have a name like `1.21.1-looker` (if the most
recent official Calcite release is `1.21`) and have a git tag
`calcite-1.21.1-looker`.
Each Looker release will have a name like `1.38.1-looker` (if the most
recent official Calcite release that it is based off of is `1.38`) and have a git tag
`calcite-1.38.1-looker`.

You should make it from a branch that differs from Calcite's
`master` branch in only minor ways:
@@ -101,5 +69,4 @@ You should make it from a branch that differs from Calcite's
so the version in trunk is generally decremented
while adding the `-looker` suffix.

Check the artifacts
[on Nexus](https://nexusproxy.looker.com/#browse/search=keyword%3Dorg.apache.calcite).
Check the artifacts on Artifact Registry.

Unchanged files with check annotations Beta

View file
core/src/main/java/org/apache/calcite/sql/SqlOperator.java
final List<RelDataType> operandTypes = opBinding.collectOperandTypes();
if (operandTypes.stream().anyMatch(t -> t.getSqlTypeName() == SqlTypeName.ANY)) {
final RelDataType[] operandTypes2 = operandTypes.toArray(new RelDataType[0]);
operandTypeInference.inferOperandTypes(callBinding, returnType, operandTypes2);

Check failure on line 555 in core/src/main/java/org/apache/calcite/sql/SqlOperator.java

GitHub Actions / CheckerFramework (JDK 11) 

[Task :core:compileJava] [argument.type.incompatible] incompatible argument for parameter returnType of inferOperandTypes.
          operandTypeInference.inferOperandTypes(callBinding, returnType, operandTypes2);
                                                              ^
  found   : @initialized @nullable RelDataType

Check failure on line 555 in core/src/main/java/org/apache/calcite/sql/SqlOperator.java

GitHub Actions / CheckerFramework (JDK 11), oldest Guava 

[Task :core:compileJava] [argument.type.incompatible] incompatible argument for parameter returnType of inferOperandTypes.
          operandTypeInference.inferOperandTypes(callBinding, returnType, operandTypes2);
                                                              ^
  found   : @initialized @nullable RelDataType
((SqlValidatorImpl) callBinding.getValidator())
.callToOperandTypesMap
.put(callBinding.getCall(), ImmutableList.copyOf(operandTypes2));
}
}
return returnType;

Check failure on line 562 in core/src/main/java/org/apache/calcite/sql/SqlOperator.java

GitHub Actions / CheckerFramework (JDK 11) 

[Task :core:compileJava] [return.type.incompatible] incompatible types in return.
      return returnType;
             ^
  type of expression: @initialized @nullable RelDataType

Check failure on line 562 in core/src/main/java/org/apache/calcite/sql/SqlOperator.java

GitHub Actions / CheckerFramework (JDK 11), oldest Guava 

[Task :core:compileJava] [return.type.incompatible] incompatible types in return.
      return returnType;
             ^
  type of expression: @initialized @nullable RelDataType
}
// Derived type should have overridden this method, since it didn't
View file
core/src/main/java/org/apache/calcite/jdbc/CalciteMetaColumnFactoryImpl.java
dataType,
typeName,
columnSize,
decimalDigits,

Check failure on line 60 in core/src/main/java/org/apache/calcite/jdbc/CalciteMetaColumnFactoryImpl.java

GitHub Actions / CheckerFramework (JDK 11) 

[Task :core:compileJava] [argument.type.incompatible] incompatible argument for parameter decimalDigits of MetaColumn.
        decimalDigits,
        ^
  found   : @initialized @nullable Integer

Check failure on line 60 in core/src/main/java/org/apache/calcite/jdbc/CalciteMetaColumnFactoryImpl.java

GitHub Actions / CheckerFramework (JDK 11), oldest Guava 

[Task :core:compileJava] [argument.type.incompatible] incompatible argument for parameter decimalDigits of MetaColumn.
        decimalDigits,
        ^
  found   : @initialized @nullable Integer
numPrecRadix,
nullable,
charOctetLength,