Knative Knative Gcp release v0.18.2

Release notes for release-0.18

Documentation

Changelog since v0.17.0

Changes by Kind

Action Required

• V1alpha1 sources and channels will be removed in 0.19. (#1676, @danyinggu)

Breaking Changes

• Change storage versions of all below resources from v1beta1 to v1
• Since we will remove v1alpha1 in 0.19, please make sure resources are migrated to storage version v1beta1

Action Required

• You must run pre-install job. (#1653, @danyinggu)

• You must run pre-install job prior to upgrading to get these resources to v1 API.
  ko apply -f config/pre-install/v0.18.0/ (#1644, @danyinggu)

Detailed Changes

• Change storage versions of all below resources from v1beta1 to v1:
  - "cloudauditlogssources.events.cloud.google.com"
  - "cloudpubsubsources.events.cloud.google.com"
  - "cloudstoragesources.events.cloud.google.com"
  - "cloudschedulersources.events.cloud.google.com"
  - "pullsubscriptions.internal.events.cloud.google.com"
  - "topics.internal.events.cloud.google.com"

• Since we will remove v1alpha1 in 0.19, please make sure the below resources are migrated to storage version v1beta1:
  - "channels.messaging.cloud.google.com"
  - "cloudbuildsources.events.cloud.google.com"

Resource Usage

• Default ingress CPU to 2000m and memory to 2000Mi
  • Adjust default HPA parameters accordingly
  • Default pubsub publish buffer limit to 300Mi
  • Loosen probe check threshold to 5 times (#1656, @yolocs)
• Update broker deployments spec for better availability during upgrades
  • Add MinReadySeconds=60s for broker deployments
  • Add a safer rolling update strategy
  • Relax probe period (#1690, @yolocs)

Reliability

• 🐛 Fix broker config reconcilation delay. (#1705, @ian-mi)
• 🐛 Fix bug status messag showing error (#1675, @zhongduo)
• Allows istio sidecar injection for all broker components (#1736, @yolocs)

Observability

• 🎁 Add exemplar traces to dispatch time and processing time, the exemplar can be viewed in Stackdriver dashboard in heatmap view. (#1638, @zhongduo)
• Provide additional error info when auth with pubsub fails in broker ingress. (#1666, @cathyzhyi)
• Return 429 (too many messages) when the ingress broker is sending too many messages, instead of 500 (internal server error). (#1737, @tayarani)
• 🎁 Add data residency support through configuration map. (#1681, @zhongduo)

Sources

• Add v1 API for CloudBuildSource. (#1657, @danyinggu)
• CloudPubSubSource now populates the dataschema CloudEvent attribute. The value is: https://raw.githubusercontent.com/googleapis/google-cloudevents/master/proto/google/events/cloud/pubsub/v1/data.proto (#1667, @nachocano)
• The CloudBuildSource reconciler is now using v1 CloudBuildSource API. (#1662, @danyinggu)

Knative Knative Gcp release v0.18.1

Release notes for release-0.18

Documentation

Changelog since v0.17.0

Changes by Kind

Action Required

• V1alpha1 sources and channels will be removed in 0.19. (#1676, @danyinggu)

Breaking Changes

• Change storage versions of all below resources from v1beta1 to v1
• Since we will remove v1alpha1 in 0.19, please make sure resources are migrated to storage version v1beta1

Action Required

• You must run pre-install job. (#1653, @danyinggu)

• You must run pre-install job prior to upgrading to get these resources to v1 API.
  ko apply -f config/pre-install/v0.18.0/ (#1644, @danyinggu)

Detailed Changes

• Change storage versions of all below resources from v1beta1 to v1:
  - "cloudauditlogssources.events.cloud.google.com"
  - "cloudpubsubsources.events.cloud.google.com"
  - "cloudstoragesources.events.cloud.google.com"
  - "cloudschedulersources.events.cloud.google.com"
  - "pullsubscriptions.internal.events.cloud.google.com"
  - "topics.internal.events.cloud.google.com"

• Since we will remove v1alpha1 in 0.19, please make sure the below resources are migrated to storage version v1beta1:
  - "channels.messaging.cloud.google.com"
  - "cloudbuildsources.events.cloud.google.com"

Resource Usage

• Default ingress CPU to 2000m and memory to 2000Mi
  • Adjust default HPA parameters accordingly
  • Default pubsub publish buffer limit to 300Mi
  • Loosen probe check threshold to 5 times (#1656, @yolocs)
• Update broker deployments spec for better availability during upgrades
  • Add MinReadySeconds=60s for broker deployments
  • Add a safer rolling update strategy
  • Relax probe period (#1690, @yolocs)

Reliability

• 🐛 Fix broker config reconcilation delay. (#1705, @ian-mi)
• 🐛 Fix bug status messag showing error (#1675, @zhongduo)
• Allows istio sidecar injection for all broker components (#1736, @yolocs)

Observability

• 🎁 Add exemplar traces to dispatch time and processing time, the exemplar can be viewed in Stackdriver dashboard in heatmap view. (#1638, @zhongduo)
• Provide additional error info when auth with pubsub fails in broker ingress. (#1666, @cathyzhyi)
• Return 429 (too many messages) when the ingress broker is sending too many messages, instead of 500 (internal server error). (#1737, @tayarani)
• 🎁 Add data residency support through configuration map. (#1681, @zhongduo)

Sources

• Add v1 API for CloudBuildSource. (#1657, @danyinggu)
• CloudPubSubSource now populates the dataschema CloudEvent attribute. The value is: https://raw.githubusercontent.com/googleapis/google-cloudevents/master/proto/google/events/cloud/pubsub/v1/data.proto (#1667, @nachocano)
• The CloudBuildSource reconciler is now using v1 CloudBuildSource API. (#1662, @danyinggu)

Knative Knative Gcp release v0.19.1

Release notes for release-0.19

Documentation

Changelog since v0.18.0

Changes by Kind

Action Required

Breaking Changes

• Dropped v1alpha1 support for the following resources:

  • Channel (#1853, @eclipselu)

• Dropped v1alpha1 support for the following resources:

  • CloudPubSubSource
  • CloudStorageSource
  • CloudSchedulerSource
  • CloudAuditLogsSource
  • CloudBuildSource
  • Topic
  • PullSubscription (#1858, @eclipselu)

• Removed v1alpha1 CRD api versions for following resources:

  • CloudPubSubSource
  • CloudStorageSource
  • CloudSchedulerSource
  • CloudAuditLogsSource
  • CloudBuildSource
  • Topic
  • PullSubscription
  • Channel (#1861, @eclipselu)

Action Required

• Change the storage version of "cloudbuildsources.events.cloud.google.com" from v1beta1 to v1:

You must run pre-install job prior to upgrading to get these resources to v1 API.
ko apply -f config/pre-install/v0.19.0/ (#1797, @eclipselu)

Detailed Changes

Upgrade Notes

You must run pre-install job prior to upgrading to get these resources to v1 API.
ko apply -f config/pre-install/v0.19.0/ (#1797, @eclipselu)

Authorization

• Infer project ID from environmental variable in all sources and brokers (#1799, @zhongduo)

Resource Usage

• Adjust default brokercell parameters to match latest performance target (#1828, @yolocs)

Reliability

Observability

• Invalid Broker Ingress events now produce the event_count metric with event type invalid_cloud_event. (#1798, @AlexandraRoatis)

Sources

• Update pubsub dependency to 1.8.0 (#1782, @zhongduo)

Knative Knative Gcp release v0.17.6

Notices

• V1alpha1 resources (Source, Topic, PullSubscription and Channel) have been deprecated and will be removed in 0.19.
  Please use v1 (v1beta1 for Channel) instead. (#1562, @danyinggu)

New Features

• Add back-off in IAMPolicyManager (#1461, @grac3gao)
• Add timeout delivery metrics to broker fanout and retry (#1578, @grac3gao)

Other Changes

• Change v1beta1 channel to use duckv1 identifiable
  • Change v1beta1 cloudbuildsource to use duckv1 identifiable and duckv1.PubSubable (#1505, @capri-xiyue)

Removed Features

• Explicitly disallow istio sidecar injection for controller/webhook/broker fanout/retry (#1476, @yolocs)
• Remove spec.payloadFormat from the v1 type. (#1433, @danyinggu)
• IdentityStatus.ServiceAccountName is removed. PullSubscriptionSpec.Mode is removed. (#1441, @danyinggu)

Bug Fixes

• Malformed replies will be treated as error (instead of being silently ignored). (#1419, @yolocs)

Knative Knative Gcp release v0.19.0

Release notes for release-0.19

Documentation

Changelog since v0.18.0

Changes by Kind

Action Required

Breaking Changes

• Dropped v1alpha1 support for the following resources:

  • Channel (#1853, @eclipselu)

• Dropped v1alpha1 support for the following resources:

  • CloudPubSubSource
  • CloudStorageSource
  • CloudSchedulerSource
  • CloudAuditLogsSource
  • CloudBuildSource
  • Topic
  • PullSubscription (#1858, @eclipselu)

• Removed v1alpha1 CRD api versions for following resources:

  • CloudPubSubSource
  • CloudStorageSource
  • CloudSchedulerSource
  • CloudAuditLogsSource
  • CloudBuildSource
  • Topic
  • PullSubscription
  • Channel (#1861, @eclipselu)

Action Required

• Change the storage version of "cloudbuildsources.events.cloud.google.com" from v1beta1 to v1:

You must run pre-install job prior to upgrading to get these resources to v1 API.
ko apply -f config/pre-install/v0.19.0/ (#1797, @eclipselu)

Detailed Changes

Upgrade Notes

You must run pre-install job prior to upgrading to get these resources to v1 API.
ko apply -f config/pre-install/v0.19.0/ (#1797, @eclipselu)

Authorization

• Infer project ID from environmental variable in all sources and brokers (#1799, @zhongduo)

Resource Usage

• Adjust default brokercell parameters to match latest performance target (#1828, @yolocs)

Reliability

Observability

• Invalid Broker Ingress events now produce the event_count metric with event type invalid_cloud_event. (#1798, @AlexandraRoatis)

Sources

• Update pubsub dependency to 1.8.0 (#1782, @zhongduo)

Knative Knative Gcp release v0.17.5

Notices

• V1alpha1 resources (Source, Topic, PullSubscription and Channel) have been deprecated and will be removed in 0.19.
  Please use v1 (v1beta1 for Channel) instead. (#1562, @danyinggu)

New Features

• Add back-off in IAMPolicyManager (#1461, @grac3gao)
• Add timeout delivery metrics to broker fanout and retry (#1578, @grac3gao)

Other Changes

• Change v1beta1 channel to use duckv1 identifiable
  • Change v1beta1 cloudbuildsource to use duckv1 identifiable and duckv1.PubSubable (#1505, @capri-xiyue)

Removed Features

• Explicitly disallow istio sidecar injection for controller/webhook/broker fanout/retry (#1476, @yolocs)
• Remove spec.payloadFormat from the v1 type. (#1433, @danyinggu)
• IdentityStatus.ServiceAccountName is removed. PullSubscriptionSpec.Mode is removed. (#1441, @danyinggu)

Bug Fixes

• Malformed replies will be treated as error (instead of being silently ignored). (#1419, @yolocs)

Knative Knative Gcp release v0.17.4

Notices

• V1alpha1 resources (Source, Topic, PullSubscription and Channel) have been deprecated and will be removed in 0.19.
  Please use v1 (v1beta1 for Channel) instead. (#1562, @danyinggu)

New Features

• Add back-off in IAMPolicyManager (#1461, @grac3gao)
• Add timeout delivery metrics to broker fanout and retry (#1578, @grac3gao)

Other Changes

• Change v1beta1 channel to use duckv1 identifiable
  • Change v1beta1 cloudbuildsource to use duckv1 identifiable and duckv1.PubSubable (#1505, @capri-xiyue)

Removed Features

• Explicitly disallow istio sidecar injection for controller/webhook/broker fanout/retry (#1476, @yolocs)
• Remove spec.payloadFormat from the v1 type. (#1433, @danyinggu)
• IdentityStatus.ServiceAccountName is removed. PullSubscriptionSpec.Mode is removed. (#1441, @danyinggu)

Bug Fixes

• Malformed replies will be treated as error (instead of being silently ignored). (#1419, @yolocs)

Knative Knative Gcp release v0.16.3

Release notes for 0.16.1

The minimum supported Kubernetes version is now 0.16. (#1156, @Harwayne)

Changes by Kind

Action Required

• Added upgrade job/yaml that deletes legacy {pullsubscription,topic}.pubsub.cloud.google.com COs
  Before updating to 0.16, run the upgrade Job. Only after that Job has succeeded, then update to 0.16. Failure to run the update Job first, may leak the resources associated with the Kubernetes resources that were to be deleted, namely Topics and PullSubscriptions in GCP. (#1383, @yolocs)
• Changes to the Events sent by CloudAuditLogsSource, CloudSchedulerSource, and CloudStorageSource. See the Breaking Changes section for more information on what changes occurred and what actions are required. (#1340, @yolocs)

Breaking Changes

• All event types are updated to match event types in googleapis/google-cloudevents.
• CloudAuditLogs/Storage/Scheduler events have changed their CE source and subject attributes.
• Scheduler events have a new data schema.

Action Required

• Users who only depend on source, type, and subject attributes in the Trigger filters must create new Triggers with the new filter values (see details below).
• Users who depend on source, type, and subject attributes in their receiver code must update their code.
• Users who depend on Scheduler event data must update their code.

Detailed Changes

1. All event types and data schemas were updated to be compliant with https://github.com/googleapis/google-cloudevents/tree/master/proto/google/events/cloud. There are no substantive changes to the CloudAuditLogs/PubSub/Storage event schemas. Scheduler event schema was updated to https://github.com/googleapis/google-cloudevents/blob/master/proto/google/events/cloud/scheduler/v1/data.proto.
2. Source changes:
   • CloudAuditLogs: was //[service_name]/projects/[project_id] => now //cloudaudit.googleapis.com/projects/[project_id]/logs/[activity/data_access]
   • Scheduler: was //cloudscheduler.googleapis.com/projects/[project_id]/locations/[location]/schedulers/[k8s_scheduler_source_name] => now //cloudscheduler.googleapis.com/projects/[project_id]/locations/[location]/jobs/[job_name]
   • Storage: was //storage.googleapis.com/buckets/[bucket_name] => now //storage.googleapis.com/projects/_/buckets/[bucket_name]
3. Subject changes:
   • CloudAuditLogs: was [resource_name] => now [service_name]/[resource_name]
   • Scheduler: was jobs/[job_name] => now NONE (removed)
   • Storage: was [object_id] => now objects/[object_id]
4. Type changes:
   • CloudAuditLogs: was com.google.cloud.auditlog.event => now google.cloud.audit.log.v1.written
   • PubSub: was com.google.cloud.pubsub.topic.publish => now google.cloud.pubsub.topic.v1.messagePublished
   • Scheduler: was com.google.cloud.scheduler.job.execute => now google.cloud.scheduler.job.v1.executed
   • Storage:
     • Was com.google.cloud.storage.object.finalize => now google.cloud.storage.object.v1.finalized
     • Was com.google.cloud.storage.object.delete => now google.cloud.storage.object.v1.deleted
     • Was com.google.cloud.storage.object.archive => now google.cloud.storage.object.v1.archived
     • Was com.google.cloud.storage.object.metadataUpdate => now google.cloud.storage.object.v1.metadataUpdated

Upgrade Notes

• We have changed the naming convention we use for GCP resources in order to make it easy for users to understand what created them (e.g., a particular source or channel). The convention is cre-<owner_type>_<namespace>_<name>_<uid>. For example if a Source mysource in the namespace default with uid 47163a creates a Pub/Sub subscription, then the subscription will be called cre-src_default_mysource_47163a. (#1207, @nachocano)
• Given that we have renamed many resources in #1207, when upgrading to 0.16 we will delete those old resources and re-create them with the newer names. You may experience some delay in getting your resources back to the ready state. (#1380, @nachocano)
• pullsubscription.pubsub.events.google.com and topic.pubsub.events.google.com have been removed. Make sure to run the pre-upgrade job in the Action Required section to properly clean these resources up. (#1129, @Harwayne)

Authorization

• config-gcp-auth is a new ConfigMap in the cloud-run-events namespace. It is used to control which GCP credentials are defaulted into Channels and Sources. By default, it will use the same default which was already present, a secret named google-cloud-key. (#1183, @Harwayne)
• spec.googleServiceAccount has been removed. All automatic Workload Identity related controller work is now handled via config-gcp-auth. (#1249, @grac3gao)
• If either spec.serviceAccountName or spec.secret is specified, then no additional credential defaulting will be applied to the Source or Channel. (#1214, @grac3gao)

Resource Usage

• Adjusted Broker components memory resource
  • Increased Broker Ingress memory limit to 1000Mi.
  • Set memory limits much high than requested.
• Adjusted HPA
  • The avg memory usage is set to half of the limit for Broker Fanout/Retry (this hopefully helps to mitigate some problem from surging memory usage).
  • Lower the max replicas for Broker Fanout/Retry as we've seen a higher DNS error rate with more than 10 replicas.
• Relaxed liveness probe timeout (helps reduce some noise)
• Override MaxIdleConns for delivery HTTP client (helps with reusing connections and reduce the chance of DNS errors) (#1269, @yolocs)
• Adjusted PullSubscription receiver adapter CPU and memory requests and limits. We expect this to support 1000 QPS. (#1327, @capri-xiyue)
  • CPU request: 500m
  • CPU limit: 600m
  • memory request: 50mi
  • memory limit: 600mi
• Garbage collect BrokerCell if no Brokers are using it (#1162, @liu-cong)
• CloudStorageSource, CloudSchedulerSource and CloudAuditLogsSource do not create a Knative Service along with a Topic. (#1226, @nachocano)
• We are using CE SDK v2 for Sources and Channels, with all its performance improvements (#1296, @nachocano)

Reliability

• Add Broker Ingress liveness probe (#1179, @yolocs)
• Add Broker Ingress readiness probe (#1322, @yolocs)
• Add liveness probes for Broker Fanout and Retry (#1185, @yolocs)
• Add client side backoff retries for PubSub messages (#1218, @yolocs)
• Drop non-event messages. (#1212, @liu-cong)
• Broker Ingress rejects events when the Broker under request is not ready (#1272, @yolocs)
• If the Topic of a Broker/Trigger is deleted out-of-band, the Topic and Subscription will be recreated. This may cause unacknowledged events to be lost. (#1268, @liu-cong)
• The Webhook will disallow mutations to the _example section of ConfigMaps. This is to prevent accidentally altering the example, while intending to alter the actual, used configuration. (#1260, @Harwayne)

Observability

• Broker metrics won't include specific custom event types in Stackdriver. Only GCP defined event types will be present. All others will use the value custom. (#1352, @zargarpur)

Sources

• CloudBuildSource is now v1beta1. (#1264, @capri-xiyue)

Knative Knative Gcp release v0.17.3

Notices

• V1alpha1 resources (Source, Topic, PullSubscription and Channel) have been deprecated and will be removed in 0.19.
  Please use v1 (v1beta1 for Channel) instead. (#1562, @danyinggu)

New Features

• Add back-off in IAMPolicyManager (#1461, @grac3gao)
• Add timeout delivery metrics to broker fanout and retry (#1578, @grac3gao)

Other Changes

• Change v1beta1 channel to use duckv1 identifiable
  • Change v1beta1 cloudbuildsource to use duckv1 identifiable and duckv1.PubSubable (#1505, @capri-xiyue)

Removed Features

• Explicitly disallow istio sidecar injection for controller/webhook/broker fanout/retry (#1476, @yolocs)
• Remove spec.payloadFormat from the v1 type. (#1433, @danyinggu)
• IdentityStatus.ServiceAccountName is removed. PullSubscriptionSpec.Mode is removed. (#1441, @danyinggu)

Bug Fixes

• Malformed replies will be treated as error (instead of being silently ignored). (#1419, @yolocs)

Knative Knative Gcp release v0.17.2

Notices

• V1alpha1 resources (Source, Topic, PullSubscription and Channel) have been deprecated and will be removed in 0.19.
  Please use v1 (v1beta1 for Channel) instead. (#1562, @danyinggu)

New Features

• Add back-off in IAMPolicyManager (#1461, @grac3gao)
• Add timeout delivery metrics to broker fanout and retry (#1578, @grac3gao)

Other Changes

• Change v1beta1 channel to use duckv1 identifiable
  • Change v1beta1 cloudbuildsource to use duckv1 identifiable and duckv1.PubSubable (#1505, @capri-xiyue)

Removed Features

• Explicitly disallow istio sidecar injection for controller/webhook/broker fanout/retry (#1476, @yolocs)
• Remove spec.payloadFormat from the v1 type. (#1433, @danyinggu)
• IdentityStatus.ServiceAccountName is removed. PullSubscriptionSpec.Mode is removed. (#1441, @danyinggu)

Bug Fixes

• Malformed replies will be treated as error (instead of being silently ignored). (#1419, @yolocs)