Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add ManagedKafkaTopic as a direct resource #3585

Merged

Conversation

jingyih
Copy link
Collaborator

@jingyih jingyih commented Feb 1, 2025

No description provided.

@yuwenma
Copy link
Collaborator

yuwenma commented Feb 3, 2025

/assign @jasonvigil

Copy link
Collaborator

@jasonvigil jasonvigil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good, just a couple questions, and a small nit.

// connecting directly to the cluster. Structured like:
// projects/{project}/locations/{location}/clusters/{cluster}/topics/{topic}
// +kcc:proto:field=google.cloud.managedkafka.v1.Topic.name
Name *string `json:"name,omitempty"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the same as the externalRef?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is an output field that essentially is the same as externalRef. I can see that we might want to ignore it since it serves the same purpose. We just need to be consistent cross the resources.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have not been adding this field to observedState for my resources, since it is the same as externalRef.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'll remove it from this resource and also from "ManagedKafkaCluster."

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added TODOs to remove the field in a followup PR.

apis/managedkafka/v1alpha1/topic_types.go Outdated Show resolved Hide resolved
return mapCtx.Err()
}

// cannot use common.CompareProtoMessage because some proto fields are not correctly labeled as output only, which means the path may include a output only field
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which fields are incorrectly labeled? Seems the name field is the only output field. Reason I ask is because name is common to a lot of resources. Should we be treating name as an output field?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, the name field is not explicitly labeled as "output only." This means the YAML does not include name in spec since it isn't a spec field, but the actual GCP resource has a computed name field. As a result, the function detects a diff in the name field.

However, for this resource, the name field is indeed an output-only field. In the Create API, the "topic_id" is required, while the name field is explicitly ignored.

https://github.com/googleapis/google-cloud-go/blob/0ddf33831c5f0bed7072d988795de353956f22db/managedkafka/apiv1/managedkafkapb/managed_kafka.pb.go#L683-L690

Should we be treating name as an output field?

I believe it depends on the resource. If a resource ignores the "name" field during creation and computes its value afterward, then it should be considered an output-only field.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a result, the function detects a diff in the name field.

To work around this issue, I have been explicitly setting the name field in the proto object (after mapping ToProto), before comparing. Here is an example:

// Set name and etag manually, because they are not filled-in by WorkstationConfigSpec_ToProto.
desiredPb.Name = a.id.String()
desiredPb.Etag = a.actual.Etag
paths, err := common.CompareProtoMessage(desiredPb, a.actual, common.BasicDiff)
if err != nil {
return err
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sound good. I will give it a try. Thanks!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR updated.

@jasonvigil
Copy link
Collaborator

/lgtm

@google-oss-prow google-oss-prow bot added the lgtm label Feb 4, 2025
@jasonvigil
Copy link
Collaborator

/assign yuwenma

commonv1alpha1.CommonSpec `json:",inline"`

// +required
Location string `json:"location"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest adding descriptions for Location and ClusterRef, because these comments are the source to generate the reference doc, which is the most straightforward (if not only) way for users to configure the KCC object

Copy link
Collaborator Author

@jingyih jingyih Feb 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! 32b1013

// connecting directly to the cluster. Structured like:
// projects/{project}/locations/{location}/clusters/{cluster}/topics/{topic}
// +kcc:proto:field=google.cloud.managedkafka.v1.Topic.name
Name *string `json:"name,omitempty"` // TODO(jingyih): remove this field as it is the same as externalRef
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, any reasons to add this field (with a cleanup TODO) if this is a pure new Alpha resource?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this is a new resource. I am removing the field in a followup PR.

@yuwenma
Copy link
Collaborator

yuwenma commented Feb 4, 2025

/approve
/hold

Left a few nits. Non blockers. Hold in case you can take a look

Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: yuwenma

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow bot added approved and removed lgtm labels Feb 4, 2025
@jingyih jingyih force-pushed the managed_kafka_topic branch from c6056a0 to 32b1013 Compare February 4, 2025 23:24
@jingyih
Copy link
Collaborator Author

jingyih commented Feb 4, 2025

PR rebased to pick up the fix for broken HEAD.

@yuwenma
Copy link
Collaborator

yuwenma commented Feb 4, 2025

/lgtm

@jingyih
Copy link
Collaborator Author

jingyih commented Feb 5, 2025

/hold cancel

@google-oss-prow google-oss-prow bot merged commit 2aa4fe3 into GoogleCloudPlatform:master Feb 5, 2025
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants