Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add forceTcp and keepSocket options #324

Merged
merged 3 commits into from
Aug 22, 2024
Merged

add forceTcp and keepSocket options #324

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9e48e21
add forceTcp and keepSocket options
leo-astorsky Aug 13, 2024
d3ec8c0
fixes (naming, omitempty, move to separate canon test)
leo-astorsky Aug 21, 2024
d1e4d66
fixes (getStrawberryController, ptr.To, err)
leo-astorsky Aug 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions api/v1/ytsaurus_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -571,6 +571,13 @@ type CommonSpec struct {
//+kubebuilder:default:=false
//+optional
UseIPv4 bool `json:"useIpv4"`
//+kubebuilder:default:=true
//+optional
KeepSocket bool `json:"keepSocket"`
//+kubebuilder:default:=false
//+optional
ForceTcp bool `json:"forceTcp"`
l0kix2 marked this conversation as resolved.
Show resolved Hide resolved

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

force_tcp - false
keep_socket - true

cause of https://github.com/ytsaurus/ytsaurus/blob/b41294f14e346c455b1d69d268b029a5d07f71f5/yt/yt/core/dns/config.cpp#L21-L24

//+kubebuilder:default:=true
//+optional
UseShortNames bool `json:"useShortNames"`
Expand Down
6 changes: 6 additions & 0 deletions config/crd/bases/cluster.ytsaurus.tech_remoteexecnodes.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -689,6 +689,9 @@ spec:
additionalProperties:
type: string
type: object
forceTcp:
default: false
type: boolean
hostNetwork:
default: false
description: Use the host's network namespace for all components.
Expand Down Expand Up @@ -901,6 +904,9 @@ spec:
resources required.
type: object
type: object
keepSocket:
default: true
type: boolean
locations:
items:
properties:
Expand Down
6 changes: 6 additions & 0 deletions config/crd/bases/cluster.ytsaurus.tech_ytsaurus.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10024,6 +10024,9 @@ spec:
additionalProperties:
type: string
type: object
forceTcp:
default: false
type: boolean
hostNetwork:
default: false
description: Use the host's network namespace for all components.
Expand Down Expand Up @@ -12495,6 +12498,9 @@ spec:
jobImage:
description: Default docker image for user jobs.
type: string
keepSocket:
default: true
type: boolean
masterCaches:
properties:
affinity:
Expand Down
6 changes: 6 additions & 0 deletions docs/api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,8 @@ _Appears in:_
| `ephemeralCluster` _boolean_ | Allow prioritizing performance over data safety. Useful for tests and experiments. | false | |
| `useIpv6` _boolean_ | | false | |
| `useIpv4` _boolean_ | | false | |
| `keepSocket` _boolean_ | | true | |
| `forceTcp` _boolean_ | | false | |
| `useShortNames` _boolean_ | | true | |
| `hostNetwork` _boolean_ | Use the host's network namespace for all components. | false | |
| `usePorto` _boolean_ | | false | |
Expand Down Expand Up @@ -1145,6 +1147,8 @@ _Appears in:_
| `ephemeralCluster` _boolean_ | Allow prioritizing performance over data safety. Useful for tests and experiments. | false | |
| `useIpv6` _boolean_ | | false | |
| `useIpv4` _boolean_ | | false | |
| `keepSocket` _boolean_ | | true | |
| `forceTcp` _boolean_ | | false | |
| `useShortNames` _boolean_ | | true | |
| `hostNetwork` _boolean_ | Use the host's network namespace for all components. | false | |
| `usePorto` _boolean_ | | false | |
Expand Down Expand Up @@ -1693,6 +1697,8 @@ _Appears in:_
| `ephemeralCluster` _boolean_ | Allow prioritizing performance over data safety. Useful for tests and experiments. | false | |
| `useIpv6` _boolean_ | | false | |
| `useIpv4` _boolean_ | | false | |
| `keepSocket` _boolean_ | | true | |
| `forceTcp` _boolean_ | | false | |
| `useShortNames` _boolean_ | | true | |
| `hostNetwork` _boolean_ | Use the host's network namespace for all components. | false | |
| `usePorto` _boolean_ | | false | |
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetControllerAgentsConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetDataNodeConfig/with-trash-ttl/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetDataNodeConfig/with-watermark/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetDataNodeConfig/without-trash-ttl/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetDataNodeWithoutYtsaurusConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%false;
"enable_ipv6"=%true;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetDiscoveryConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetExecNodeConfig/with-job-resources/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetExecNodeConfig/without-job-resources/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions ...ondata/TestGetExecNodeConfigWithCri/isolated-containers-with-job-resources/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions ...ata/TestGetExecNodeConfigWithCri/isolated-containers-without-job-resources/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions ...canondata/TestGetExecNodeConfigWithCri/single-container-with-job-resources/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions ...ondata/TestGetExecNodeConfigWithCri/single-container-without-job-resources/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetExecNodeWithoutYtsaurusConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%false;
"enable_ipv6"=%true;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetHTTPProxyConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetMasterCachesConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetMasterCachesWithFixedHostsConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetMasterConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetMasterWithFixedHostsConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%false;
"enable_ipv6"=%true;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetNativeClientConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
logging={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetQueryTrackerConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetQueueAgentConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetRPCProxyConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetRPCProxyWithoutOauthConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%false;
"enable_ipv6"=%true;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetSchedulerConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetSchedulerWithFixedMasterHostsConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%false;
"enable_ipv6"=%true;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
4 changes: 3 additions & 1 deletion pkg/ytconfig/canondata/TestGetStrawberryControllerConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@
chyt={
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%true;
"enable_ipv6"=%false;
Copy link
Contributor Author

@leo-astorsky leo-astorsky Aug 14, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cause of

and maybe it should be done another way

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it was hardcoded for strawberry, but now it is configured according to the common value, ok

Copy link
Collaborator

@l0kix2 l0kix2 Aug 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a second thought that may be backward incompatible change and maybe we need to think here more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So users, who have ipv4=false;ipv6=true or ipv4=true;ipv6=false (assume false/false is invalid configuration) will redeploy strawberry on operator update with this settings, not true/true as before which can break things for them. Not sure yet how problematic such update could be.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose we can describe breaking changes in the next release about strawberry ipv4/6.

"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
};
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetTCPProxyConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetTabletNodeConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetTabletNodeWithoutYtsaurusConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%false;
"enable_ipv6"=%true;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/canondata/TestGetYQLAgentConfig/test.canondata
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
"address_resolver"={
"enable_ipv4"=%true;
"enable_ipv6"=%false;
"keep_socket"=%true;
"force_tcp"=%false;
retries=1000;
};
"solomon_exporter"={
Expand Down
15 changes: 13 additions & 2 deletions pkg/ytconfig/chyt.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,18 @@ type ChytInitCluster struct {
Families []string `yson:"families"`
}

func getStrawberryController() StrawberryController {
type ChytConfig struct {
AddressResolver AddressResolver `yson:"address_resolver"`
}

func getStrawberryController(resolver *AddressResolver) StrawberryController {
chytConfig := ChytConfig{
AddressResolver: *resolver,
}
chytYsonConfig, err := marshallYsonConfig(chytConfig)
if err != nil {
chytYsonConfig = yson.RawValue("{address_resolver={enable_ipv4=%true;enable_ipv6=%true;retries=1000}}")
l0kix2 marked this conversation as resolved.
Show resolved Hide resolved
}
return StrawberryController{
Strawberry: Strawberry{
Root: "//sys/strawberry",
Expand All @@ -36,7 +47,7 @@ func getStrawberryController() StrawberryController {
RobotUsername: consts.StrawberryControllerUserName,
},
Controllers: map[string]yson.RawValue{
"chyt": yson.RawValue("{address_resolver={enable_ipv4=%true;enable_ipv6=%true;retries=1000}}"),
"chyt": chytYsonConfig,
},
HTTPAPIEndpoint: fmt.Sprintf(":%v", consts.StrawberryHTTPAPIPort),
}
Expand Down
2 changes: 2 additions & 0 deletions pkg/ytconfig/common.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ type ClusterConnection struct {
type AddressResolver struct {
EnableIPv4 bool `yson:"enable_ipv4"`
EnableIPv6 bool `yson:"enable_ipv6"`
KeepSocket bool `yson:"keep_socket"`
ForceTcp bool `yson:"force_tcp"`
l0kix2 marked this conversation as resolved.
Show resolved Hide resolved
Retries *int `yson:"retries,omitempty"`

LocalhostNameOverride *string `yson:"localhost_name_override,omitempty"`
Expand Down
Loading
Loading