Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable real chunks job #412

Merged
merged 5 commits into from
Jan 17, 2025
Merged

Enable real chunks job #412

merged 5 commits into from
Jan 17, 2025

Conversation

l0kix2
Copy link
Collaborator

@l0kix2 l0kix2 commented Jan 8, 2025

Fix #396

I've added k8s job in the master component. The job simply checks if ytsaurus image version is between 23.2—24.1 including and if yes — sets //sys/@config/node_tracker/enable_real_chunk_locations=%true according to the issue.
It is being executed:

  • when the cluster is Initilazing (after master pods are created and before dnd pods are created)
  • on each master update (in FullUpdate mode since we want dnds to be restarted also) after master in safe mode but before all pods are recreated (because we want flag before dnd restart).

This is required to prepare user clusters for 24.2. We will ask users to update their cluster to some (23.2—24.1) version with ytop version with this code prior to 24.2 update.

How it was tested

  1. I've added that flag is set to true after update for current versions we test against (create 23.2.0 and update to 23.2.1 + create 23.2.0 and update to 24.1.0) + no node with use_imaginary_chunk_locations=true exist after the update.
  2. I've manually tested that flag is not set for 23.1 (we don't test against it, kinda unsuported i'd say)
  3. I've manually tested that update from 23.1 (flag false) to 23.2 sets flag to true.

Other

We also may implement some check for possibility of update to 24.2 checking data nodes meta information in a separate issue.

@l0kix2 l0kix2 changed the title Enable real chunks job draft Enable real chunks job Jan 9, 2025
@l0kix2 l0kix2 force-pushed the l0kix2/396_enable_real_chunk_locations branch 3 times, most recently from ec3cbaa to a2d14b0 Compare January 10, 2025 13:56
@l0kix2 l0kix2 requested a review from achulkov2 January 10, 2025 15:32
@l0kix2 l0kix2 marked this pull request as ready for review January 10, 2025 15:36
Copy link
Collaborator

@achulkov2 achulkov2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All contents LGTM, but I think it is useful to add a validation job that would stop the update to 24.2 if it would lead to master cores.

@l0kix2 l0kix2 force-pushed the l0kix2/396_enable_real_chunk_locations branch from 46e8a96 to dedebf5 Compare January 17, 2025 14:31
@l0kix2 l0kix2 merged commit e7a93a3 into main Jan 17, 2025
6 checks passed
@l0kix2 l0kix2 deleted the l0kix2/396_enable_real_chunk_locations branch January 17, 2025 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable enable_real_chunk_locations before update to 24.2
2 participants