Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle case for element that is older than all migrations provided #2094

Open
bossenti opened this issue Oct 27, 2023 · 2 comments · May be fixed by #3195
Open

Handle case for element that is older than all migrations provided #2094

bossenti opened this issue Oct 27, 2023 · 2 comments · May be fixed by #3195
Assignees
Labels
backend Everything that is related to the StreamPipes backend enhancement New feature or request
Milestone

Comments

@bossenti
Copy link
Contributor

Body

Within #2001 we introduced a basic support for migrating pipeline elements.
One corner case that was thereby neglected is the following:

Let's assume an existing StreamPipes instance has an adapter of type AdapterA that is still in version 0. This StreamPipes has now updated to a new version that is far ahead of the current one and therefore only provides migrations for AdapterA from version 1 onwards.

As of now, this adapter instance is simply not recognized and the adapter is not usable anymore.
This case should also be handled properly, at least by logging it somewhere.

StreamPipes Committer

I acknowledge that I am a maintainer/committer of the Apache StreamPipes project.

@bossenti bossenti added enhancement New feature or request backend Everything that is related to the StreamPipes backend labels Oct 27, 2023
@bossenti bossenti added this to the 0.95.0 milestone Oct 27, 2023
@bossenti bossenti modified the milestones: 0.95.0, 0.97.0 Apr 8, 2024
@IsaakKrut
Copy link
Contributor

IsaakKrut commented Aug 26, 2024

Hi @bossenti, I'd like to learn more about migrations. I took a look at this issue and there is already logging in place in AdapterMigrationManager on line 108.
else { LOG.info( "Migration is not applicable for adapter '{}' because of a version mismatch - " + "adapter version: '{}', migration starts at: '{}'", adapterDescription.getElementId(), adapterVersion, migrationConfig.fromVersion() ); }
It was introduced as part of #2077 . However, the adapter is not stopped when the adapter's version is lower than the migrationConfig's fromVersion. It's stopped only when versions are equal and migration fails.

My questions are:

  1. Should the adapter be stopped when its version is lower than the available migrations? What happens if it is not?
  2. With logging already in place, can this issue be closed?

@tenthe
Copy link
Contributor

tenthe commented Aug 27, 2024

Hi @IsaakKrut,

to your first question. I would say that the adapter should be stopped if it cannot be migrated. So if this has not been done yet, I would suggest implementing this.

Regarding the second question: I think it is good that it is logged.

However, the logs are not shown to the user in the UI, so it's not obvious to the user.
It would be cool to have a similar mechanism in the adapter overview as we have for pipelines to indicate problems.
For adapters, the status currently only shows whether the adapter is running or not:

grafik

Pipelines have a health status (PipelineHealthStatus) which is used to indicate problems to the user, maybe we can do something similar for adapters here and indicate to the user that the migration has failed and some manual work needs to be done to fix it.

The UI code for the pipelines is in PipelineOverviewComponent and for the adapters ExistingAdaptersComponent.

What do you think of this? If you are interested in implementing this, let me know, I can also try to help you if you have any problems.

Cheers,
Philipp

IsaakKrut added a commit to IsaakKrut/streampipes that referenced this issue Aug 28, 2024
@IsaakKrut IsaakKrut self-assigned this Aug 28, 2024
IsaakKrut added a commit to IsaakKrut/streampipes that referenced this issue Aug 28, 2024
@IsaakKrut IsaakKrut linked a pull request Aug 28, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Everything that is related to the StreamPipes backend enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants