Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(SqlExecutionRepository): fixed bug in sql repository in orca-sql … #4697

Merged
merged 3 commits into from
Apr 15, 2024
Merged

fix(SqlExecutionRepository): fixed bug in sql repository in orca-sql … #4697

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
93fdf9f
fix(SqlExecutionRepository): fixed bug in sql repository in orca-sql …
juangod-wise Apr 9, 2024
42081af
test(sql): demonstrate behavior of retrievePipelinesForApplication
dbyron-sf Apr 15, 2024
5d1eec3
Merge branch 'master' into fix-execution-repository-bug
mergify[bot] Apr 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions ...main/kotlin/com/netflix/spinnaker/orca/sql/pipeline/persistence/SqlExecutionRepository.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1119,12 +1119,16 @@ class SqlExecutionRepository(
type,
conditions = {
if (cursor == null) {
it.where("1=1")
if (where == null) {
it.where("1=1")
} else {
where(it)
}
Comment on lines +1122 to +1126
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the case in which the cursor is null (it always starts as null in the first iteration), the custom where statements that are passed to the function are not used on the query, effectively not filtering the first page.

} else {
if (where == null) {
it.where(field("id").gt(cursor))
it.where(field("id").lt(cursor))
} else {
where(it).and(field("id").gt(cursor))
where(it).and(field("id").lt(cursor))
Comment on lines +1129 to +1131
Copy link
Contributor Author

@juangod-wise juangod-wise Apr 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pagination is broken, which makes it only show the first page, for example, in the unit tests on orca-sql/src/test/groovy/com/netflix/spinnaker/orca/sql/cleanup/OldPipelineCleanupPollingNotificationAgentSpec.groovy, we have a page size 10, and executions with id: 1, 2, 11, 12, ..., 20

Then, iteration 1 returns the first 10 elements in desc order:

--> 20, ..., 11 (10 elements)

And cursor = 11

Then the second page gets the elements greater than the cursor, which are

--> 20, ..., 12 (9 elements)

Then the stop condition in the PagedIterator is triggered:

isLastPage = currentPage.size < pageSize

Thus, the second page is never returned and only 10 elements are present, when there should be 12.

When we change greater than, for lesser than, we get:

Iteration 1:

--> 20, ..., 11 (10 elements)
Cursor = 11

Iteration 2, elements lesser than 11:

--> 1, 2

Then we get the correct response.

}
}
},
Expand Down
4 changes: 2 additions & 2 deletions .../netflix/spinnaker/orca/sql/cleanup/OldPipelineCleanupPollingNotificationAgentSpec.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ abstract class OldPipelineCleanupPollingNotificationAgentSpec extends Specificat
def allExecutions = executionRepository.retrievePipelinesForApplication(app).toList().toBlocking().first().unique()

then:
allExecutions.size() == 10
allExecutions.size() == 12

when:
cleanupAgent.tick()
Expand All @@ -125,7 +125,7 @@ abstract class OldPipelineCleanupPollingNotificationAgentSpec extends Specificat
def allExecutions = executionRepository.retrievePipelinesForApplication(app).toList().toBlocking().first().unique()

then:
allExecutions.size() == 10
allExecutions.size() == 12

when:
cleanupAgent.tick()
Expand Down
15 changes: 15 additions & 0 deletions .../kotlin/com/netflix/spinnaker/orca/sql/pipeline/persistence/SqlExecutionRepositoryTest.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,21 @@ class SqlExecutionRepositoryTest : JUnit5Minutests {
assertThat(actualPipelineExecution).isEqualTo(pipelineExecution)
}
}

context("retrievePipelinesForApplication") {
val pipelineExecution1 = PipelineExecutionImpl(ExecutionType.PIPELINE, "application-1")
val pipelineExecution2 = PipelineExecutionImpl(ExecutionType.PIPELINE, "application-2")

test("correctly use where clause") {
// Store pipelines in two different applications
sqlExecutionRepository.store(pipelineExecution1)
sqlExecutionRepository.store(pipelineExecution2)

val observable = sqlExecutionRepository.retrievePipelinesForApplication("application-2")
val executions = observable.toList().toBlocking().single()
assertThat(executions.map(PipelineExecution::getApplication).single()).isEqualTo("application-2")
}
}
}

private inner class Fixture {
Expand Down