-
Notifications
You must be signed in to change notification settings - Fork 53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor persistence strategy for dlo generation #292
Conversation
...ark/src/main/java/com/linkedin/openhouse/jobs/spark/DataLayoutStrategyGeneratorSparkApp.java
Outdated
Show resolved
Hide resolved
...out/src/main/java/com/linkedin/openhouse/datalayout/persistence/StrategiesDaoTableProps.java
Outdated
Show resolved
Hide resolved
libs/datalayout/src/main/java/com/linkedin/openhouse/datalayout/persistence/StrategiesDao.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left a few comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly nits.
...ark/src/main/java/com/linkedin/openhouse/jobs/spark/DataLayoutStrategyGeneratorSparkApp.java
Outdated
Show resolved
Hide resolved
...src/test/java/com/linkedin/openhouse/jobs/spark/DataLayoutStrategyGeneratorSparkAppTest.java
Outdated
Show resolved
Hide resolved
...src/test/java/com/linkedin/openhouse/jobs/spark/DataLayoutStrategyGeneratorSparkAppTest.java
Outdated
Show resolved
Hide resolved
...src/test/java/com/linkedin/openhouse/jobs/spark/DataLayoutStrategyGeneratorSparkAppTest.java
Show resolved
Hide resolved
log.info( | ||
"Generated {} strategies {}", | ||
strategies.size(), | ||
strategies.stream().map(Object::toString).collect(Collectors.joining(", "))); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you preserve this logging line, it's helpful to see the values for an app run
Summary
Made following changes:
isPartitioned
to thedlo_strategies
table.appendToDloStrategiesTable
so that the persistence strategy can be overridden.Changes
For all the boxes checked, please include additional details of the changes made in this pull request.
Testing Done
Added E2E test for DLO generation app logic.
Additional Information
For all the boxes checked, include additional details of the changes made in this pull request.