Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test the server with the delta sharing client #129

Merged

Conversation

matar993
Copy link
Collaborator

@matar993 matar993 commented Nov 3, 2023

wip testing queryTable using a spark-shell with the delta-sharing-spark connector

@matar993 matar993 linked an issue Nov 3, 2023 that may be closed by this pull request
@matar993 matar993 changed the title wip testing queryTable using a spark-shell with the delta-sharing-spa… Test the server with the delta sharing client Nov 3, 2023
@tmnd1991 tmnd1991 self-requested a review November 6, 2023 08:30
@tmnd1991
Copy link

tmnd1991 commented Nov 6, 2023

In order to run the test (that we run through spark-shell last Friday) I suggest you to create a separate gradle module, make it depend on the app and run the app as a simple jar (https://stackoverflow.com/questions/20728621/running-an-executable-jar-file-built-from-a-gradle-based-project). Then you can write a test using spark (i.e. the test is a spark project), having a class path completely separated from the server one.

Copy link

@tmnd1991 tmnd1991 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good first step, I think to complete we need:

  • add needed gradle code to start the server before running the test
  • add needed gradle code to stop the server at the end of the tests
  • run these tests on (a dedicated step of) CI

Copy link

@tmnd1991 tmnd1991 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall a very good job, a couple of points not pointed out in comments:

  • you can generate the http client used in the integration tests starting from the openapi spec, it support java native http client as well (the one you used) that should reduce code duplication and avoid maintenance burden
  • you can share a bit more "build side" in this new module (through the internal plugin)

@matar993 matar993 force-pushed the feature/128-test-the-server-with-the-delta-sharing-client branch from 68ae711 to 065171d Compare November 23, 2023 13:29
@tmnd1991 tmnd1991 merged commit 68377bf into main Dec 1, 2023
@tmnd1991 tmnd1991 deleted the feature/128-test-the-server-with-the-delta-sharing-client branch December 1, 2023 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Test the server with the delta sharing client
3 participants