Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

include results from integration test of archive db based cooperation #147

Merged
merged 29 commits into from
Nov 19, 2023

Conversation

cherusk
Copy link
Collaborator

@cherusk cherusk commented Nov 17, 2023

No description provided.

@cherusk cherusk added checkout and removed checkout labels Nov 17, 2023
@cherusk cherusk added checkout and removed checkout labels Nov 17, 2023
Dependencies are not to be installed on the airflow container anymore,
hence we have to resolve dependencies in the tasks that are run on the
dask workers.
@cherusk cherusk added checkout and removed checkout labels Nov 17, 2023
@cherusk cherusk added checkout and removed checkout labels Nov 17, 2023
@cherusk cherusk force-pushed the godon_cooperative_integration_remediation branch from 140dd28 to c613d33 Compare November 17, 2023 18:09
@cherusk cherusk added checkout and removed checkout labels Nov 17, 2023
Those need access to the dags and have to have
airflow preinstalled.
@cherusk cherusk added checkout and removed checkout labels Nov 17, 2023
@cherusk cherusk added checkout and removed checkout labels Nov 17, 2023
To be compatible with preinstalled components.
@cherusk cherusk force-pushed the godon_cooperative_integration_remediation branch from f073a1c to 398e9c1 Compare November 18, 2023 22:34
@cherusk cherusk added checkout and removed checkout labels Nov 18, 2023
@cherusk cherusk added checkout and removed checkout labels Nov 19, 2023
Keep objective seperate and include it in the optuna submitting code
paths.

Keeping it in the submitting code path is important otherwise the
function pickling of recent dask distributed will fail that it cannot
find the root dag module at unpickling side, the dask worker.
@cherusk cherusk force-pushed the godon_cooperative_integration_remediation branch from 7f839e1 to 108a614 Compare November 19, 2023 12:45
@cherusk cherusk added checkout and removed checkout labels Nov 19, 2023
@cherusk cherusk added checkout and removed checkout labels Nov 19, 2023
Let's not use sqlalchemy to generate the archive db interacting sql
queries because that has led to complications. bindparams was not
working as expect for String/Text type.
@cherusk cherusk force-pushed the godon_cooperative_integration_remediation branch from 699a958 to e8f07a0 Compare November 19, 2023 16:03
@cherusk cherusk added checkout and removed checkout labels Nov 19, 2023
@cherusk cherusk added checkout and removed checkout labels Nov 19, 2023
@cherusk cherusk merged commit 2839cc7 into master Nov 19, 2023
4 checks passed
@cherusk cherusk deleted the godon_cooperative_integration_remediation branch November 19, 2023 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant