Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fivetran Sync Issue when running Airflow DAG #106

Open
prashanttiwaritrellix opened this issue Sep 2, 2024 · 7 comments
Open

Fivetran Sync Issue when running Airflow DAG #106

prashanttiwaritrellix opened this issue Sep 2, 2024 · 7 comments

Comments

@prashanttiwaritrellix
Copy link

Deferred Sync Log.log
successful Sync Log.log

Hi Team,

I am running Fivetran Sync using Airflow Astronomer by importing
"from fivetran_provider_async.operators import FivetranOperatorAsync" ,
when job is running in Airflow then sometime Fivetran completes the job but does not send signal to Airflow, so my airflow job get stuck in deferred mode for infinite time. Please help me how to fix this.

Airflow version - Astro Runtime
8.8.0 (based on Airflow v)

We are using Airflow-provider-fivetran-async package version - 1.0.0a5 (also tried 2.1.0)

@pankajastro
Copy link
Contributor

Could you please upgrade the package and test with https://github.com/astronomer/airflow-provider-fivetran-async/releases/tag/2.1.0

@pankajastro
Copy link
Contributor

I noticed you've mentioned trying version 2.1.0 as well, but FivetranOperatorAsync isn't available in any version beyond 2.0.0. A similar issue #93 was reported earlier. However, after upgrading, that issue was resolved for them.

@prashanttiwaritrellix
Copy link
Author

prashanttiwaritrellix commented Sep 3, 2024 via email

@pankajastro
Copy link
Contributor

Hey, I'm curious whether you’ve recently started using this package or if there have been any changes or upgrade on your end that might have led to this error. Could you provide the steps and DAG that would help in reproducing the issue?

There little chance why it would work for some run and not work some run give that Fivetran response is correct and Airflow behaving correctly.

I think it might be helpful to create a support ticket with Astronomer. They could have better access to the system and might be able to assist you more effectively.

@prashanttiwaritrellix
Copy link
Author

prashanttiwaritrellix commented Sep 5, 2024 via email

@pankajastro
Copy link
Contributor

Hey @prashanttiwaritrellix, Do you know what are delta between working and not working?

@pankajastro
Copy link
Contributor

Could you please validate #109 (comment) and this true then you should report issue in Airflow repo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants