Issue 910 - Added timeout to BigQuery load_job.result()

[bear-jordan]

Overview

This PR addresses Issue #910. A timeout is added to prevent indefinite hanging.

From the original issue:

Per Google BQ query job documentation, it's possible to pass a default timeout parameter to the synchronous call that awaits a response.

This change would require choosing a reasonable maximum default (30s?), setting that up as a configurable parameter, and passing that parameter into load_job.result.

Changes

27/1/2024

• Added MAX_TIMEOUT = 30 constant
• Updated load_job.result() to use MAX_TIMEOUT

31/1/2024 — @Jason94

• Added max_timeout: int = 30 as a keyword argument
• Updated load_job.result() to use max_timeout
• Added DeadlineExceeded error handling (docs)

[Jason94]

Instead of using a constant, perhaps this should be a keyword parameter that defaults to 30. Something like:

def get_table_ref(client, table_name, timeout = 30):

That way if 30 isn't enough for my use case, or is too long, I can adapt the behavior.

[bear-jordan]

Hi, for sure! I think that is a cleaner implementation, good suggestion

[Jason94]

Same comment

[bear-jordan]

Hi, looked into why the Mac build is failing (see here). Github isn't reporting any outages — not too sure what is going on

Warning: Failed to download action 'https://api.github.com/repos/actions/cache/tarball/e12d46a63a90f2fae62d114769bbf2a179198b5c'. Error: Operation timed out (codeload.github.com:443)
Warning: Back off 26.77 seconds before retry.
Warning: Failed to download action 'https://api.github.com/repos/actions/cache/tarball/e12d46a63a90f2fae62d114769bbf2a179198b5c'. Error: nodename nor servname provided, or not known (codeload.github.com:443)
Warning: Back off 21.133 seconds before retry.
Error: Operation timed out (api.github.com:443)

[Jason94]

Hi, looked into why the Mac build is failing (see here). Github isn't reporting any outages — not too sure what is going on

Warning: Failed to download action 'https://api.github.com/repos/actions/cache/tarball/e12d46a63a90f2fae62d114769bbf2a179198b5c'. Error: Operation timed out (codeload.github.com:443)
Warning: Back off 26.77 seconds before retry.
Warning: Failed to download action 'https://api.github.com/repos/actions/cache/tarball/e12d46a63a90f2fae62d114769bbf2a179198b5c'. Error: nodename nor servname provided, or not known (codeload.github.com:443)
Warning: Back off 21.133 seconds before retry.
Error: Operation timed out (api.github.com:443)

Sometimes this happens with these mac tests :'( I'm rerunning it and hopefully it'll pass.

[bear-jordan]

Ominous that it's a timeout error 😂

[shaunagm]

Looks like the tests are passing now! What's the status of this PR? Are we waiting on re-review, or on changes?

[bear-jordan]

I might have missed something, but I think all changes have been addressed.

[shaunagm]

Awesome! @Jason94 whenever you get a chance can you re-review?