Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement OperationsRestAsyncTransport to support long running operations #700

Merged

Conversation

parthea
Copy link
Collaborator

@parthea parthea commented Sep 20, 2024

Towards b/362946071

This PR depends on #701

@product-auto-label product-auto-label bot added the size: l Pull request size is large. label Sep 20, 2024
@parthea parthea force-pushed the operations-rest-async-transport branch 7 times, most recently from 21bdadd to 1e108b4 Compare September 20, 2024 20:30
@parthea parthea changed the base branch from main to add-extra-for-aiohttp September 20, 2024 20:30
@parthea parthea force-pushed the operations-rest-async-transport branch 2 times, most recently from 0ee82e0 to 6e51532 Compare September 20, 2024 20:50
@parthea parthea force-pushed the operations-rest-async-transport branch 2 times, most recently from b6c1c6c to eb78445 Compare September 23, 2024 16:43
@parthea parthea marked this pull request as ready for review September 25, 2024 15:06
@parthea parthea requested review from a team as code owners September 25, 2024 15:06
@ohmayr ohmayr changed the title feat: Add OperationsRestAsyncTransport to support long running operations feat: implement OperationsRestAsyncTransport to support long running operations Sep 30, 2024
Base automatically changed from add-extra-for-aiohttp to main September 30, 2024 20:58
@ohmayr ohmayr force-pushed the operations-rest-async-transport branch from bf76338 to f880c72 Compare September 30, 2024 20:59
@ohmayr ohmayr changed the base branch from main to async-rest-lro-support-in-core October 1, 2024 19:45
tests/unit/operations_v1/test_operations_rest_client.py Outdated Show resolved Hide resolved
from google.auth.aio import credentials as ga_credentials_async

try:
import aiohttp
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we use aiohttp in this file? A search reveals no references

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't. We're only importing it to check that the library is installed with the async_rest extra.

Now, the actual problem is that if the libraries within the try block aren't installed, we silently skip the tests for async. Happy to discuss that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we try importing aiohttp and transport because (1) we'll need them elsewhere for async REST, and (2) we want to set the flag GOOGLE_AUTH_AIO_INSTALLED. We only look at the flag in this file to do the right thing.

I agree with doing the right thing. What makes me uneasy is the mysterious coupling with other modules "elsewhere" in (1). I think it would be more elegant to have a package-wide function or constant that takes care of querying the right things for async REST, probably by delegating to the files that use these imports.

However, this is a test file, so no need to block on this. But we should follow up.

google/api_core/operations_v1/transports/__init__.py Outdated Show resolved Hide resolved
google/api_core/operations_v1/transports/rest_asyncio.py Outdated Show resolved Hide resolved
google/api_core/operations_v1/transports/rest_asyncio.py Outdated Show resolved Hide resolved
self,
request: operations_pb2.ListOperationsRequest,
*,
timeout: Optional[float] = None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we have retry as in the sync case?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

retry isn't passed down to the auth layer i.e. it's unused. We should file an issue to fix that for the case of sync and add this as a follow up feature to the async transport class.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you file an issue to do those things?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed issues for each.

google/api_core/gapic_v1/client_info.py Outdated Show resolved Hide resolved
from google.auth.aio import credentials as ga_credentials_async

try:
import aiohttp
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we try importing aiohttp and transport because (1) we'll need them elsewhere for async REST, and (2) we want to set the flag GOOGLE_AUTH_AIO_INSTALLED. We only look at the flag in this file to do the right thing.

I agree with doing the right thing. What makes me uneasy is the mysterious coupling with other modules "elsewhere" in (1). I think it would be more elegant to have a package-wide function or constant that takes care of querying the right things for async REST, probably by delegating to the files that use these imports.

However, this is a test file, so no need to block on this. But we should follow up.

self,
request: operations_pb2.ListOperationsRequest,
*,
timeout: Optional[float] = None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you file an issue to do those things?

google/api_core/operations_v1/transports/rest.py Outdated Show resolved Hide resolved
google/api_core/operations_v1/transports/rest_asyncio.py Outdated Show resolved Hide resolved
DEFAULT_CLIENT_INFO = gapic_v1.client_info.ClientInfo(
gapic_version=BASE_DEFAULT_CLIENT_INFO.gapic_version,
grpc_version=None,
rest_version=auth_version,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also just filed #719 to rationalize this further, separately.

@ohmayr ohmayr merged commit 3c7e43e into async-rest-lro-support-in-core Oct 3, 2024
5 checks passed
@ohmayr ohmayr deleted the operations-rest-async-transport branch October 3, 2024 05:06
ohmayr added a commit that referenced this pull request Oct 7, 2024
* feat: implement `OperationsRestAsyncTransport` to support long running operations (#700)

* feat: Add OperationsRestAsyncTransport to support long running operations

* update TODO comment

* update TODO comment

* address feedback

* address feedback

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* fix mypy and lint issues

* minor fix

* add no cover

* fix no cover tag

* link coverage issue

* silence coverage issue

* fix statement name error

* address PR feedback

* address PR feedback

* address PR comments

---------

Co-authored-by: ohmayr <[email protected]>
Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* feat: implement async client for LROs (#707)

* feat: implement `AbstractOperationsAsyncClient` to support long running operations

* remove coverage guards

* address presubmit failures

* fix coverage for cancel operation

* tests cleanup

* fix incorrect tests

* file bugs

* add auth import

* address PR comments

* address PR comments

* fix unit tests and address more comments

* disable retry parameter

* add retry parameter

* address PR comments

---------

Co-authored-by: ohmayr <[email protected]>
Co-authored-by: ohmayr <[email protected]>

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Anthonios Partheniou <[email protected]>
Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants