-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move tokenstorage #1065
Merged
Merged
Move tokenstorage #1065
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
|
||
Changed | ||
~~~~~~~ | ||
|
||
- TokenStorages have been moved from ``globus_sdk.experimental.tokenstorage`` | ||
to ``globus_sdk.tokenstorage``. (:pr:`NUMBER`) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
|
||
Token Caching | ||
============= | ||
|
||
The documentation in this section provides references for interfaces and standard | ||
implementations for caching OAuth2 tokens. While there are two distinct class | ||
hierarchies, :ref:`token_storages` and its predecessor :ref:`storage_adapters`, we | ||
recommend using the former. ``TokenStorage`` is a newer iteration of the token storage | ||
interface and includes a superset of the functionality previously supported in | ||
``StorageAdapter``. | ||
|
||
All constructs from both hierarchies are importable from the ``globus_sdk.tokenstorage`` | ||
namespace. | ||
|
||
.. toctree:: | ||
:maxdepth: 1 | ||
|
||
token_storages | ||
storage_adapters | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,133 @@ | ||
.. _storage_adapters: | ||
|
||
Storage Adapters (Legacy) | ||
========================= | ||
|
||
.. warning:: | ||
|
||
The class hierarchy documented here is legacy. | ||
|
||
We recommend using the newer class hierarchy documented at :ref:`token_storages`. | ||
|
||
The StorageAdapter component provides a way of storing and loading the tokens | ||
received from authentication and token refreshes. | ||
|
||
Usage | ||
----- | ||
|
||
StorageAdapter is available under the name ``globus_sdk.tokenstorage``. | ||
|
||
Storage adapters are the main objects of this subpackage. Primarily, usage | ||
should revolve around creating a storage adapter, potentially loading data from | ||
it, and using it as the ``on_refresh`` handler for an authorizer. | ||
|
||
For example: | ||
|
||
.. code-block:: python | ||
|
||
import os | ||
import globus_sdk | ||
from globus_sdk.tokenstorage import SimpleJSONFileAdapter | ||
|
||
my_file_adapter = SimpleJSONFileAdapter(os.path.expanduser("~/mytokens.json")) | ||
|
||
if not my_file_adapter.file_exists(): | ||
# ... do a login flow, getting back initial tokens | ||
# elided for simplicity here | ||
token_response = ... | ||
# now store the tokens, and pull out the tokens for the | ||
# resource server we want | ||
my_file_adapter.store(token_response) | ||
by_rs = token_response.by_resource_server | ||
tokens = by_rs["transfer.api.globus.org"] | ||
else: | ||
# otherwise, we already did this whole song-and-dance, so just | ||
# load the tokens from that file | ||
tokens = my_file_adapter.get_token_data("transfer.api.globus.org") | ||
|
||
|
||
# RereshTokenAuthorizer and ClientCredentialsAuthorizer both use | ||
# `on_refresh` callbacks | ||
# this feature is therefore only relevant for those auth types | ||
# | ||
# auth_client is the internal auth client used for refreshes, | ||
# and which was used in the login flow | ||
# note that this is all normal authorizer usage wherein | ||
# my_file_adapter is providing the on_refresh callback | ||
auth_client = ... | ||
authorizer = globus_sdk.RefreshTokenAuthorizer( | ||
tokens["refresh_token"], | ||
auth_client, | ||
access_token=tokens["access_token"], | ||
expires_at=tokens["expires_at_seconds"], | ||
on_refresh=my_file_adapter.on_refresh, | ||
) | ||
|
||
# or, for client credentials | ||
authorizer = globus_sdk.ClientCredentialsAuthorizer( | ||
auth_client, | ||
["urn:globus:auth:transfer.api.globus.org:all"], | ||
access_token=tokens["access_token"], | ||
expires_at=tokens["expires_at_seconds"], | ||
on_refresh=my_file_adapter.on_refresh, | ||
) | ||
|
||
# and then use the authorizer on a client! | ||
tc = globus_sdk.TransferClient(authorizer=authorizer) | ||
|
||
|
||
Complete Example Usage | ||
~~~~~~~~~~~~~~~~~~~~~~ | ||
|
||
The :ref:`Group Listing With Token Storage Script <example_group_listing_with_token_storage>` | ||
provides a complete and runnable example which leverages ``tokenstorage``. | ||
|
||
|
||
Adapter Types | ||
------------- | ||
|
||
.. module:: globus_sdk.tokenstorage | ||
|
||
``globus_sdk.tokenstorage`` provides base classes for building your own storage | ||
adapters, and several complete adapters. | ||
|
||
The :class:`SimpleJSONFileAdapter` is good for the "simplest possible" | ||
persistent storage, using a JSON file to store token data. | ||
|
||
:class:`MemoryAdapter` is even simpler still, and is great for writing and | ||
testing code which uses the `StorageAdapter` interface backed by an in-memory | ||
structure. | ||
|
||
The :class:`SQLiteAdapter` is more complex, for applications like the | ||
globus-cli which need to store various tokens and additional configuration. In | ||
addition to basic token storage, the :class:`SQLiteAdapter` provides for namespacing | ||
of the token data, and for additional configuration storage. | ||
|
||
Reference | ||
--------- | ||
|
||
.. autoclass:: StorageAdapter | ||
:members: | ||
:member-order: bysource | ||
:show-inheritance: | ||
|
||
.. autoclass:: MemoryAdapter | ||
:members: | ||
:member-order: bysource | ||
:show-inheritance: | ||
|
||
.. autoclass:: FileAdapter | ||
:members: | ||
:member-order: bysource | ||
:show-inheritance: | ||
|
||
.. autoclass:: SimpleJSONFileAdapter | ||
:members: | ||
:member-order: bysource | ||
:show-inheritance: | ||
|
||
.. autoclass:: SQLiteAdapter | ||
:members: | ||
:member-order: bysource | ||
:show-inheritance: | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
.. _token_storages: | ||
|
||
.. currentmodule:: globus_sdk.tokenstorage | ||
|
||
Token Storages | ||
============== | ||
|
||
Interacting with Globus services requires the use of Globus Auth-issued OAuth2 tokens. | ||
To assist in reuse of these tokens, the SDK provides an interface to store and | ||
retrieve this data across different storage backends. | ||
|
||
In addition to the interface, :class:`TokenStorage`, the SDK provides concrete | ||
implementations for some of the most common storage backends: | ||
|
||
- :class:`JSONTokenStorage` for storing tokens in a local JSON file. | ||
- :class:`SQLiteTokenStorage` for storing tokens in a local SQLite database. | ||
- :class:`MemoryTokenStorage` for storing tokens in process memory. | ||
|
||
|
||
Reference | ||
--------- | ||
|
||
.. autoclass:: TokenStorage | ||
:members: | ||
:member-order: bysource | ||
|
||
|
||
.. autoclass:: TokenStorageData | ||
:members: | ||
:member-order: bysource | ||
|
||
|
||
File-based Token Storages | ||
^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
||
.. autoclass:: JSONTokenStorage | ||
|
||
.. autoclass:: SQLiteTokenStorage | ||
:members: close, iter_namespaces | ||
|
||
|
||
Ephemeral Token Storages | ||
^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
||
.. autoclass:: MemoryTokenStorage | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,126 +1,3 @@ | ||
.. _tokenstorage: | ||
:orphan: | ||
|
||
TokenStorage | ||
============ | ||
|
||
The TokenStorage component provides a way of storing and loading the tokens | ||
received from authentication and token refreshes. | ||
|
||
Usage | ||
----- | ||
|
||
TokenStorage is available under the name ``globus_sdk.tokenstorage``. | ||
|
||
Storage adapters are the main objects of this subpackage. Primarily, usage | ||
should revolve around creating a storage adapter, potentially loading data from | ||
it, and using it as the ``on_refresh`` handler for an authorizer. | ||
|
||
For example: | ||
|
||
.. code-block:: python | ||
|
||
import os | ||
import globus_sdk | ||
from globus_sdk.tokenstorage import SimpleJSONFileAdapter | ||
|
||
my_file_adapter = SimpleJSONFileAdapter(os.path.expanduser("~/mytokens.json")) | ||
|
||
if not my_file_adapter.file_exists(): | ||
# ... do a login flow, getting back initial tokens | ||
# elided for simplicity here | ||
token_response = ... | ||
# now store the tokens, and pull out the tokens for the | ||
# resource server we want | ||
my_file_adapter.store(token_response) | ||
by_rs = token_response.by_resource_server | ||
tokens = by_rs["transfer.api.globus.org"] | ||
else: | ||
# otherwise, we already did this whole song-and-dance, so just | ||
# load the tokens from that file | ||
tokens = my_file_adapter.get_token_data("transfer.api.globus.org") | ||
|
||
|
||
# RereshTokenAuthorizer and ClientCredentialsAuthorizer both use | ||
# `on_refresh` callbacks | ||
# this feature is therefore only relevant for those auth types | ||
# | ||
# auth_client is the internal auth client used for refreshes, | ||
# and which was used in the login flow | ||
# note that this is all normal authorizer usage wherein | ||
# my_file_adapter is providing the on_refresh callback | ||
auth_client = ... | ||
authorizer = globus_sdk.RefreshTokenAuthorizer( | ||
tokens["refresh_token"], | ||
auth_client, | ||
access_token=tokens["access_token"], | ||
expires_at=tokens["expires_at_seconds"], | ||
on_refresh=my_file_adapter.on_refresh, | ||
) | ||
|
||
# or, for client credentials | ||
authorizer = globus_sdk.ClientCredentialsAuthorizer( | ||
auth_client, | ||
["urn:globus:auth:transfer.api.globus.org:all"], | ||
access_token=tokens["access_token"], | ||
expires_at=tokens["expires_at_seconds"], | ||
on_refresh=my_file_adapter.on_refresh, | ||
) | ||
|
||
# and then use the authorizer on a client! | ||
tc = globus_sdk.TransferClient(authorizer=authorizer) | ||
|
||
|
||
Complete Example Usage | ||
~~~~~~~~~~~~~~~~~~~~~~ | ||
|
||
The :ref:`Group Listing With Token Storage Script <example_group_listing_with_token_storage>` | ||
provides complete and runnable example which leverages ``tokenstorage``. | ||
|
||
|
||
Adapter Types | ||
------------- | ||
|
||
.. module:: globus_sdk.tokenstorage | ||
|
||
``globus_sdk.tokenstorage`` provides base classes for building your own storage | ||
adapters, and several complete adapters. | ||
|
||
The :class:`SimpleJSONFileAdapter` is good for the "simplest possible" | ||
persistent storage, using a JSON file to store token data. | ||
|
||
:class:`MemoryAdapter` is even simpler still, and is great for writing and | ||
testing code which uses the `StorageAdapter` interface backed by an in-memory | ||
structure. | ||
|
||
The :class:`SQLiteAdapter` is more complex, for applications like the | ||
globus-cli which need to store various tokens and additional configuration. In | ||
addition to basic token storage, the :class:`SQLiteAdapter` provides for namespacing | ||
of the token data, and for additional configuration storage. | ||
|
||
Reference | ||
--------- | ||
|
||
.. autoclass:: StorageAdapter | ||
:members: | ||
:member-order: bysource | ||
:show-inheritance: | ||
|
||
.. autoclass:: MemoryAdapter | ||
:members: | ||
:member-order: bysource | ||
:show-inheritance: | ||
|
||
.. autoclass:: FileAdapter | ||
:members: | ||
:member-order: bysource | ||
:show-inheritance: | ||
|
||
.. autoclass:: SimpleJSONFileAdapter | ||
:members: | ||
:member-order: bysource | ||
:show-inheritance: | ||
|
||
.. autoclass:: SQLiteAdapter | ||
:members: | ||
:member-order: bysource | ||
:show-inheritance: | ||
The documentation which was found on this page has moved to :ref:`storage_adapters`. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe controversially I titled this section "token caching" rather than "token storage". This is to disambiguate with the subpages that provide references for specific versions of the same concept.
Alternative Structure
and you'd end up with pages titled "Token Storage", "Token Storages", and "Storage Adapters (Legacy)".