Support Numpy-like arrays #397
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Several libraries exist that support a broad subset of the Numpy API, but with different underlying data structures: - CuPy for GPU arrays - Sparse arrays - Dask array for parallel and distributed arrays These all support numpy functions like np.sin directly Previously these would not work with autograd because their types and functions were not registered. However, to the extent that they support the numpy API directly (which is increasingly becoming the case) we can just hijack the existing function mappings for Numpy functions. This commit replaces two explicit type checks with more flexible tests that check for "array-like" behavior. If so then they return the corresponding numpy function.
|
Boxes and VSpaces are designed to be extensible via their I will try to code up an example of how this might look for Dask arrays. |
|
I wrote up my thoughts on this topic in this blogpost:
http://matthewrocklin.com/blog/work/2018/05/27/beyond-numpy
…On Fri, Jun 1, 2018 at 12:04 PM, Jamie Townsend ***@***.***> wrote:
Boxes and VSpaces are designed to be extensible via their .register
method. For example, the Box mapping for ndarray is registered here
<https://github.com/HIPS/autograd/blob/master/autograd/numpy/numpy_boxes.py#L50>.
Can't we just use this system to register the extra array-like types?
I will try to code up an example of how this might work for Dask arrays.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#397 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AASszM8lYydqtR43NJ7qAGn8TO0V5YFkks5t4WYmgaJpZM4T43lj>
.
|
|
There is now a NEP and an associated mailing list discussion. Engagement from this community on that mailing list would be quite welcome: http://numpy-discussion.10968.n7.nabble.com/NEP-Dispatch-Mechanism-for-NumPy-s-high-level-API-td45638.html |
agriyakhetarpal
force-pushed
the
master
branch
from
86820fd to
2f6cc22
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Several libraries exist that support a broad subset of the Numpy API,
but with different underlying data structures:
With the
__array_ufunc__protocol these can support numpy functionslike np.sin directly.
Previously these would not work with autograd because their types and
functions were not registered.
However, to the extent that they support the numpy API directly (which
is increasingly becoming the case) we can just hijack the existing
function mappings for Numpy functions.
This commit replaces two explicit type checks with more flexible tests
that check for "array-like" behavior. If so then they return the
corresponding numpy function.
There are several things wrong with this commit as it stands:
typeofwhich can probably be improvedMy initial goal here is just to start conversation and see if this is of interest.