Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Tensor.<func> methods like reduce, activation functions, etc #145

Open
albertz opened this issue May 1, 2022 · 0 comments
Open

Add Tensor.<func> methods like reduce, activation functions, etc #145

albertz opened this issue May 1, 2022 · 0 comments

Comments

@albertz
Copy link
Member

albertz commented May 1, 2022

PyTorch provides all tensor functions also on the Tensor type itself, doc.

This includes:

  • all the activation functions (abs, tanh, log_softmax, etc)
  • sum (like reduce_sum in TF, or our reduce)
  • matmul, etc
  • transpose, reshape, etc
  • basically everything...

TF does not do this. However, it can often lead to easier code (in my opinion). Examples:

  • indices.gather(axis=k_dim, position=i)
  • indices.range_in_axis(axis=k_dim)
  • x.length(axis=spatial_dim)
  • num.reduce(mode="max", axis=num.shape_ordered)
  • (a - b).abs().reduce(mode="mean", axis=axis)
  • data.length(axis=time_dim).reduce(mode="max", axis=nn.batch_dim)

So first we should clarify if we want this or not.

Then on the technical question how to do it:

I think PyCharm do this automatically on-the-fly when the module is loaded. But I think they provide (auto-generated) pyi files such that autocompletion works with PyCharm etc.

We could also load it on-the-fly. But when we auto-generate pyi files, I think instead we could directly auto-generate some automatic tensor mixin type or so, and then have it more direct.

Or we could explicitly/manually add all the functions. Currently we already do that manually for all the base arithmetic functions like __add__ etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant