Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge with AxisArrayConversion #4

Open
jw3126 opened this issue Feb 23, 2021 · 4 comments
Open

Merge with AxisArrayConversion #4

jw3126 opened this issue Feb 23, 2021 · 4 comments

Comments

@jw3126
Copy link

jw3126 commented Feb 23, 2021

From discourse merge this package with AxisArrayConversion. As you say we should probably merge the two packages.
So how about merge!(YAXArrayBase, AxisArrayConversion) and deprecating my package.
I think our packages have a bit different philosophy though. So here are some features/goals of AxisArrayConversion
and I would like to know whether you are open to supporting them here:

  • Should be possible to convert from and to a NamedTuple
yaxconvert(AxisArray, (axes=(x=1:2, y=1:3), values=randn(2,3))
yaxconvert(NamedTuple, some_array)
  • Package should provide helper functions to lift objects to axis array world. For instance say you have something like an axis array, but whose dimensions don't have names.
obj = (axes=(1:2, 1:3), values=reshape(1:6, 2, 3))
lift(obj) == (axes=(x1=1:2, x2=1:3), values=obj.values)
  • Package should provide helper functions, for writing axis array backend agnostic code
@meggart
Copy link
Member

meggart commented Feb 25, 2021

Thanks for opening this. Regarding your first 2 points I have created #5 which implements the interface for Named Tuples. Is this what you envisioned. I would still like to discuss if the code should remain in this package or should be renamed and moved to a different place/package.

To provide some background I have a package that provides some processing on top of large disk-based AxisArray-like arrays and initially started with my homegrown axis array type. Then I came to the point where I wanted to discard my own type and move to a different package and be independent of the actual axis array implementation, so I made this package to support different types of axis arrays and called it YAXArrayBase, because it provides the interface to work with the YAXArray.jl package.

My main concern with keeping the code inside this package would be that it might be recognized as biased towards the usage within YAXArrays, Another concern is that there is also a second interface defined for datasets, which mimic file data structures like NetCDF CF conventions which don't really belong to this concept.

So my suggestion would be to take the code inside src/axisarray and put it into a new package which might live in an organisation or think about if some of the functions defined here might become part of Arrayinterface.jl

@jw3126
Copy link
Author

jw3126 commented Feb 25, 2021

#5 is what I had in mind. There are minor details, like the default dim names, that I would personally do differently, but I can live with that.
I agree that putting a common interface in ArrayInterface.jl would be awesome. However, I am not sure if or when that will happen. I think a tiny AxisArrayInterface.jl package owned by JuliaArrays would be a good step forward. We can later merge it into ArrayInterface.jl.

@meggart
Copy link
Member

meggart commented Feb 26, 2021

Yes, having a AxisArrayInterface.jl sounds reasonable. Regarding the default dim names, I am completely open, what would be your preferred solution? I think using x,y,z does not really work because there are higher-dimensional arrays as well and we would run out of letters (at least when starting at x). We might let the user decide about their preferred fallback axis naming scheme using https://github.com/JuliaPackaging/Preferences.jl

@jw3126
Copy link
Author

jw3126 commented Feb 26, 2021

Yes, having a AxisArrayInterface.jl sounds reasonable. Regarding the default dim names, I am completely open, what would be your preferred solution?

My personal preference is x1, x2, x3... but I don't care that much. Personally, I would like to keep it simple so I am not a fan of using Preferences.jl here.
There is some discussion on moving to ArrayInterface
JuliaArrays/ArrayInterface.jl#125
https://discourse.julialang.org/t/converting-between-axisarray-like-packages/52510/14?u=jw3126
So maybe wait for two weeks and see if it leads to somewhere. If not then we create AxisArrayInterface. Sounds good?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants