Skip to content

Commit 3488ff1

Browse files
paleolimbotbkietz
andauthored
chore(docs): Ensure roadmap is up-to-date (#615)
The roadmap contained some (very) oudated ideas regarding ideas for future scope. This PR updates them with more recent text based on the existing list of issues and implementation work that has happened over the past few months. --------- Co-authored-by: Benjamin Kietzman <[email protected]>
1 parent 6118e24 commit 3488ff1

File tree

1 file changed

+42
-39
lines changed

1 file changed

+42
-39
lines changed

docs/source/roadmap.md

+42-39
Original file line numberDiff line numberDiff line change
@@ -27,60 +27,63 @@ have not yet been scheduled for implementation.
2727
## C library
2828

2929
- **Type coverage**: The C library currently provides support for all types that
30-
are available via the Arrow C Data interface. When the recently-added run-end
31-
encoded (REE) types and potentially forthcoming string view/list view types
32-
are available via the Arrow C Data interface, support should be added in
33-
nanoarrow as well.
34-
- **Array append**: The `ArrowArrayAppend*()` family of functions provide a means
35-
by which to incrementally build arrays; however, there is no built-in way to
36-
append an `ArrowArrayView`, potentially more efficiently appending multiple
37-
values at once. Among other things, this would provide a route to an
38-
unoptimized filter/take implementation.
30+
are available via the Arrow C Data interface except string view/list view types.
31+
Support for these should be added in nanoarrow as well
32+
([#583](https://github.com/apache/arrow-nanoarrow/issues/583),
33+
[#616](https://github.com/apache/arrow-nanoarrow/issues/616),
34+
[#510](https://github.com/apache/arrow-nanoarrow/issues/510)).
3935
- **Remove Arrow C++ dependency for tests**: The C library and IPC extension rely
4036
on Arrow C++ for some test code that was written early in the library's development.
4137
These tests are valuable to ensure compatibility between nanoarrow and Arrow C++;
4238
however, including them in the default test suite complicates release verification
4339
for some users and prevents testing in environments where Arrow C++ does not
44-
currently build (e.g., WASM, compilers without C++17 support).
40+
currently build (e.g., WASM, compilers without C++17 support)
41+
([#619](https://github.com/apache/arrow-nanoarrow/issues/619)).
42+
- **Test verbosity**: Tests for the C library were written before testing utilities
43+
in the `nanoarrow_testing` library were available (and before there was a
44+
`nanoarrow_testing` library in which to put new ones). As a result, some of them
45+
are very verbose and can be difficult to read, which can and should be improved
46+
([#577](https://github.com/apache/arrow-nanoarrow/issues/577),
47+
[#566](https://github.com/apache/arrow-nanoarrow/issues/566)).
4548
- **C++ integration**: The existing C++ integration is intentionally minimal;
4649
however, there are likely improvements that could be made to better integrate
47-
nanoarrow into existing C++ projects.
50+
nanoarrow into existing C++ projects
51+
([#599](https://github.com/apache/arrow-nanoarrow/issues/599)).
4852
- **Documentation**: As the C library and its user base evolves, documentation
49-
needs to be refined and expanded to support the current set of use cases.
50-
51-
## IPC extension
52-
53-
- **Write support**: The IPC extension currently provides support for reading
54-
IPC streams but not writing them.
55-
- **Dictionary support**: The IPC extension does not currently support reading
56-
dictionary messages an IPC stream.
57-
- **Compression**: The IPC extension does not currently support compressed streams.
58-
59-
## Device extension
60-
61-
This entire extension is currently experimental and awaiting use-cases that will
62-
drive future development.
53+
needs to be refined and expanded to support the current set of use cases
54+
([#187](https://github.com/apache/arrow-nanoarrow/issues/187),
55+
[#497](https://github.com/apache/arrow-nanoarrow/issues/497)).
56+
- **IPC Dictionary support**: The IPC extension does not currently support reading
57+
dictionary messages an IPC stream
58+
([#622](https://github.com/apache/arrow-nanoarrow/issues/622)).
59+
- **IPC Compression support**: The IPC extension does not currently support
60+
compressed streams using per-buffer compression, although streams can be compressed
61+
outside the nanoarrow library (e.g., gzip compression of the entire stream)
62+
([#621](https://github.com/apache/arrow-nanoarrow/issues/621))
6363

6464
## R bindings
6565

66-
- **Type support**: The R bindings currently do not provide support for extension
67-
types and relies on Arrow C++ for some dictionary-encoded types.
66+
- **Conversion internals**: The initial implementation of conversion from
67+
Arrow data to R vectors was implemented in C and its verbosity makes it
68+
difficult to add support for new types. The internals should be refactored
69+
to make the conversion code easier to understand for new developers
70+
([#392](https://github.com/apache/arrow-nanoarrow/pull/392)).
71+
- **Type support**: The R bindings currently rely on the Arrow R package for
72+
conversion of some R types (e.g., list_of), and some types are not supported
73+
in nanoarrow nor the arrow R package (e.g., run-end encoding, list view, and
74+
string/binary view)
75+
([#617](https://github.com/apache/arrow-nanoarrow/issues/617)).
6876
- **ALTREP support**: A recent R release added enhanced ALTREP support such that
6977
types that convert to `list()` can defer materialization cost/allocation.
7078
Arrow sources that arrive in chunks (e.g., from a `Table` or `ChunkedArray`)
7179
currently can't be converted via any ALTREP mechanism and support could be
72-
added.
73-
- **IPC support**: The IPC reader is not currently exposed in the R bindings.
80+
added ([#219](https://github.com/apache/arrow-nanoarrow/issues/219)).
7481

7582
## Python bindings
7683

77-
- **Packaging**: The Python bindings are currently unpublished (pypi or conda) and
78-
are not included in release verification.
79-
- **Element conversion**: There is currently no mechanism to extract an element
80-
of an `ArrowArrayView` as a Python object (e.g., an `int` or `str`).
81-
- **numpy/Pandas conversion**: The Python bindings currently expose the `ArrowArrayView`
82-
but do not provide a means by which to convert to popular packages such as
83-
numpy or Pandas.
84-
- **Creating arrays**: The Python bindings do not currently provide a means by
85-
which to create an `ArrowArray` from buffers or incrementally.
86-
- **IPC support**: The IPC reader is not currently exposed in the Python bindings.
84+
- **Type support**: The Python bindings do not currently support unions,
85+
string/binary view, or list view, or run-end-encoded types. When creating
86+
Arrow arrays from iterables of Python objects, some types are not yet
87+
supported (e.g., struct, list, datetime objects)
88+
([#618](https://github.com/apache/arrow-nanoarrow/issues/618),
89+
[#620](https://github.com/apache/arrow-nanoarrow/issues/620)).

0 commit comments

Comments
 (0)