Skip to content

Commit

Permalink
Improve links in documentation.
Browse files Browse the repository at this point in the history
  • Loading branch information
orenbenkiki committed Apr 16, 2024
1 parent e7f675b commit 567f872
Show file tree
Hide file tree
Showing 20 changed files with 191 additions and 110 deletions.
2 changes: 1 addition & 1 deletion docs/v0.1.0/.documenter-siteinfo.json
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"documenter":{"julia_version":"1.10.0","generation_timestamp":"2024-04-16T08:13:50","documenter_version":"1.3.0"}}
{"documenter":{"julia_version":"1.10.0","generation_timestamp":"2024-04-16T09:01:55","documenter_version":"1.3.0"}}
22 changes: 13 additions & 9 deletions docs/v0.1.0/adapters.html
Original file line number Diff line number Diff line change
Expand Up @@ -322,8 +322,10 @@ <h1 id="Adapters">
</a>. If the
<code>computation
</code> was annotated by
<a href="computations.html#Daf.Computations.@computation">
<code>@computation
</code>, then its
</code>
</a>, then its
<a href="contracts.html#Daf.Contracts.Contract">
<code>Contract
</code>
Expand All @@ -342,22 +344,24 @@ <h1 id="Adapters">
<code>chain_writer
</code>
</a> and a new
<a href="formats.html#Daf.Formats.DafWriter">
<code>DafWriter
</code> to
</code>
</a> to
<code>capture
</code> the output; by default, this will be a [
</code> the output; by default, this will be a
<a href="memory_format.html#Daf.MemoryFormat.MemoryDaf">
<code>MemoryDaf
</code>]@(ref)), but it can be any function that takes a
</code>
</a>), but it can be any function that takes a
<code>name
</code> (named) parameter and returns a
<code>DafWriter
</code>.
</li>
<li>Once the
<code>computation
</code> is done, create a new view of the output, which presents the subset of the output data properties you are interested in, with the names you would like to store them as. Again, if the
<code>computation
</code> was annotated by
</code> is done, create a new view of the output, which presents the subset of the output data properties you are interested in, with the names you would like to store them as. Again, if the computation was annotated by
<a href="computations.html#Daf.Computations.@computation">
<code>@computation
</code>
Expand Down Expand Up @@ -395,8 +399,8 @@ <h1 id="Adapters">
<p>If the names of the properties in the input already match the contract of the
<code>computation
</code>, you can pass the data set directly as the
<span>$view$
</span>. The call to
<code>view
</code>. The call to
<code>adapter
</code> may still be needed to filter or rename the
<code>computation
Expand Down
24 changes: 16 additions & 8 deletions docs/v0.1.0/anndata_format.html
Original file line number Diff line number Diff line change
Expand Up @@ -245,8 +245,8 @@ <h1 id="AnnData-Format">
<p>Import/export
<code>Daf
</code> data from/to
<code>AnnData
</code>.
<a href="https://pypi.org/project/anndata/">AnnData
</a>.
</p>
<p>Due to the different data models, not all the content of
<code>AnnData
Expand Down Expand Up @@ -388,18 +388,24 @@ <h1 id="AnnData-Format">
</code> invariant that all accessed data is column-major, at least for square matrices. This is bad because the invariant greatly simplifies
<code>Daf
</code> client code. Forcing clients to check the data layout and calling
<a href="matrix_layouts.html#Daf.MatrixLayouts.relayout!">
<code>relayout!
</code> would add a lot of error-prone boilerplate to our users.
</code>
</a> would add a lot of error-prone boilerplate to our users.
</li>
<li>We can
<a href="matrix_layouts.html#Daf.MatrixLayouts.relayout!">
<code>relayout!
</code> the data when copying it between
</code>
</a> the data when copying it between
<code>AnnData
</code> and
<code>Daf
</code>. This is bad because, it would force us to duplicate the data. More importantly, there is typically a good reason for the layout of the data. For example, assume a directed graph between cells. A common way to store is is to have a square matrix where each row contains the weights of the edges originating in one cell, connecting it to all other cells. This allows code to efficiently &quot;loop on all cells; loop on all outgoing edges&quot;. If we
</code>. This is bad because it would force us to duplicate the data. More importantly, there is typically a good reason for the layout of the data. For example, assume a directed graph between cells. A common way to store is is to have a square matrix where each row contains the weights of the edges originating in one cell, connecting it to all other cells. This allows code to efficiently &quot;loop on all cells; loop on all outgoing edges&quot;. If we
<a href="matrix_layouts.html#Daf.MatrixLayouts.relayout!">
<code>relayout!
</code> the data, then such a loop would become extremely inefficient.
</code>
</a> the data, then such a loop would become extremely inefficient.
</li>
<li>We can return the transposed matrix from
<code>Daf
Expand All @@ -410,7 +416,7 @@ <h1 id="AnnData-Format">
</code> in Julia).
</li>
</ul>
<p>Having to pick between these bad options, we chose the last one as the lesser evil. The assumption is that Julia code is written separately from the Python code anyway. If the same algorithm is implemented in both systems, it would work (efficiently!), as long as the developer read this warning and flipped the order of the indices, that is.
<p>Having to pick between these bad options, we chose the last one as the lesser evil. The assumption is that Julia code is written separately from the Python code anyway. If the same algorithm is implemented in both systems, it would work (efficiently!) - that is, as long as the developer read this warning and flipped the order of the indices.
</p>
<p>We do
<strong>not
Expand All @@ -423,8 +429,10 @@ <h1 id="AnnData-Format">
</code> with the row-major data from
<code>AnnData
</code> and if asked for the outher layout, will
<a href="matrix_layouts.html#Daf.MatrixLayouts.relayout!">
<code>relayout!
</code> it (and store/cache the result).
</code>
</a> it (and store/cache the result).
</p>
</div>
</section>
Expand Down
22 changes: 15 additions & 7 deletions docs/v0.1.0/chains.html
Original file line number Diff line number Diff line change
Expand Up @@ -273,8 +273,10 @@ <h1 id="Chains">
<code>DafReader
</code>
</a>s, presenting them as a single
<a href="formats.html#Daf.Formats.DafReader">
<code>DafReader
</code>. When accessing the content, the exposed value is that provided by the last data set that contains the data, that is, later data sets can override earlier data sets. However, if an axis exists in more than one data set in the chain, then its entries must be identical. This isn&#39;t typically created manually; instead call
</code>
</a>. When accessing the content, the exposed value is that provided by the last data set that contains the data, that is, later data sets can override earlier data sets. However, if an axis exists in more than one data set in the chain, then its entries must be identical. This isn&#39;t typically created manually; instead call
<a href="chains.html#Daf.Chains.chain_reader">
<code>chain_reader
</code>
Expand Down Expand Up @@ -315,16 +317,18 @@ <h1 id="Chains">
<code>DafReader
</code>
</a> data, presenting them as a single
<a href="formats.html#Daf.Formats.DafWriter">
<code>DafWriter
</code>. This acts similarly to
</code>
</a>. This acts similarly to
<a href="chains.html#Daf.Chains.chain_reader">
<code>chain_reader
</code>
</a>, but requires the final entry to be a
</a>, but requires the final entry in the chain to be a
<a href="formats.html#Daf.Formats.DafWriter">
<code>DafWriter
</code>
</a>. Any modifications or additions to the chain are directed at this final writer.
</a>. Any modifications or additions to the chain are directed only at this final writer.
</p>
<div class="admonition is-info">
<header class="admonition-header">Note
Expand Down Expand Up @@ -390,11 +394,15 @@ <h1 id="Chains">
<a href="formats.html#Daf.Formats.DafReader">
<code>DafReader
</code>
</a> data, with a final [
</a> data, with a final
<a href="formats.html#Daf.Formats.DafWriter">
<code>DafWriter
</code>], presenting them as a single
</code>
</a>, presenting them as a single
<a href="formats.html#Daf.Formats.DafWriter">
<code>DafWriter
</code>. When accessing the content, the exposed value is that provided by the last data set that contains the data, that is, later data sets can override earlier data sets (where the writer has the final word). However, if an axis exists in more than one data set in the chain, then its entries must be identical. This isn&#39;t typically created manually; instead call
</code>
</a>. When accessing the content, the exposed value is that provided by the last data set that contains the data, that is, later data sets can override earlier data sets (where the writer has the final word). However, if an axis exists in more than one data set in the chain, then its entries must be identical. This isn&#39;t typically created manually; instead call
<a href="chains.html#Daf.Chains.chain_reader">
<code>chain_reader
</code>
Expand Down
40 changes: 30 additions & 10 deletions docs/v0.1.0/copies.html
Original file line number Diff line number Diff line change
Expand Up @@ -291,12 +291,16 @@ <h1 id="Copies">
</code> from some
<code>source
</code>
<a href="formats.html#Daf.Formats.DafReader">
<code>DafReader
</code> into some
</code>
</a> into some
<code>destination
</code>
<a href="formats.html#Daf.Formats.DafWriter">
<code>DafWriter
</code>.
</code>
</a>.
</p>
<p>The scalar is fetched using the
<code>name
Expand Down Expand Up @@ -341,12 +345,16 @@ <h1 id="Copies">
</code> from some
<code>source
</code>
<a href="formats.html#Daf.Formats.DafReader">
<code>DafReader
</code> into some
</code>
</a> into some
<code>destination
</code>
<a href="formats.html#Daf.Formats.DafWriter">
<code>DafWriter
</code>.
</code>
</a>.
</p>
<p>The axis is fetched using the
<code>name
Expand Down Expand Up @@ -390,12 +398,16 @@ <h1 id="Copies">
<p>Copy a vector from some
<code>source
</code>
<a href="formats.html#Daf.Formats.DafReader">
<code>DafReader
</code> into some
</code>
</a> into some
<code>destination
</code>
<a href="formats.html#Daf.Formats.DafWriter">
<code>DafWriter
</code>.
</code>
</a>.
</p>
<p>The vector is fetched using the
<code>axis
Expand Down Expand Up @@ -454,12 +466,16 @@ <h1 id="Copies">
<p>Copy a matrix from some
<code>source
</code>
<a href="formats.html#Daf.Formats.DafReader">
<code>DafReader
</code> into some
</code>
</a> into some
<code>destination
</code>
<a href="formats.html#Daf.Formats.DafWriter">
<code>DafWriter
</code>.
</code>
</a>.
</p>
<p>The matrix is fetched using the
<code>rows_axis
Expand Down Expand Up @@ -529,12 +545,16 @@ <h1 id="Copies">
<p>Copy all the content of a
<code>source
</code>
<a href="formats.html#Daf.Formats.DafReader">
<code>DafReader
</code> into a
</code>
</a> into a
<code>destination
</code>
<a href="formats.html#Daf.Formats.DafWriter">
<code>DafWriter
</code>. If
</code>
</a>. If
<code>overwrite
</code>, this will overwrite existing data in the target. If
<code>relayout
Expand Down
24 changes: 22 additions & 2 deletions docs/v0.1.0/formats.html
Original file line number Diff line number Diff line change
Expand Up @@ -281,8 +281,28 @@ <h1 id="Formats">
<p>In general, storage format objects are as &quot;dumb&quot; as possible, to make it easier to support new storage formats. The required functions implement a glorified key-value repository, with the absolutely minimal necessary logic to deal with the separate property namespaces listed above.
</p>
<p>For clarity of documentation, we split the type hierarchy to
<code>DafWriter &lt;: FormatWriter &lt;: DafReader &lt;: FormatReader
</code>.
<a href="formats.html#Daf.Formats.DafWriter">
<code>DafWriter
</code>
</a>
<code>&lt;:
</code>
<a href="formats.html#Daf.Formats.FormatWriter">
<code>FormatWriter
</code>
</a>
<code>&lt;:
</code>
<a href="formats.html#Daf.Formats.DafReader">
<code>DafReader
</code>
</a>
<code>&lt;:
</code>
<a href="formats.html#Daf.Formats.FormatReader">
<code>FormatReader
</code>
</a>.
</p>
<p>The functions listed here use the
<a href="formats.html#Daf.Formats.FormatReader">
Expand Down
8 changes: 5 additions & 3 deletions docs/v0.1.0/h5df_format.html
Original file line number Diff line number Diff line change
Expand Up @@ -249,10 +249,12 @@ <h1 id="H5DF-Format">
</code> data in HDF5 files, which can be used to contain &quot;anything&quot;, as HDF5 is essentially &quot;a filesystem inside a file&quot;, with &quot;groups&quot; instead of directories and &quot;datasets&quot; instead of files. Therefore HDF5 is very generic, and there are various specific formats which use specific internal structure to hold some data in it - for example,
<code>h5ad
</code> files have a specific internal structure for representing
<code>AnnData
</code> objects. To represent
<a href="https://pypi.org/project/anndata/">AnnData
</a> objects. To represent
<code>Daf
</code> data in HDF5 storage, we use the following internal structure (which is
</code> data in HDF5 storage, we use the following internal structure (which
</p>
<p>is
<strong>not
</strong> compatible with
<code>h5ad
Expand Down
4 changes: 2 additions & 2 deletions docs/v0.1.0/matrix_layouts.html
Original file line number Diff line number Diff line change
Expand Up @@ -287,8 +287,8 @@ <h1 id="Matrix-layouts">
</a>-major data, the values of each row are laid out consecutively in memory (each row is a single contiguous vector). In contrast, the values of each column are stored far apart from each other. In this case, summing columns would be slow, and summing rows would be fast.
</p>
<p>This is much simpler than the
<code>ArrayLayouts
</code> module which attempts to fully describe the layout of N-dimensional arrays, a much more ambitious goal which is an overkill for our needs.
<a href="https://github.com/JuliaLinearAlgebra/ArrayLayouts.jl">ArrayLayouts
</a> module which attempts to fully describe the layout of N-dimensional arrays, a much more ambitious goal which is an overkill for our needs.
</p>
<div class="admonition is-info">
<header class="admonition-header">Note
Expand Down
18 changes: 12 additions & 6 deletions docs/v0.1.0/queries.html
Original file line number Diff line number Diff line change
Expand Up @@ -1663,9 +1663,11 @@ <h2 id="Syntax">
<a href="queries.html#Daf.Queries.AXIS_MASK">
<code>AXIS_MASK
</code>
</a>* [
</a>*
<a href="queries.html#Daf.Queries.VECTOR_LOOKUP">
<code>VECTOR_LOOKUP
</code>]
</code>
</a>
<a href="queries.html#Daf.Queries.VECTOR_FETCH">
<code>VECTOR_FETCH
</code>
Expand Down Expand Up @@ -1715,9 +1717,11 @@ <h2 id="Syntax">
</code>
</a> )?
</p>
<p>A [
<p>A
<a href="queries.html#Daf.Queries.Lookup">
<code>Lookup
</code>] of a vector property (e.g.,
</code>
</a> of a vector property (e.g.,
<code>/ cell : type
</code> will return the type of each cell).
</p>
Expand Down Expand Up @@ -3383,9 +3387,11 @@ <h2 id="Query-Operators">
<code>: version || 1.0
</code>).
</p>
<p>If any of the axes has a single entry selected using [
<p>If any of the axes has a single entry selected using
<a href="queries.html#Daf.Queries.IsEqual">
<code>IsEqual
</code>]@(ref), this will reduce the dimension of the result (e.g.,
</code>
</a>, this will reduce the dimension of the result (e.g.,
<code>/ cell / gene = FOX1 : UMIs
</code> is a vector, and both
<code>/ cell = C1 / gene = FOX1 : UMI
Expand Down
16 changes: 12 additions & 4 deletions docs/v0.1.0/readers.html
Original file line number Diff line number Diff line change
Expand Up @@ -890,16 +890,24 @@ <h2 id="Matrix-properties">
<p>If
<code>relayout
</code> (the default), then if the matrix is only stored in the other memory layout (that is, with flipped axes), then automatically call
<a href="matrix_layouts.html#Daf.MatrixLayouts.relayout!">
<code>relayout!
</code> to compute the result. If
<code>daf isa DafWriter
</code>, then store the result for future use; otherwise, just cache it as
</code>
</a> to compute the result. If
<code>daf
</code> isa
<a href="formats.html#Daf.Formats.DafWriter">
<code>DafWriter
</code>
</a>, then store the result for future use; otherwise, just cache it as
<a href="formats.html#Daf.Formats.CacheType">
<code>MemoryData
</code>
</a>. This may lock up very large amounts of memory; you can call
<a href="formats.html#Daf.Formats.empty_cache!">
<code>empty_cache!
</code> to release it.
</code>
</a> to release it.
</p>
<p>This first verifies the
<code>rows_axis
Expand Down
2 changes: 1 addition & 1 deletion docs/v0.1.0/search_index.js

Large diffs are not rendered by default.

Loading

0 comments on commit 567f872

Please sign in to comment.