Skip to content

Commit

Permalink
Unify column and vector view queries.
Browse files Browse the repository at this point in the history
  • Loading branch information
@orenbenkiki
orenbenkiki committed Jun 1, 2024
1 parent c21aed8 commit f508663
Show file tree
Hide file tree
Showing 10 changed files with 215 additions and 238 deletions.
2 changes: 1 addition & 1 deletion docs/v0.1.0/.documenter-siteinfo.json
View file
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"documenter":{"julia_version":"1.10.0","generation_timestamp":"2024-06-01T13:01:18","documenter_version":"1.4.1"}}
{"documenter":{"julia_version":"1.10.0","generation_timestamp":"2024-06-01T14:35:03","documenter_version":"1.4.1"}}
12 changes: 6 additions & 6 deletions docs/v0.1.0/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1832,6 +1832,12 @@ <h1 id="Index">
</a>
</li>
<li>
<a href="queries.html#Daf.Queries.full_vector_query">
<code>Daf.Queries.full_vector_query
</code>
</a>
</li>
<li>
<a href="queries.html#Daf.Queries.get_frame">
<code>Daf.Queries.get_frame
</code>
Expand All @@ -1856,12 +1862,6 @@ <h1 id="Index">
</a>
</li>
<li>
<a href="queries.html#Daf.Queries.is_query_suffix">
<code>Daf.Queries.is_query_suffix
</code>
</a>
</li>
<li>
<a href="queries.html#Daf.Queries.query_result_dimensions">
<code>Daf.Queries.query_result_dimensions
</code>
Expand Down
Binary file modified docs/v0.1.0/objects.inv
View file
Binary file not shown.
170 changes: 95 additions & 75 deletions docs/v0.1.0/queries.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -301,8 +301,10 @@ <h2 id="Construction">
<section>
<div>
<pre>
<code class="language-julia hljs">Query(query_string::AbstractString) &lt;: QueryOperation
Query(query::Query) &lt;: QueryOperation
<code class="language-julia hljs">Query(
query::Union{AbstractString, Query},
operand_only::Maybe{Type{QueryOperation}} = nothing,
) &lt;: QueryOperation
</code>
</pre>
<p>A query is a description of a (sub-)process for extracting some data from a
Expand All @@ -317,15 +319,7 @@ <h2 id="Construction">
<a href="formats.html#Daf.Formats.DafReader">
<code>DafReader
</code>
</a>, result in a scalar, vector or matrix result. A single
<a href="queries.html#Daf.Queries.Lookup">
<code>Lookup
</code>
</a> or a single
<a href="queries.html#Daf.Queries.Axis">
<code>Axis
</code>
</a> are also valid complete queries.
</a>, result in a scalar, vector or matrix result.
</p>
<p>To apply a query, invoke
<a href="queries.html#Daf.Queries.get_query">
Expand Down Expand Up @@ -403,19 +397,19 @@ <h2 id="Construction">
<code>Regex
</code> strings.
</p>
<p>As a shorthand, the query of a simple name is interpreted as a lookup of that name (e.g.,
<code>q&quot;foo&quot;
</code> is equivalent to
<code>q&quot;: foo&quot;
</code>). This is convenient when expressing suffix queries (e.g., for
<a href="queries.html#Daf.Queries.QueryColumns">
<code>QueryColumns
</code>
</a> for
<p>If the provided query string contains only an operand, and
<code>operand_only
</code> is specified, it is used as the operator (i.e.,
<code>Query(&quot;metacell&quot;)
</code> is an error, but
<code>Query(&quot;metacell&quot;, Axis)
</code> is the same as
<code>Axis(&quot;metacell&quot;)
</code>). This is useful when providing suffix queries (e.g., for
<a href="queries.html#Daf.Queries.get_frame">
<code>get_frame
</code>
</a>.
</a>).
</p>
<p>Being able to represent queries as strings allows for reading them from configuration files and letting the user input them in an application UI (e.g., allowing the user to specify the X, Y and/or colors of a scatter plot using queries). At the same time, being able to incrementally build queries using code allows for convenient reuse (e.g., reusing axis sub-queries in
<code>Daf
Expand Down Expand Up @@ -654,47 +648,65 @@ <h2 id="Functions">
</code>
</a> for some axis. This is a vector of pairs, where the key is the column name, and the value is a query that computes the data of the column.
</p>
<p>If the query is a string, then a value of
<p>The query is combined with the axis query as follows (using
<a href="queries.html#Daf.Queries.full_vector_query">
<code>full_vector_query
</code>
</a>:
</p>
<ul>
<li>
<p>If the query is the special string
<code>=
</code> is a shorthand for looking up the column name as a property (e.g.,
</code>, then the full query is
<code>axis_query |&gt; Lookup(column_name)
</code>. For example, the full query for the axis query
<code>cell &amp; batch = B1
</code> and the pair
<code>&quot;age&quot; =&gt; &quot;=&quot;
</code> is equivalent to
<code>&quot;age&quot; =&gt; q&quot;: age&quot;
</code>). If the query string is a simple name, it is a shorthand for looking up that name as a property (e.g.,
<code>&quot;Age&quot; =&gt; &quot;age&quot;
</code> is equivalent to
<code>&quot;Age&quot; =&gt; q&quot;: age&quot;
</code>).
</p>
<p>The query is typically a suffix of a query which is combined with the axis lookup. For example, if the axis is
<code>cell
</code>, the query suffix may be
<code>q&quot;: batch&quot;
</code> to lookup the batch of each cell.
</p>
<p>The query may also already be a full query. For example, if the axis is
<code>metacell
</code>, the query may be
<code>/ cell : age @ metacell %&gt; Mean
</code> to compute the mean age of the cells of each metacell.
</code> is
<code>cell &amp; batch = B1 : age
</code>.
</p>
<div class="admonition is-info">
<header class="admonition-header">Note
</header>
<div class="admonition-body">
<p>When specifying a full (
</li>
<li>
<p>If the query contains
<a href="queries.html#Daf.Queries.GroupBy">
<code>GroupBy
</code>
</a> query), if the axis is masked, (e.g.,
<code>metacell &amp; type = Bcell
</code>, it is your responsibility to include the same mask in the full query, e.g.
<code>/ cell &amp; metacell =&gt; type = Bcell : age @ metacell %&gt; Mean
</code>). Sorry for the inconvenience.
</a>, then the query must repeat any mask specified for the axis query. That is, if the axis query is
<code>metacell &amp; type = B
</code>, then the column query must be
<code>/ cell &amp; metacell =&gt; type = B @ metacell : age %&gt; Mean
</code>. Sorry for the inconvenience. TODO: Automatically inject the mask into
<a href="queries.html#Daf.Queries.GroupBy">
<code>GroupBy
</code>
</a> column queries.
</p>
</div>
</div>
<p>In both cases the (full) query must return a value for each entry of the axis.
</li>
<li>
<p>Otherwise, if the query starts with a (single) axis, then it should only contain a reduction; the axis query is automatically injected following it. That is, if the axis query is
<code>gene &amp; is_marker
</code>, then the full query for the column query
<code>/ metacell : fraction %&gt; Mean
</code> will be
<code>/ metacell / gene : fraction %&gt; Mean
</code> (the mean gene expression in all metacells). We can&#39;t just concatenate the axis query and the columns query here, is because Julia, in its infinite wisdom, uses column-major matrices, like R and matlab; so reduction eliminates the rows instead of the columns of the matrix.
</p>
</li>
<li>
<p>Otherwise, we simply concatenate the axis query and the column query. That is, of the axis query is
<code>cell &amp; batch = B1
</code> and the column query is
<code>: age
</code>, then the full query will be
<code>cell &amp; batch = B1 : age
</code>. This is the simplest and most common case.
</p>
</li>
</ul>
<p>In all cases the (full) query must return a value for each entry of the axis.
</p>
<div class="admonition is-info">
<header class="admonition-header">Note
Expand Down Expand Up @@ -727,8 +739,8 @@ <h2 id="Functions">
<header>
<a class="docstring-article-toggle-button fa-solid fa-chevron-down" href="javascript:;" title="Collapse docstring">
</a>
<a class="docstring-binding" id="Daf.Queries.query_result_dimensions" href="#Daf.Queries.query_result_dimensions">
<code>Daf.Queries.query_result_dimensions
<a class="docstring-binding" id="Daf.Queries.full_vector_query" href="#Daf.Queries.full_vector_query">
<code>Daf.Queries.full_vector_query
</code>
</a> —
<span class="docstring-category">Function
Expand All @@ -737,12 +749,22 @@ <h2 id="Functions">
<section>
<div>
<pre>
<code class="language-julia hljs">query_result_dimensions(query::Union{Query, AbstractString})::Int
<code class="language-julia hljs">full_vector_query(
axis_query::Query,
vector_query::Union{AbstractString, Query},
vector_name::Maybe{AbstractString} = nothing,
)::Query
</code>
</pre>
<p>Return the number of dimensions (-1 - names, 0 - scalar, 1 - vector, 2 - matrix) of the results of a
<code>query
</code>. This also verifies the query is syntactically valid, though it may still fail if applied to specific data due to invalid data values or types.
<p>Given a query for an axis, and some suffix query for a vector property, combine them into a full query for the vector values for the axis. This is used by
<a href="queries.html#Daf.Queries.QueryColumns">
<code>QueryColumns
</code>
</a> for
<a href="queries.html#Daf.Queries.get_frame">
<code>get_frame
</code>
</a> and also for queries of vector data in views.
</p>
</div>
</section>
Expand All @@ -751,8 +773,8 @@ <h2 id="Functions">
<header>
<a class="docstring-article-toggle-button fa-solid fa-chevron-down" href="javascript:;" title="Collapse docstring">
</a>
<a class="docstring-binding" id="Daf.Queries.is_query_suffix" href="#Daf.Queries.is_query_suffix">
<code>Daf.Queries.is_query_suffix
<a class="docstring-binding" id="Daf.Queries.query_result_dimensions" href="#Daf.Queries.query_result_dimensions">
<code>Daf.Queries.query_result_dimensions
</code>
</a> —
<span class="docstring-category">Function
Expand All @@ -761,14 +783,12 @@ <h2 id="Functions">
<section>
<div>
<pre>
<code class="language-julia hljs">is_query_suffix(query::Query)::Bool
<code class="language-julia hljs">query_result_dimensions(query::Union{Query, AbstractString})::Int
</code>
</pre>
<p>Return whether the query is actually a query suffix to be applied to some axis. Such suffix queries are used when fetching multiple data for the same axis, e.g. in
<a href="queries.html#Daf.Queries.get_frame">
<code>get_frame
</code>
</a>.
<p>Return the number of dimensions (-1 - names, 0 - scalar, 1 - vector, 2 - matrix) of the results of a
<code>query
</code>. This also verifies the query is syntactically valid, though it may still fail if applied to specific data due to invalid data values or types.
</p>
</div>
</section>
Expand Down Expand Up @@ -4039,6 +4059,12 @@ <h2 id="Index">
</a>
</li>
<li>
<a href="queries.html#Daf.Queries.full_vector_query">
<code>Daf.Queries.full_vector_query
</code>
</a>
</li>
<li>
<a href="queries.html#Daf.Queries.get_frame">
<code>Daf.Queries.get_frame
</code>
Expand All @@ -4063,12 +4089,6 @@ <h2 id="Index">
</a>
</li>
<li>
<a href="queries.html#Daf.Queries.is_query_suffix">
<code>Daf.Queries.is_query_suffix
</code>
</a>
</li>
<li>
<a href="queries.html#Daf.Queries.query_result_dimensions">
<code>Daf.Queries.query_result_dimensions
</code>
Expand Down
2 changes: 1 addition & 1 deletion docs/v0.1.0/search_index.js
View file

Large diffs are not rendered by default.

40 changes: 5 additions & 35 deletions docs/v0.1.0/views.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -501,43 +501,13 @@ <h1 id="Views">
</p>
<p>The value for vectors must be the suffix of a vector query based on the appropriate axis; a value of
<code>&quot;=&quot;
</code> is again used to expose the property as-is. That is, the value for the vector will normally start with the
<code>:
</code> (
<a href="queries.html#Daf.Queries.Lookup">
<code>Lookup
</code>
</a>) query operator.
</p>
<p>That is, specifying that
<code>axes = [&quot;gene&quot; =&gt; q&quot;/ gene &amp; marker&quot;]
</code>, and then that
<code>data = [(&quot;gene&quot;, &quot;forbidden&quot;) =&gt; q&quot;: lateral&quot;]
</code>, then the view will expose a
<code>forbidden
</code> vector property for the
<code>gene
</code> axis, by applying the combined query
<code>/ gene &amp; marker : lateral
</code> to the original
<code>daf
</code> data.
</p>
<p>This gets trickier when using a query reducing a matrix to a vector. In these cases, the value query will start with
<code>/
</code> (
<a href="queries.html#Daf.Queries.Axis">
<code>Axis
</code> is again used to expose the property as-is. This works in the same way as a column query in
<a href="queries.html#Daf.Queries.QueryColumns">
<code>QueryColumns
</code>
</a>) query operator to specify the reduced matrix axis, followed by the
<code>:
</code> (
<a href="queries.html#Daf.Queries.Lookup">
<code>Lookup
</code>
</a>) operator. When constructing the full query for the data, we can&#39;t simply concatenate the suffix to the axis query prefix; instead we need to swap the order of the axes (this is because Julia, in its infinite wisdom, uses column-major matrices, like R and matlab; so reduction eliminates the rows instead of the columns of the matrix).
</a>.
</p>
<p>That is, specifying
<p>For example, specifying
<code>axes = [&quot;cell&quot; =&gt; q&quot;/ cell &amp; type = TCell&quot;]
</code>, and then
<code>data = [(&quot;cell&quot;, &quot;total_noisy_UMIs&quot;) =&gt; q&quot;/ gene &amp; noisy : UMIs %&gt; Sum
Expand Down
Loading

0 comments on commit f508663

Please sign in to comment.