Enhancement Idea seeds

[thorwhalen]

DAG Factories

Make func_nodes (i.e. dag) factory to allow the user to use (or make) ways to efficiently
express how names and therefore connections should be made.

Connection using type annotations

A function providing Featurizer as an output can connect to (i.e. feed) functions that take Featurizers as inputs

structure through nested lists

[..., [a, b, c],...] meaning we want a -> b -> c

from python code

Make a graphviz.Digraph object based on code (string or function's body)

from a set name-consistent functions

Moved to name-based connection

ch_names

Method to change the names (roots, inner, leafs, funcs)

order

Method to make order compliant with some given order-determining list.
Ordering roots and leafs should make inputs and outputs as wished, but doesn't
carry on to sub-DAGs. Do we need more control on topological sort?

request-triggered data access

obj.prop asks obj._dag for the sub-dag to compute
said prop and if a maintained obj._scope (the cache) contains sufficient data,
it computes it (and caches it) with minimum computional path.

Basically the pattern I often use lazyprop for.

code_to_dag

(Enable code_to_dag to use wrapped function signature for the dag signature it produces)[https://github.com//issues/37]

Parse return (& yield?) statements

This can be used to post-process the DAG's output nodes, returning only some of the values, in a tuple, or a dict or something else.
Further, it's natural for a function to have a return statement, so parsing it would help make code_to_dag more natural.

Allow code to not have an assignee

Some functions are called for side effects only (store something, or forward to viz, etc.). It's a shame to have to have a return value.

Now, the thing with this is that a DAG, which is just a set of FuncNodes, will necessarily have an out, so it will store the output of the function anyway (which will probably just be a None). We probably don't want to have to change that simplicity.

[thorwhalen]

name-based connection

For example, a DAG could be created from a module, taking only

• functions defined within the module
• other imported functions "marked" (though attribute adding decorator or class wrap)

This kind of approach has echoes in domain-driven design (see this comment about that).
Here, the functions (through their names) express not only the input-output behavior, but where they are to source their input and put their output. For example, a same function, say multiply, could appear twice, once as total_price(items, price) and another as taxes(amount, tax_rate). The mechanics of what they compute are the same, but their role (or "function") within the system is different.

Such an approach enables the developer, or the no-code user, to quickly and correctly create "programs" by selecting the components that self-organize.

This list, funcs, of functions would be used, as is, to make the dag = DAG(funcs). For this to have the desirable effect, the function parameters would have to use consistent (both argument and function) names. Function names would have to reflect what var node they source (i.e. the func name would need to reflect the desired out of the func node) and argument names where to source from.

This mapping could be done with i2.wrapper tools, but perhaps it's clearer and cleaner, in this context, to do this via an explicit function definition, like so:

def name_of_out_var_node(first_source_var_node, second_source_var_node):
    return actual_underlying_function(second_source_var_node, y=first_source_var_node)  # note how argument binding is done here

In the more general Mesh case, a var node may have multiple sources and a function could be used to source multiple var nodes (or have multiple options to source a single var node).
One way to attach this information to the function itself is to add attributes to it. A decorator, such as a provides function, can be a convenient way to do so. It will be used when we don't want to rename the function to communicate the target var node information.
Also, we may want to make provides be able to list multiple names

Moreover, we can use similar approaches for FuncNode.bind. We may want to inform a function about several var node names (as an explicit list, or "relevance" scoring function) it may find what it's looking for.

Note that what has been discussed here, based on naming conventions, can also be done (with some differences) with typing.
With typing, we can express connection in a more general way, relying on the "purpose" of a var node or parameter rather than their specific name. This is what atypes.typ aims at setting up, for the signal ML domain.