Distinguish between GIDCollections with or without metadata

[stinebuu]

This is a continuation of a discussion carried out on e-mail by @babsey, @heplesser, @hakonsbm and me. See also #1, #4 and nest#455 for other discussions on NEST-3 semantics.

The issue concerns two points about distinguishing between a plain GIDCollection and a GIDCollection with metadata. So far the only collection with metadata is layers.

Show metadata type when printing

We should have a way of discerning if a GIDCollection has metadata or not, and if it has metadata, what kind. A way to distinguish between the two would be to show the information while printing the GIDCollection. Suggestions to printing outputs are

[[type=plain, model=iaf_psc_alpha, size=100 (101..200)]] 
[[type=layer, model=iaf_psc_alpha, size=100 (101..200)]] 

or, closer to normal Python print rules

GIDCollection(model=iaf_psc_alpha, size=100 (101..200)])
LayerGIDCollection(model=iaf_psc_alpha, size=100 (101..200)])

Add property for metadata information and let get() return node information

Right now get() returns something different if we have a layer or if we have a plain GIDCollection. On a layer, get() returns layer information, extent, rows etc, while it returns information about the nodes in a normal GIDCollection:

a = nest.Create('iaf_psc_alpha', 100)
b = topo.CreateLayer({'elements': 'iaf_psc_alpha', 'rows': 10, 'columns': 10})

a[0].get()    # returns specs of the first node
b.get()       # returns specs of the layer
b.get('nodes').get()       # returns specs of all nodes in this layer
b[0].get('nodes').get()    # returns specs of the first node in this layer

But, as a layer is a GIDCollection (with added metadata) this might not be what one anticipates. It could therefore be better if instead of using get() to return layer information, we add a property returning metadata information for GIDCollections with metadata, something like

a[0].get()  # specs of first node

b.get()               # specs of all nodes in layer
b[0].get()            # specs of first node in layer
b.geometry            # property providing the geometry
b.geometry['extent']  # extent of layer

The property name (geometry) could depend on the metadata type.

[jougs]

@stinebuu: I like the print-out

GIDCollection(model=iaf_psc_alpha, size=100 (101..200)])
LayerGIDCollection(model=iaf_psc_alpha, size=100 (101..200)])

a lot! Regarding the semantics of get() I agree that the current ones are confusing. What about just adding a property metadata, which provides access to a dictionary with the metadata? That would ease to write code that iterates the metadata.

[clinssen]

Re the printing: I just wanted to observe that the printing style might suggest the presence of a certain underlying data structure. Printing GIDCollection and LayerGIDCollection suggests an inheritance hierachy, while printing type=plain or layer suggests a data field.

Re getting properties from the "layer" itself or from the nodes within the layer, is the solution proposed by @hep here under "Inhomogenous collections", point 4.iii, workable? #1 (comment)

[stinebuu]

I tend to agree with @clinssen about the printing, and is a bit hesitant about suggesting a hierarchy where there is none. I do see that that way of printing makes it extremely easy to spot the difference between GIDCollection with or without metadata though.

@clinssen Regarding your last point, about point 4.iii. It is not really applicable for layers, because a layer is a Primitive GIDCollection, and not an Inhomogenous (or composite) one. That is because with nest-3, layers can only contain one element type, so all nodes in the layer are of the same model. And so layers are automatically a Primitive GIDCollection. It is good you bring it up though, because we (or at least I) have not really thought of composite GIDCollections when working on get(). Right now an exception is thrown, but this is something we should fix.

[stinebuu]

We discussed printing at NMBU yesterday. To make sure we don't suggest an inheritance hierarchy, it might be an idea to merge the two print ideas. That is, show the type, but keep the pythonian print:

GIDCollection(type=plain, model=iaf_psc_alpha, size=100, (101..200))
GIDCollection(type=layer, model=iaf_psc_alpha, size=100, (101..200))

Another question that has come up, is what we then would do with Composite GIDCollections. Printing GIDCollection(...) for each primitive would be messy I think. Adding the model, size, and GID range for each primitive might work:

GIDCollection(type=plain, size=50:
              model=iaf_psc_alpha, size=10, (1..10)
              model=iaf_psc_delta, size=20, (11..30)
              model=iaf_psc_exp, size=10, (31..40)
              model=iaf_cond_alpha, size=10, (41..50))

We could also add brackets around each model=iaf_psc_alpha, size=10, (1..10) to distinguish each primitive more. 🎉

[heplesser]

@stinebuu Looks nice to me. Instead of brackets I would suggest semicolons for partitioning. I would also suggest to change type to metadata as in the following examples. I will discuss this in a separate comment below.

GIDCollection(metadata=None, size=50:
              model=iaf_psc_alpha, size=10, (1..10);
              model=iaf_psc_delta, size=20, (11..30);
              model=iaf_psc_exp, size=10, (31..40);
              model=iaf_cond_alpha, size=10, (41..50))

[heplesser]

@jougs @clinssen Concerning the name of the property carrying metadata information, I don't think that using metadata as property name really helps much, because to use it you would still need to know which keys are valid for that metadata structure. Iterating like

for gc in net:
   e = gc.metadata['extent']

would only work if all entries in net are topology layers. At the same time, metadata is less informative for the reader of the code, so I would suggest using geometry as an informative name when the metadata is that for a topology layer, and something else if the GC, e.g., presents a ROS interface with other metadata. In print, we could then do

GIDCollection(metadata=None, model=iaf_psc_alpha, size=100, (101..200))
GIDCollection(metadata=geometry, model=iaf_psc_alpha, size=100, (101..200))

We can discuss if geometry should be a string 'geometry' of maybe a different name, e.g., spatial. For creation, I would then actually propose not to have

Create()    # plain layer
CreateSpatial()     # layer with spatial metadata

but

Create()    # plain layer
Create(..., spatial={...})   # layer with spatial metadata

I think this leads to a consistent and reasonably pythonesque UI.

[jougs]

@stinebuu, @heplesser: I very much like the final state you arrived at and would be fine if you implemented it that way.

[stinebuu]

This issue is fixed by #13 and #14