Selectors

An annotation is made up of a reference and (optionally) information about the referenced items(s). The reference is a specification of what item(s) the information is about, or simply we wish to single out: It acts as a selector.

These categories are crucially meant for us developers, so we separate concerns, invert control, and all that jazz. But the level I’m talking about here is directly meant to express the space where product design can take place.

• Selector: The abstract definition of “what to select”. The rules of selection. Example “last 200 sounds”, or “all sounds in the last 10mn (if any)”, or “all tagged sounds”, etc.
• Selection: The result of a selection. These can be used on the fly, or named/tagged for future reference and use
• DoWithSelection: Something you can do with a selection. For example, splatter it, swidget it, find similar sounds, learn an outlier model, learn a detection model (with tagged sounds of it).

Note that usually Selectors and Selections will be chained. For example, if you want to splatter tagged data, this will entail selecting data from your account, getting a selection in the form of references only (to be light weight) then if the section size is more than 200, you’ll need to use a subset selector (or filter) to choose “the best” 200 (according to the context, best means something different)

Note: An s-word! SElector (Sound Elector)

Examples

Simple pipeline

Note the node shapes that reflect the kind of node.

Example pipeline

Note in the flow below that two Selectors (i.e. selection RULES) are chained without going through a concrete selection. What is meant to convey here is that these two can be composed, resulting in an aggregate selection rule that will actually be applied. This can be done if we represent selectors in a language that is easily composable (such as mongo’s query language, and UNLIKE sql’s language which would need to be parsed).

Appendix: More details about some words

Selectors

Could call it “Selection Rule”, “Selection Logic”, “Abstract Selection” or “Selection Definition”? Usually, if the data is static (i.e. doesn’t change), a selector will give you the same selection anytime you apply it.

Swidgets

For those of you who don’t know: Swidgets are “sound widgets”; they were meant to be visual/functional items that would allow the user to view and operate on a single sound chunk (for chunks not too big). The idea was to be able to define in a settings section, what we want to see, from a menu of options, like waveform, spectrogram, play button, tag (read and/or write), timestamp, associations (asset, sound sets, etc.).

Appendix: Older rambles about selectors

This is stuff is old, so current wording and ideas may be different

A selector is a component that takes an input stream and outputs subsets of this input. There are two notable types of selectors:

• Chunkers: Outputs chunks of an input stream.
• Filters: Decides what items should go on to further processing or not. These “items” are usually chunks (of a waveform), or information about a chunks (outlier scores, detection probabilities, etc.).

This selection could be over a large corpus of stored sounds on a server, or could be an on-the-fly selection of streaming audio. The selection could be based on a stream, a sequence of (waveform) chunks thereof, FVs of chunks, contextual data (e.g. CAN, PLC, time), model outputs, etc.

Examples of selectors and use cases:

• Get the data I can access, given my credentials
• Get all data relevant with a given monitored target, for a given time frame
• Chunk a continuous signal stream using a fixed chunk size and step (sliding window)
• Chunk a continuous signal stream according to given patterns of the signal (for example, a sudden "burst" of energy, would could be a bang, a click, or such)
• Devices only send “interesting” sounds (however that’s specified) to be processed and stored
• Thin out already stored data (example, delete silence or background sounds)
• Get representative sounds
• Get outliers
• Get representative outliers (i.e. getting outliers, but avoiding having a bunch of similar outliers)
• Get more examples of targeted sounds, or that may lead to confusion between two targets. This can be because we want these sounds to be stored, or because we want to nourish a detection model after verification (of the prospect tag).
• Get sounds with given acoustic characteristics. Feature based: For example, retrieve only sounds above a given intensity, saturation, or autocorrelation, or only retrieve burst sounds.
• Get sounds occurring in a given context. This context could be as simple as time--for example, one might wish to only monitor sounds at night--or a complex formula involving other data feeds--for example “when the car is going over 60mph and the steering wheel angle is over 20 degrees”.