introduce methods close and query in the base API and refactor accordingly

[michaelsbradleyjr]

The goal of this PR is to make Datastore and derived types more flexible, e.g. in nim-libp2p-dht we want to specify providers: Datastore instead of providers: SQLiteDatastore and have the freedom to choose one implementation or another without having to change the type definition (see codex-storage/nim-codex-dht#41 (comment)).

method close is introduced in a similar manner as was done for BlockStore in codex-storage/nim-codex#160.

method query required a bit more experimentation, but the end result seems viable. The challenge is that Nim's iterator does not support the dynamic dispatch of method. However, we can specify a single return type (see type QueryImpl in this PR) for implementations of method query, and then in the returned queryImpl iterator use object conversion to get back to the correct specific type derived from Datastore.

I did a small amount of additional cleanup, e.g. it's not necessary to specify the async pragma in base methods of Datastore.

In the tests for SQLiteDatastore, in some places I had to change the name of a variable key to something else (I chose rkey) otherwise I was getting compile-time errors from clang/gcc:

CC: datastore/test_sqlite_datastore.nim
/path/to/.cache/nim/test_all_d/@mdatastore@stest_sqlite_datastore.nim.c:1087:39: error: duplicate member 'key89'
tyObject_Key__6b3P9aUji2WzyTKPwAJWNUQ key89;
                                      ^
/path/to/.cache/nim/test_all_d/@mdatastore@stest_sqlite_datastore.nim.c:1007:39: note: previous declaration is here
tyObject_Key__6b3P9aUji2WzyTKPwAJWNUQ key89;
                                      ^
1 error generated.
Error: execution of an external compiler program 'clang -c  -w -ferror-limit=3 -pthread -DSQLITE_ENABLE_COLUMN_METADATA   -I'/path/to/.choosenim/toolchains/nim-#devel/lib' -I/path/to/repos/nim-datastore/tests -o /path/to/.cache/nim/test_all_d/@mdatastore@stest_sqlite_datastore.nim.c.o /path/to/.cache/nim/test_all_d/@mdatastore@stest_sqlite_datastore.nim.c' failed with exit code: 1

I've seen similar previously, I believe it's owing to a bug/limitation in Nim's compiler.

---

Closes #28.

[dryajov]

I don't think there is such a thing as a * glob, in the key semantics. A key already has hierarchy implied and allows /a would mean to match all the keys with prefix a, a.k.a do a * match.

[michaelsbradleyjr]

That goes back to #19. The key semantics are a bit underspecified in various respects, and adopting SQLite's GLOB syntax seemed natural. I was going to add some review comments on #26 but it's already been merged, so I'll create an issue, and/or we can discuss in discord, etc.

[dryajov]

The key semantics are pretty well defined, which is what the query interface uses, there are no special semantics outside of the key itself + some filtering facilities. The original python implementation and it's derivations take a very straightforward approach to deriving this semantics, which are -

(pseudocode)

• Keys are hierarchical, and each level of the key gets more and more specific
  • /a - select * from <table> where key like '/a'%
  • /a/b - select * from <table> where key like '/a/b'%
• You can specify keys with namespaces /a/[field]:[name]
  • /a/name:Bob - select * from <table> where key like '/a/name:Bob'%

They query interface then allows you to add filters.

Consider a key such as - /a/b/name:Bob/age:33, to filter by name and age we would do something like:

• Key.init('/a/b') and Filter.init('name', 'Bob', '==') and Filter.init('age', '30', '=>')
  • This roughly translates into
    • select * from <table> where name = 'Bob' and age => 30 (select substr(...) as name, substr(...) as age from <table> where key like '/a/b'%)