Working on a document storage backend for indexing documents for vector and time-series, wondering about time-travel

[LrWm3]

Hey, thanks for making cozodb. It seems really cool.

I'm working on a project where I store a bunch of documents in a database and do similarity search across them.

the idea is it is meant to then integrate with an llm for both question answering and 'smart' actions similar to openais' plugins system.

A simplified & flattened version of my schema I import data into looks as follows:

-- Stores the original data from the source documents, 
CREATE TABLE source_document (
    id UUID PRIMARY KEY NOT NULL DEFAULT uuid_generate_v4(),
    source_type VARCHAR,
    source_name VARCHAR,
    source_document_extension VARCHAR,
    source_document_version VARCHAR,
    last_modified TIMESTAMPZ,
    parent_document_id VARCHAR,
    vec vector,
    self_ref VARCHAR,
    metadata JSONB,
);

I am wondering if and how I would use timetravel in cozo to retrieve the latest version of each document at a point in time in history, if such a thing is even possible.

Would I create a stored relation like so?

:create source_document {source_type: String, source_id: String, source_document_id: String, last_modified: Validity => parent_document_id: String, source_document_extension: String, vec: <F32; 3>, self_ref: String, metadata: Json}

?[source_type, 
  source_id, 
  source_document_id, 
  last_modified, 
  parent_document_id, 
  source_document_extension, 
  vec, 
  self_ref, 
  metadata] <- [
    ['git', 'id-of-my-specific-git-repo', 'git-document-id', '2020-01-01T00:00:00.000+00:00', 'git-repo-id-or-commit-hash-not-sure-yet', 'ts', vec([0.1, 0.2, 0.3]), 'https://<git-hosted-link>', parse_json('{}')]
]

:put source_document {source_type, source_id, source_document_id, last_modified => parent_document_id, source_document_extension, vec, self_ref, metadata}

Also, does timetravel work across multiple joins as well if I provide the same @ 'NOW' expression, for example?

Thinking it will be really useful to be able to search over only documents in the past for what I am working on.

Any guidance or recommendations are appreciated.

edit: made some corrections; there's also supposed to be a version field but I forgot that.