Querying architecture and design in Quartz based applications

[davekaj]

Background

The conversation in #145 sparked a deeper conversation about long term protocol query design. You can read the starting conversation there, but the intent of this new issue is to allow #145 to simply be the short term query implementation for quartz v0.1, whereas all discussion for the future protocol can happen here.

This is a very large discussion topic for crypto in general, as private transactions become mainstream, solving private querying will be one of the hardest, most important problems to solve, especially from the user experience perspective.

Discussion

Open Discussion on Design

• User Experience of our users querying - what kind of access do we expect them to need?
  • They likely have to share access with third parties (i.e. an accountant). We can generally assume the average user would only need a few different api keys
  • We have a unique scenario where there are encrypted objects like obligations that 2 parties need access to, thus we need to consider how we can let 2 users query the same data (maybe similar to Signal messaging app, which is a shared state between 2 users?)
• Querying nodes / Infrastructure for querying
  • We likely need to separate the nodes that respond to queries from the solvers with the encrypted storage. In some sort of distributed network with load balancing, possibly indexing / aggregating data
  • A reason to separate them is because you don't want queries to overload the solvers that are writing to the chain and running the graph algorithm
• Aggregate Information
  • We need a way to aggregate some data from cycles protocol. Such as - number of obligations cleared, how much cleared, etc. There are 2 obvious ways to go about it:
    • Post important aggregated data to a smart contract on-chain that is not encrypted (Seems like the easiest solution) (Also should work with a multi-solver architecture)
    • Have the aggregate data stored in the enclave, and allow only specific users to or specific values to be queried from the outside (This is complex because we need to decide how to give these permissions)
  • One thing that is not answered, is doing complex, multi-user aggregations. On a normal blockchain like Ethereum you can aggregate something like "give me all the users who trade on Uniswap that also have a Maker CDP, and sort them." We can't do that, unless we use something like Fully Homomorphic Encryption.

Suggested Designs

• API Key design (see issue 145, thanes comment)
  • Please click the link shared to see the description.
• Doing what secret network has done, but adding ORAM

Existing private querying designs to use for inspiration

• Penumbra design
  • Users must run a server that builds their own view into their txs and data
  • Doesn't seem to be a good way to allow fine tuned access to txs and data, seems it is all or nothing for access
• Secret network design
  • You create a key pair to match you account, sign that with your account, publish it to the blockchain. Secret network will then encrypt your data to that key
  • Uses an ordinary key value store and encrypts the keys and values. (We could do something like this, but we would want to add ORAM)
  • Permits are very similar to the api key design thane wrote
• Signal Messaging Design
  • Signal uses SGX+ORAM for private contact discovery. The idea is there is some metadata that they only ever store in an encrypted form (your phone number list) and in order to do private phone number suggestions, it lets you do an intersection query between your contact list and their dataset. SGX is used for just this one feature.

Open Questions / Further Discussion Needed

• In a multi solver architecture, a user could have obligations on multiple solvers. In order to get all the obligations, you would need keys/access to every solver, query all the info, and aggregate them. How can we do that with the best user experience?
• Should we provide and run the querying infrastructure, as well as releasing a local querying infra that our users could run to aggregate their own data?
• What can we release as a querying sdk / package that quartz users could use? Obviously querying infra is app dependant. Would the SDK be general enough for most apps to use?
• How will we pay / charge / meter for queries?
• How far back do we need to store state? Is it what the various tax authorities recommend (7 years in Canada)?
• Can we predict how much state will need to be stored? Is there some point we need to prune?
• Will we run into the problems that typical L1 blockchains have when there are so many txs, its hard to index them to have an up to date view of the state?
• Can we have something like a block explorer with rich data? If not, do we provide a front end that can help users view their encrypted queries that they can run locally?
• What will our database architecture be in order to facilitate the type of querying we need to offer to users? Postgres? K/V store? (Given that the data needs to be encrypted)

Action Items

• Reach out to Penumbra and Secret and ask how they have thought it through. Can start in slack, and if that's not enough, book a call. The goal is to just get an understanding of how they thought through the problem, since they have been thinking about it longer than us.
• Look into Signal (the messaging app) for any inspiration
• We should try to answer these open questions asynchronously. Some research will be needed. We can then follow up in the near future with a new meeting to discuss further.