Resolver Refactor #422
Replies: 7 comments 25 replies
-
|
I really like the idea of caching for future resolutions. I wonder if we need to expose the possibility to to specify the TTL of the cache. I guess with the current proposal implementers can choose when to reuse the resolver or create a new one, but maybe we could build in support for configurable caches (see https://github.com/decentralized-identity/did-resolver#caching for thoughts on that). |
Beta Was this translation helpful? Give feedback.
-
|
Where would that theoretical standard compliant resolver live? It sounds that would be external to identity.rs? Would that be external to the IF? |
Beta Was this translation helpful? Give feedback.
-
|
We could add it, but it would just be another API for the sake of being compliant, IMO cluttering the repository. Historically, it has shown that those kinds of decisions only lead to confusion. I would rather commit something to the Universal Identity Resolver and claim compliance through that and reference it as a "reference implementation for following the spec". |
Beta Was this translation helpful? Give feedback.
-
|
I'd also like to voice some hesitation for removing a standard compliant resolver from the identity.rs repository (or even further, external to the IF). In no particular order:
|
Beta Was this translation helpful? Give feedback.
-
|
Good points, thanks @m-renaud :) I am very happy to have you respond here! Clutter: The wrapper would indeed be minimal, so that is good, but inside the API reference it will be mentioned on a similar level as the preferred and cleaner non-standard resolver, making it difficult to follow "what do I need to use"? I already find the repo quite large, as we are the largest framework in the IOTA Foundation, but we also have a LOT more planned. Immediate Feedback: Agreed, it would be nice. But as long as our API returns all compliant data structures, we already know we can provide compatibility. Honestly, I am tempted to just PR the DID standard with a request to focus on this data structure compliance, rather than an API. Maintenance & Universal Resolver: No counter-arguments =D Testing: That Test Suite was a big disappointment tbh. It's not really automated at all :/ |
Beta Was this translation helpful? Give feedback.
-
I think this could be resolved with documentation structure and formatting. The low-level and "standard" API reference pages could include at the top a: "TIP: Prefer to use the high-level API (link)".
It looks like there's only a few tens of thousands of lines of code across everything, including examples and tests. Are there specifics about the current code base organization that are tricky to navigate? I've been using a combination of
Awe darn, I'm sorry to hear that :( Do you have any specific issues/annoyances you ran into written down? It would be nice to be like: "See! The implementation complies with the spec!" |
Beta Was this translation helpful? Give feedback.
-
|
The annoyance was that you don't implement your framework, but rather copy paste your resulting DID Document into their test and they just check the data structure. |
Beta Was this translation helpful? Give feedback.
-
|
In general, this seems like a step in the right direction 👍. I agree with @eike-hass on the standard compliance. We can offer this better interface for our users, but should probably also provide a compliant resolve function.
I don't fully understand the idea here. If the I also don't get why the So unless I misunderstand, I would let let did: IotaDid = "did:iota:Gh9LYviikgCnakMtrLcCCk7F4zUsjKzZGwikRi5gMvCr".parse().unwrap();
let resolver_factory: ResolverFactory = ResolverFactory::new(client_map, resolution_options);
let resolver: Resolver = resolver_factory.resolve(did).await?;
let full_document: CoreDocument = resolver.document(); // instead of getResult()
let older_document: CoreDocument = resolver.document_at(5); // instead of getDocument()
let verification_method: DocumentProperty = resolver.dereference("#key-1")?;
// Match and see what we got
match verification_method {
DocumentProperty::VerificationMethod(method) => {},
DocumentProperty::Service(service) => {},
}
|
Beta Was this translation helpful? Give feedback.
-
|
Strings are fine, it's just another variant in the let did: IotaDid = "did:iota:Gh9LYviikgCnakMtrLcCCk7F4zUsjKzZGwikRi5gMvCr".parse().unwrap();
let resolver: Resolver = Resolver::new(client_map, resolution_options);
let result: ResolutionResult = resolver.resolve(did).await?;
match result {
// `resolve` returned a document
ResolutionResult::Document(document) => {
// Now we may want to additionally dereference a specific verification method in that document
let method: IotaVerificationMethod = document.dereference("#key-1").method()?;
}
// We "resolved" a verification method inside a document
ResolutionResult::DocumentProperty(DocumentProperty::VerificationMethod(method)) => {},
// Anything else...
_ => ()
}This gets rid of one level while we could still cache the result of reading a particular did inside the let older_doc: ResolutionResult = resolver.resolve_at(did, VersionTime::new("2021-09-29")?).await?;
// Suppose we have a DidUrl that points to a service endpoint
let did_url: DidUrl = did.to_url().set_fragment("#my-service");
// Can also resolve to strings...
let service_endpoint: String = resolver
.resolve(did_url)
.await?
.property()?
.string()?;Seems like a nicer API which should be able to do the same as the original proposal, but with one less level of indirection. |
Beta Was this translation helpful? Give feedback.
-
|
Mhm, I think my API is still misunderstood which shows it has some problems but your example has the same complexity and function calls as I suggest. I suggested that the resolving is actually done during the construction of the let did: IotaDid = "did:iota:Gh9LYviikgCnakMtrLcCCk7F4zUsjKzZGwikRi5gMvCr".parse().unwrap();
let resolver: Resolver = Resolver::new(did, resolution_options).await?;
let result: ResolutionResult = resolver.getResult();
match result {
// `resolve` returned a document
ResolutionResult::Document(document) => {
// Now we may want to additionally dereference a specific verification method in that document
let method: IotaVerificationMethod = document.dereference("#key-1").method()?;
}
// We "resolved" a verification method inside a document
ResolutionResult::DocumentProperty(DocumentProperty::VerificationMethod(method)) => {},
// Anything else...
_ => ()
}I did just realize that a lot of languages don't like async constructor calls (aka make it impossible), so I am not sure if mine even works. Although if we have a factory object, it would be possible again. The advantage of what I propose is that the other functions such as retrieving metadata always return something that makes sense, since the resolve is done during creation. In your API, the developer must always first call |
Beta Was this translation helpful? Give feedback.
-
I don't think we need a factory object, just a factory function (aka. any function which constructs an object): pub async fn resolve_document(did, resolution_options) -> ResolutionResult {
// Create the RelolutionResult here, but the async part lives outside the constructor.
}My understanding of motivation for the API change is to avoid having to re-resolve a document every time you want the document or want to dereference something else in the same document; not to minimize the amount of code you need to write to dereference one thing. I'm thinking of something like: let result: ResolutionResult = resolve_document(did, resolution_options).await?;
do_something_with(result.document());
do_something_with_method(result.dereference("#verification"));
do_something_else_with(result.dereference("#some-other-key"));
and_finally(result.dereference("/some/resource?key=val"));This ignores the complexities of |
Beta Was this translation helpful? Give feedback.
-
|
I'm not fundamentally convinced that we need this generic API that can return a full document or a part of it at the same time. I think it's fine if you need to resolve the document first, and then dereference something on it. We should be able to only allow relative DIDs on the dereference method, to avoid the case where a DIDUrl is just a DID and would thus point to the document itself. It's two operations instead of one (resolve, dereference) which is acceptable, follows the standard, and you have stronger typing, since you know what you're getting instead of having to match on the result. It feels sensible to me because it follows the typical pattern of nesting, i.e. methods are nested in a document, so you first need a document to get to the method. @m-renaud's approach is also nice, it seems similar to what I proposed. The main difference seems to be whether we want to pre-configure a resolver with a client map and resolution options once, or pass them every time. |
Beta Was this translation helpful? Give feedback.
-
|
I agree with @PhilippGackstatter here. I believe that having to match every time we call resolve would be a bit annoying. I think that resolving to a document, from a DID, would be the best approach, and we could get the properties from there. |
Beta Was this translation helpful? Give feedback.
-
|
First off, I love the idea of designing and building an API that is specialized for the languages that developers are going to be using to build applications, and not only providing a JSON constrained spec-compliant API. Thank you for starting this discussion! As a caveat to the below, I spend way less time in this code base and using the APIs then you all do, and wrote this at the end of a long day so please forgive any misunderstandings :) What I like about the proposal:
What I find confusing and/or have questions about: A. The spec is VERY specific about the difference between "resolution" and "dereferencing" (I've read several long threads about this in various forums), especially what the allowed inputs and outputs are. The current proposal conflates these two definitions which is confusing at best, and at worst could lead to an entirely incorrect understanding of the API by someone familiar with the spec
B. One API to rule them all is not always what it's cracked up to be
C. MEGA-NIT!! Is "Factory" the right term here? Usually "factory" is used in a context when you aren't specifying the type of object being created. There's nothing magical or generic going on here so another term would probably be better :) Responses to (some of) your open questions:
Since it's only needed during the As an additional option, the
Building one ourselves would answer this for us 🙃 (see my other response) Okay, that's enough for now, I have more thoughts but this is already WAY too long for this format! |
Beta Was this translation helpful? Give feedback.
-
|
Even though I've proposed the API in the other thread, I share the concerns about the spec being specific on the resolve/dereference distinction, as well as (the more important point) the ambiguity in the
This is a great point. We're just moving the "problem" from the input to the output. It's not too much to ask someone wanting to dereference a verification method, to first resolve the document and then dereference on that. It's a two-step instead of a one-step process. That's okay. Collapsing everything into a single method might be too much. The |
Beta Was this translation helpful? Give feedback.
-
|
Alright, those are excellent points. But my problem with the differentation between |
Beta Was this translation helpful? Give feedback.
-
Sure, the same way that passing
The point of the additional function ( |
Beta Was this translation helpful? Give feedback.
-
|
In case this API is going to replace the History API:
|
Beta Was this translation helpful? Give feedback.
-
|
Excellent points, we will need to include this features in the final design. |
Beta Was this translation helpful? Give feedback.
-
|
We would like to come to a conclusion on this discussion and have converged on two options that we would like some final thoughts on, before we make a decision. In the team, we voted on being spec-compliant, but that doesn't mean we can't expose a separate API that we feel is better. So with both options we will have a Option 1// did is of type did, not did_rul
let resolution: Resolution = resolve(did, resolution_options);
let document: IotaDocument = resolution.document();
let metadata: DocumentMetadata = resolution.metadata();
let history: DocumentHistory = resolution.history();
let old_document: IotaDocument = history.document_at("2021-10-01");
// ResolutionResult is an enum that wraps IotaDocument, and all kinds of properties on the document
let service: ResolutionResult = document.dereference("key-1");
let service2: ResolutionResult = old_document.dereference("key-1");
// Additionally, we have a high-level dereference method that takes a DidUrl
let dereference: ResolutionResult = dereference(did_url, resolution_options);Here we have freestanding functions Option 2// did is of type did, not did_rul
let document: IotaDocument = resolve(did, resolution_options);
// DocumentMetadata is an IotaDocument + imetadata
let metadata: DocumentMetadata = resolve_with_metadata(did, resolution_options);
// DocumentHistory is a wrapper around an array of DocumentMetadata, so it includes metadata & document
let history: DocumentHistory = resolve_with_history(did, resolution_options);
let old_document: IotaDocument = history.document_at("2021-10-01");
let service: ResolutionResult = document.dereference(did_url);Here, we have three distinct functions, with the same functionality as option 1. Each function returns more data than the method before, in the sense that Notes on both optionsWe want to have the ability to resolve documents at some point in the past, for example, to check if a credential together with an older document state was valid. In the presented approaches, the entire history needs to be computed, to then call A side discussion: In both examples, it's also imaginable to have a
I hope that includes any points raised before, if not, feel free to point them out. |
Beta Was this translation helpful? Give feedback.
-
I think the design decision here is whether the resolution API should be allowed to make more calls to the Tangle after the initial resolution or if everything should be specified up-front. |
Beta Was this translation helpful? Give feedback.
-
I think I still prefer a solution that takes advantage of the type system, rather than the possibility of calling Maybe I'm still missing something, but how about something like this then: async fn main() {
let iota_did: IotaDID = "did:iota:...".parse().unwrap();
// Build a resolver that caches calls to the tangle for this specific did
let resolver = Resolver::new(iota_did);
// resolve is a convenience method that calls resolve_at(VerstionTime::now()). Also fetch metadata.
let doc_with_metadata: MetadataDocument = resolver.resolve().with_metadata().fetch().await;
let latest_document: IotaDocument = doc_with_metadata.document();
let latest_metadata: Metadata = doc_with_metadata.metadata();
// Fetch an older document, without metadata. Metadata could be fetched if `with_metadata` is added.
let doc: IotaDocument = Resolver::resolve_at(VersionTime::at("2021-10-01T00:00:00"))
.fetch()
.await;
// Separate API, but can share the network cache of the created resolver instance
// Perhaps this should take a VersionTime parameter?
let history: DocumentHistory = resolver.resolve_history().await;
}In short: Similar to option 1, except that 1) it can make more network calls as necessary and 2) only fetches metadata when explicitly asked for. I was thinking about adding |
Beta Was this translation helpful? Give feedback.
-
I don't think metadata should be part of the discussion on being optional. From my understanding it will be a necessary part of the documents on the Tangle post-refactor that is always retrieved. |
Beta Was this translation helpful? Give feedback.
-
We can decide what to do for our own API, as long as it is somehow accessible to build the spec-compliant resolve function on top, right? One approach would be to just combine both metadata into the
Yeah, agreed. I was mainly asking because I wanted to shoehorn everything into the original API idea of having everything retrieved upfront. But as I showed in the example, I'm definitely not insisting on that. It seems to make a lot more sense to be efficient in caching the integration chain, as you mentioned, but not in network calls. @JelleMillenaar would you be okay with that? |
Beta Was this translation helpful? Give feedback.
-
|
Quick update on this. @JelleMillenaar is fine with additional network calls, to avoid having to fetch everything upfront. Given the time constraints of finishing this for 1.0, we should start to implement this soon. Even if we haven't agreed on an exact API, it seems option 1 is the direction we want to go in, which should be good enough to get started. We can hand that over to whoever is implementing it, and request additional changes during code review. |
Beta Was this translation helpful? Give feedback.
-
|
Opened #486 to address the required metadata changes. Resolver API issue: #493 |
Beta Was this translation helpful? Give feedback.
-
Note: Discussion timeboxed until Wednesday 5th of October, after which it will be converted to an issue. Discussion will still be possible, but implementation will get started.
The
didmodule contains aresolutionmodule that has two functions:resolveanddereference. These follow the suggested API format as suggested inside the W3C DID standard. The DID standard is generally a JSON format standard rather then forcing an API design on the implementers, this is different with DID Resolution, where they enforce functionalities in a certain way. I propose to deviate from the DID Resolution standard making our framework a confirming DID, DID Document, producer, consumer and method but not a conforming DID resolver or URL dereferencer. As these are split, it doesn't break our DID compatibility and I still think we can return the data-structures as they require them, just with a different API. This means a second-layer solution on top of the IOTA Identity framework CAN be a conforming DID resolver and URL dereferencer.Motivation
DID Resolution is a resource intense operation. It produces a lot of network traffic as many API requests are send too the IOTA node and every resulting IOTA message must be parsed and validated. Caching and resolving old versions of a DID Document is currently not supported in the framework and we have a separate DID History API to at least have access to the history of a DID Document.
In addition, the DID Resolution already requires a lot of work to follow the proposed data structures from the DID standard, add support of DID parameters and move several fields from the DID Document to the
didDocumentMetadata. This is a good time to review how it works and refactor if necessary.Proposal
As such I would like to propose a design for the DID resolver for within the IOTA Identity framework. Feel free to disagree or provide a counter proposal. Data formats that follow the W3C DID Resolution spec are tagged with "(W3C)" after it.
The
resolutionmodule would contain a ResolverFactory that developers can use to set the application settings such as aClientMaporresolutionOptions(W3C). In addition it has a function toresolvegiven aDID URL, returning aresolverobject. This `resolver object has done the requires DID Resolution or URL dereferencing during its creation.The
resolverobject has the following functions:getResult: Returns the expected value of type DID Document or varied depending if aDIDorDID URLwas provided.getDocument: Returns the DID Document from the resolved DID. This could take in a parameter to also return older states of the DID Document.getResolutionMetadata: ReturnresolutionMetadata(W3C) to allow compliance with DID standard.getDocumentMetadata: ReturndidDocumentMetadata(W3C) to allow compliance with DID standard. This could also take in a parameter to return the metadata of an older DID message.dereference: Given aDID URLreturns a "part of" the DID Document.It might also need to inherit more functionalities from DID History API, but I am not familiar enough with it to determine if all functionality is properly included here.
Advantages
Given the above object, we now simplify the API from
resolve,resolutionandDID Historyinto one single powerful API call. It still is able to generate all required metadata to allow the development of a W3C compliant Resolver and a single resolve action can be reused throughout an application. A major optimization that is now possible, is too allow the verification of Verifiable Credential (VC) or Verifiable Presentation (VP) to now take an existingresolverobject, removing duplicated DID resolution actions and at the same time allowing a VC or VP to verify given an "outdated" DID Document. This allows application developers to check if a VC/VP was valid during a certain time.Additional Required Changes
Move the following fields over from DID Document to
didDocumentMetadataas suggested in the spec, keep in mind that they stay on-tangle in the DID Messages:createdandupdatedpreviousMsgId(inside amethodobject (As suggested in the example)proof(inside amethodobject)Rename a few fields and parameters that are named differently (e.g.
InputMetadatashould beresolutionOptions), that should follow the spec.Inside the
ResolutionMetadatawe should probably removeresolved(no purpose?) and introduce a boolean fieldcachedthat can if true, caches the entire history. To support caching, it would need to be part ofresolutionOptions.Add support for
versionId,versionTime,serviceandrelativeRefDID Parameters.Resources
Link to any resources relevant for the task such as Issues, PRs, reference implementations, or specifications.
message_idinIotaDocument#361Accountto resolve remote DIDs #387Open Questions
resolutionOptionsbe part of the Resolver settings instead of being passed into the Resolver Factory?resolvefunctionality be removed? ([Request] AllowAccountto resolve remote DIDs #387)resolverhave to completely replace the DID History API?Beta Was this translation helpful? Give feedback.
All reactions