Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extensibility flip #1101

Merged
merged 17 commits into from
Nov 2, 2022
Merged

Extensibility flip #1101

merged 17 commits into from
Nov 2, 2022

Conversation

dsainati1
Copy link
Contributor

Adds FLIP proposing adding Extensions to Cadence. See discussion in https://forum.onflow.org/t/extensibility/622 and onflow/cadence#357


For contributor use:

  • Targeted PR against master branch
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
  • Updated relevant documentation
  • Re-reviewed Files changed in the Github PR explorer
  • Added appropriate labels

@vercel
Copy link

vercel bot commented Aug 22, 2022

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated
flow-docs ✅ Ready (Inspect) Visit Preview Aug 22, 2022 at 8:06PM (UTC)

@bluesign
Copy link
Contributor

I am not sure of benefit vs cost here, considering what we allow is very limited.

Technically, we seem to be wrapping object via extension ( please correct me if I am wrong )

@dsainati1
Copy link
Contributor Author

I am not sure of benefit vs cost here, considering what we allow is very limited.

Are you saying you don't think this is worth doing?

@bluesign
Copy link
Contributor

@dsainati1 I mean this opens up too many edge cases and complicated problems. Not sure if they can be covered easily.

Hence, if the original type T declared a priv field named foo, an extension E of T would not be able to declare any fields or methods named foo, even though foo is not accessible to E

for example if you make an extension, then later some time composite adds a method named ‘foo’ , extension is broken (even extension owner didn’t change anything)

Another problems can be order of extensions, removing just one extension, requiring extension to use in another extension etc.

All those brings many edge cases, then we will hit the case of some resources don’t want extensions to extend them (e.g. TopShot) which don’t allow derivative works.

I love the concept of extensions, but seems like is really complicated in the current state to implement them.

@dsainati1
Copy link
Contributor Author

for example if you make an extension, then later some time composite adds a method named ‘foo’ , extension is broken (even extension owner didn’t change anything)

This is a fair concern, but also one that exists already in the system. If your composite conforms to an interface that you import from another contract, if the interface owner adds a method that can break your composite even though you didn't do anything. This is simply a fact of life when writing code that depends on someone else's code, I think. The right solution here would be to add a feature like contract versioning that would solve this problem generally, rather than limiting the language features we provide to users.

Another problems can be order of extensions, removing just one extension, requiring extension to use in another extension etc.

Yea, I think to make this tractable extensions will have to be order sensitive; you can only "push" an extension to the top of the "stack" or "pop" the most recently added one off. This simplifies behavior and also sets us up easily to later add the ability to have extensions require others or to allow extended types to be further extended.

we will hit the case of some resources don’t want extensions to extend them (e.g. TopShot) which don’t allow derivative works.

I was under the impression adding signatures to TopShot moments was one of the primary use cases for this feature. Am I wrong here? Extended versions of types would not be the same type as the original, so an extension of a TopShot moment would be no derivative of a TopShot moment than a contract that functionally duplicates the logic.

I certainly agree that this is a complex feature and will need careful consideration during design and implementation, but that isn't a reason not to do it; this would significantly improve the life of contract developers and enable an enormous number of new contracts and entire new paradigms.

@bluesign
Copy link
Contributor

bluesign commented Aug 29, 2022

Btw my initial comment was not to dismiss, I am fan of this idea from the beginning, just somehow I feel it needs to be more powerful in my opinion, if we limit the extensions capabilities too much, it will be simple wrapper.

I was under the impression adding signatures to TopShot moments was one of the primary use cases for this feature. Am I wrong here?

I mean as an example I saw that a lot, but they are too protective on the IP, so I am not so sure.

I certainly agree that this is a complex feature and will need careful consideration during design and implementation, but that isn't a reason not to do it; this would significantly improve the life of contract developers and enable an enormous number of new contracts and entire new paradigms.

I totally agree.

This is a fair concern, but also one that exists already in the system. If your composite conforms to an interface that you import from another contract

I think here is situation a bit reversed, in my imagination, here we have NFT has some extensions and NFT is hosting them somehow. So maybe isolating them from each other and NFT can be nice.

I mean something like nested resources:

pub struct S {}

pub extension E for S {
    pub let x: String
    init(_ x: String) {
        self.x = x
    }
}

x can be accessed like S.E.x and somehow E can access S fields with something like S.some_field

I think most powerful usage of extensions will be to extend the native functionality ( by hooking some functions ) So somehow extension should be able to override the existing methods.

@dsainati1
Copy link
Contributor Author

dsainati1 commented Aug 29, 2022

I think most powerful usage of extensions will be to extend the native functionality ( by hooking some functions ) So somehow extension should be able to override the existing methods.

This is an interesting idea, which would completely change the fundamental behavior of extensions as outlined in this particular proposal. At the moment the extension are designed such that an extended version of a type is still a valid instance of the original type; if you add a hat to a CryptoKitty, your Kitty still functions as a regular Kitty for any and all applications that use Kitties that don't know about hats. Similarly, if you attach a signature to your TopShot moments, your moments still function as moments for all contracts that interact with regular moments, but also can be used in places where a signature is expected as well.

Were we to allow extensions to override existing methods, this property would no longer hold; an extended Kitty could no longer be trusted to function anything like a Kitty at all. I'm interested in the idea, but I'm curious what powerful usage this would enable that is worth sacrificing this property.

@bluesign
Copy link
Contributor

bluesign commented Aug 29, 2022

Yeah this works with isolation of the extensions mostly, standalone it is problematic.

Let's say you have Kitty. And someone developed some KittyHat Extension.

Technically you will have something like this :

var kitty <- Kitties.GiveMeKitty() 
var hat <- KittyHats.GiveMeHat()
kitty.extend(<-hat) 

// kitty will be type of CryptoKitty
// kitty.kittyHat will be type of CryptoKitty+KittyHats

Here kitty.kittyHat for example can have different MetadataView. ( Putting a hat on kitty item possibly :) )

Or if we have meow() method on CryptoKitty, CryptoKitty+PirateTalk can have different meow()

Main issue is we relied on functions too much lately ( with metadata, new NFT standard etc ) extending without extending the functions will be little problematic.

I am more thinking like what would be cool to extend TopShot, FlowToken Vault etc ? then trying to get back to technical realm. For Vault, I would like to apply an extension to something on deposit for example ( send to other people, or convert to USDC etc )

@bluesign
Copy link
Contributor

bluesign commented Sep 2, 2022

Problem here was type system, ex: extended2 will be subtype of base or not ?

In the FLIP as it is currently, extended2 would be type Base with Extras, Extras2, which would be a subtype of Base; i.e. you can use it anywhere a Base is expected, as all the methods of Base would still work as normal and there is no risk of overriding.

If I am understanding your version correctly though @bluesign, here extended2 would still have type Base and also a list of extensions on it that would have type Base+Extras and Base+Extras2?

ah I was answering to @siddthesquid's example. ( wrapper kind of )

In my case, I don't think extension should be a specific type, it should be managed with hasExtension instead of isInstance

@siddthesquid
Copy link

siddthesquid commented Sep 2, 2022

Or is the difference that in your example the reference to m is separate from the self reference of the extension?

@dsainati1 I completely overlooked that the extending resources have self. to access their parent members, in which case your model is much cleaner. Also, I didn't mean to override in Extensions.Public - those were meant to be renamed to the members defined in that resource (c and h), my bad!

Problem here was type system, ex: extended2 will be subtype of base or not ?

@bluesign It would have been a subtype of whatever it was wrapping. So I was thinking it could also just be Base.Public (and not even Base.Thing). Honestly, a lot of my logic goes out the window when trying to think about how updating contracts affects all this as well, so I haven't thought about those concerns at all


One thing I'm curious of is that if multiple "Extension developers" make an NFT extension with overlapping functions, would a resource be allowed to extend both of those?

Also, if a transaction wants to use the extended functions of an extended resource, does the "Extension" contract have be imported explicitly?

Btw I like this idea - I'm currently trying to do a makeshift version of the dictionary approach for my current contract (i haven't finished so don't know if it'll work, but this would have been easier with extensions)

@dsainati1
Copy link
Contributor Author

Cadence is acting little bit unreliable here in my opinion, .... We just merged interface default conditions.

Default interface conditions are not quite the same thing as full method overriding. Interfaces are "abstract" in the sense that you cannot instantiate them; if you have some interface A you cannot have a value that is just an A. For this reason developers know when they create an interface that the methods in it will need implementation. Similarly, if I write some function with an A parameter, as a developer I know that I cannot predict with any certainty what any of the methods on A will do, because I know that A is an interface. In effect, because we only allow "overriding" like this on abstract types, developers are not mislead about the behavior of code.

However, if you have some composite B right now, we know with absolute certainty that if you call B.foo it will always execute the same code. To borrow a term from Java, all structs and resources in Cadence are final. This is a nice property to have because it makes code predictable. To allow extensions to override methods in concrete composites would break this property.

@bluesign
Copy link
Contributor

bluesign commented Sep 2, 2022

@dsainati1 I am considering you reference:

Security is a consideration when interacting with other smart contracts. Any external call potentially allows malicious code to be executed. For example, in Solidity, when the called function signature does not match any of the available ones, it triggers Solidity’s fallback functions. These functions can be used in malicious ways. Language features such as multiple inheritances and overloading or dispatch can also make it difficult to determine which code is invoked.

Similarly, if I write some function with an A parameter, as a developer I know that I cannot predict with any certainty what any of the methods on A will do, because I know that A is an interface

But this is nothing bad, even we encourage this usage no ? We want things to be composable. But this "makes it difficult to determine which code is invoked." ( which is the reason of we don't allow "overloading or dispatch".

Btw I don't defend extensions to override methods, I think extensions should be separate type, where methods fallback on to parent.

If I have kittyItem, kittyItem+hat (extension) will not be subtype of kittyItem.

but if you take extension reference: extension = kitty.getExtension(hat), it will behave like kittyItem.

@dsainati1
Copy link
Contributor Author

One thing I'm curious of is that if multiple "Extension developers" make an NFT extension with overlapping functions, would a resource be allowed to extend both of those?

Yes; if two different developers wanted to make a Hat extension for CryptoKitties, you could have a kitty that has both hats on it at the same time. You would just not be able to reference both extensions at the same time if they declare incompatible/overlapping fields or functions.

Also, if a transaction wants to use the extended functions of an extended resource, does the "Extension" contract have be imported explicitly?

Yes, in order for these functions to be available the type of the extension would need to be known, meaning the Extension contract would need to be imported, I believe.

But this is nothing bad, even we encourage this usage no ? We want things to be composable. But this "makes it difficult to determine which code is invoked." ( which is the reason of we don't allow "overloading or dispatch".

The difference is whether the behavior is expected or surprising. Interfaces being implemented is expected by users, while composites being overridden is not expected, so if it were possible it would be surprising to users and hence potentially dangerous. That said it seems like we are in agreement that extensions would not be able to implicitly override base type methods; it seems like you are suggesting that you would need to explicitly call the method on the extension itself in order to get the fallback behavior, which seems fine.

@bluesign
Copy link
Contributor

bluesign commented Sep 2, 2022

“implemented is expected by users” is tricky concept though. If we say “user” is who is reading the code, in practice there is no difference.

I dont think people capable of reading Cadence is big enough to have a concern of this.

@dsainati1
Copy link
Contributor Author

To summarize the discussion here for those following along, currently the largest point of debate is over whether or not extending a resource should change its type. The two models we have are:

  1. Static model: Extending a value of type T with an extension of type E produces a value with type T+E (or T with E, or whatever other syntax you prefer). T+E is a subtype of T, and can be used in places where only T is expected, although of course only methods and fields on T would be accessible in that context. To connect this back to a concrete example, a extending a CryptoKitty with a Hat would produce a new resource of type CryptoKitty+Hat that represents the kitty extended with the hat functionality. CryptoKitty+Hat resources would be usable in by contracts that expect a CryptoKitty only, and contracts or applications that wish to work with only CryptoKittys with Hats would also be able to statically restrict their inputs to resources of this type. This is the behavior that the FLIP currently proposes.

  2. Dynamic model: All resources would store the extensions attached to them internally, and extending a value with a resource would not change its type. Given some resource of type T, there would be no static information present about what extensions are present on the value; instead the extensions on a value would be dynamically inspectable with functions like getExtensions, hasExtension or similar. In this model, a CryptoKitty with a Hat would still be a value of type CryptoKitty, but CryptoKitty.extensions or some similar field would contain a Hat value. Users wishing to work with a Hat-extended kitty would write a function accepting values of type CryptoKitty, and then dynamically check the extensions of their input to find the appropriate extension. This is the behavior @bluesign has been describing (although please correct me if I have missed something in the summary here).

@bluesign
Copy link
Contributor

bluesign commented Sep 5, 2022

Very nice summary @dsainati1, I have few minor things to add for comparison.

Static model

In this model: We cannot extend (modify) methods, but we can add new ones. Actually this is the initial point where my concern was started.

  • For example: CryptoKitty+Hat resource cannot implement new MetadataViews or cannot modify existing ones. ( So all CryptoKitty+Hat will look like CryptoKitty )

There is a very big chance of conflicting extensions, in my opinion if 2 extensions implement an interface the base type doesn't implement, they would not be compatible.

pub struct interface I {
    pub fun foo(): Int
    pub fun bar(): String
}
resource R {
    fun foo() {}
}
extension E1 for R: I {
    fun bar() {}
}
extension E2 for R: I {
    fun bar() {}
}

Also there is a breaking situation when R decides to implement I in the future. I think we need to clear up how resources will behave in that situation. Preventing to apply extension is ok but what will happen to resources already out there with extension applied?

if I has priv member, and R already implements I, if E1 implements I, cannot access priv member right?

Btw I love the static type guarantees and X with Y approach, but I just think it is too much limiting to do anything creative.

Also some positive feedback:

R with E being R can be really useful and can be used a bit like below.

pub resource Collection{
}
access(contract) extension CollectionNFTPublic for Collection: ExampleNFTCollectionPublic, NonFungibleToken.CollectionPublic{
}
access(contract) extension CollectionNFTProvider for Collection: NonFungibleToken.Provider{
}
access(contract) extension CollectionNFTReceiver for Collection: NonFungibleToken.Receiver{
}
access(contract) extension CollectionMetadata for Collection: MetadataViews.ResolverCollection{
}

This gives ability to NFT creator to upgrade NFT or CollectionMetadata with micro-transactions. Upgrade your NFT to support VR for X FLOW, or remove royalties for X FLOW etc.

@dsainati1
Copy link
Contributor Author

Also there is a breaking situation when R decides to implement I in the future. I think we need to clear up how resources will behave in that situation. Preventing to apply extension is ok but what will happen to resources already out there with extension applied?

Updating a resource with a new method is fine, as code is not saved; existing extended resources will just become invalid statically, and extending R with E in also fail the type checker, but their data will be fine because functions are not stored as data anyways. Additionally, if/when #1097 allows adding new fields to resources in contract updates, we should still be safe if we namespace fields internally. The end of the Extended Types subsection of the FLIP says:

At runtime, fields and methods are namespaced, so if `E1` and `E2` both declare a field named `foo`, so while a type `S with E1 and E2` cannot be created, `E1` and `E2` can still both be attached to `S` at the same time at runtime

Basically what this means is that internally the foo field on the two different extensions is stored separately, and won't ever have conflict with any number of foo fields that may be added to the extension. This should in theory also hold true if foo is added to the base resource. The restriction is that multiple extensions that declare the same field/method cannot be used at the same time, i.e. you can't ever have a static type where both E1 and E2 are part of the type, even if the underlying runtime type does have both extensions present. Adding foo to the base resource would statically invalidate any existing extensions that add a foo field, but there would be no need for any kind of data migration. What we would likely need is a way to remove extensions from resources that are themselves already invalid in order to make them usable again.

@bluesign
Copy link
Contributor

bluesign commented Sep 6, 2022

Adding foo to the base resource would statically invalidate any existing extensions that add a foo field, but there would be no need for any kind of data migration. What we would likely need is a way to remove extensions from resources that are themselves already invalid in order to make them usable again.

This is good, but how we will manage to handle this I am confused ( my lack of knowledge on internals of Cadence probably )

But Imagine we have static type: Kitty+Hat, if I put this to one resource:

resource Holder{
   pub var niceKitty : @Kitty+Hat 
}

How to handle this situation ?

Copy link
Contributor

@robert-e-davidson3 robert-e-davidson3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. my only concern was that people could infect e.g. Vault with a backdoor but that requires privileged access which extensions lack

Copy link
Member

@SupunS SupunS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of being able to statically enforce the type that is extending / on which type the extension can be added.

Maybe add a section to explain a bit about how casting/assignability works (or is it already there? didn't come across any).
for e.g: Say I do let v = extend S() with E(), then:

  • Can I assign v to a S typed var, or do I have to explicitly upcast?
  • After assigning/upcasting to S, can I down cast it back to E?

that if `E1` and `E2` declare any fields or methods with the same name, this extension will fail statically, as this would result
in conflicts between the two extended types.

At runtime, fields and methods are namespaced, so if `E1` and `E2` both declare a field named `foo`, so while a type `S with E1 and E2`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps separate out this section which explains how the name collisions are handled (just adding a topic should be sufficient). This is a vital part of the FLIP, and deserves a separate section.

btw, this is a nice solution to prevent name collisions 👌


```cadence
let r <- create R()
let e <- create E()
Copy link
Member

@SupunS SupunS Sep 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer not allowing to create extensions in contexts other than when adding to a resource/struct.
i.e: forbid let e <- create E() or let e = E()
but only allow:

  • extend <- r with E() for resources.
    • Or even better extend r with E(). Here r doesn't need a move op; treat it the same way as being in a member-access. The resulting value after adding the extension is a resource, and it would require a move. e.g: let x <- extend r with E()
  • extend s with E() for structs

I'm emphasizing not to treat E() as a value constructor, but rather as an extension init, which can be only used along with with keyword. This would:

  • Eliminate the feel of E being just a wrapper.
  • Avoid having to introduce a new runtime type for 'extensions'. i.e: type of the result of E(). Plus, don't have to worry about whether to use assignment vs move for the extensions (e.g: why move operator for extension? is it a resource value?)
  • Also helps to simplify the syntax.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Allowing extensions to be created in any context, independently of the value they are being added to, is designed to support a specific use case: having extensions themselves be resources that can be traded like any other NFT. Consider, for example, if I have a Hat item that i want to add to my Kitty. The Hat is an extension for Kitties that can be bought and sold independently of the Kitty itself, and can be removed from one Kitty and added to another. It makes sense for this use case that the Hat can be created and initialized independently of any Kitty that it may be attached to in the future.

@dsainati1
Copy link
Contributor Author

dsainati1 commented Sep 22, 2022

We had a public discussion about this on Tuesday (meeting notes for this can be found here: https://github.com/onflow/cadence/blob/master/meetings/2022-09-20-Extensions.md). Thanks to everyone who attended!

Based on the discussion in that meeting, I am working on an alternative FLIP to this one proposing an "attachments" feature that is designed to solve the same problem, but taking into account the meeting feedback. This alternative would function like an additional field added to resources that would contain a list of all the "attachments" present on that resource, which each have their own functionality and fields, as well as a reference back to the resource that contains them. The new proposal will contain much of the DNA of this old one, but lacking some features and including some others to better address the identified use cases.

However, I was curious about which parts of this proposal we'd like to see in the new one. It seems to me, based on our discussion, that the primary flaw of this one is that adding an extension to a resource creates a new resource entirely, with the methods and fields of both. This has the potential to create issues with name collisions between the original type and the extension, especially as contracts are changed and updated. In the new proposal, fields and methods of attachments will be accessed explicitly through the attachment, rather than implicitly on the composite object. This will allow attachments to name members without concern for what the base or other attachments may have called their members.

It also seems that there is little desire for the ability to remove and re-attach extensions (indeed, as @dete pointed out there is no actual need for this, as "extensions with value" can be achieved by having the extension be a manager for resources rather than a resource in-and-of-itself). This feature will not be present in the new proposal.

As for the parts of this proposal that are worth keeping, in my opinion the two primary benefits of this proposal are the dedicated syntax, which makes it very clear and precise what function is being performed by the creation/removal/declaration of an extension, and the static typing. In particular, the attachment model still seems to be compatible with the R with E type syntax, where R with E1, E2 would describe a composite type R that has the attachments E1 and E2 present on it. This would function like the "opposite" of a restricted type, where the type is still an R, but has additional information that statically guarantees that E1 and E2 must be attached to it.

The benefits of statically typing this are primarily for safety; if there is no static type that expresses the attachments present on a resource, then users have no way to write code that restricts statically what attachments they want resources to have. So, for example, if you receive a resource, if you can't write any static requirements about that resource's attachments, so in order to be sure that it doesn't have any attachments you don't want, users would need to write defensive code every time they receive resources from external code that strips off any undesired attachments from the incoming resources.

Similarly, if a user wants to write a function that uses a resource R and also some attachment features from A, if they can't express the type R with A the would need to write a defensive check at the beginning like:

if !r.hasExtension<A>() {
    return nil 
}

which is a lot of defensive boilerplate if users need to do this at the top of every function intended to interact with extensions.

I am curious to get thoughts from those who weighed in on this proposal about whether this seems like an accurate summary of which features of this one we'd like to see on the next.

@dsainati1
Copy link
Contributor Author

I want to elaborate a little further on the difficulty I mentioned in the language design meeting with the Metadata use case. The core difficulty here is designing an API that allows a resource to iterate over its extensions/attachments without requiring it to know anything about those extensions/attachments in advance. In particular, we'd like for an NFT's implementation of the metadata standard to include any metadata on those attachments.

For a simplified example, let's imagine that implementing the metadata standard just involved having a .metadata() method that returned a String that contained the NFT's metadata. In order to include the metadata of its extensions/attachments, a resource would want to be able to call this metadata method on any of these extensions/attachments that implement this standard, and include all those Strings in the final String produced by its metadata method.

This would require a reference to all these extensions/attachments, either an array type [X] or an iteration function func iterExtensions(f ((extension X): Void)). Developers could then use a loop to iterate over that array, or call the iteration function to apply an operation to every extension/attachment on their resource. But what type should X be? If a resource knows nothing about what kind of extensions/attachments could be present on it, the only type that we could possibly use would be AnyStruct (or AnyResource, if attachments/extensions are resources). But if the type is this general, we can't do anything with it. Currently in Cadence there is no way to check whether a member exists (let alone exists with a specific type) on a value short of attempting a runtime cast to a type with that field. However, if we require the resource developer to attempt to cast X to the type of each extension/attachment that they wish to look for the metadata method on, this inherently requires them to know in advance what kinds of extensions/attachments they will have.

There are a couple options here, as I see it:

  1. We could add support for true runtime reflection, allowing users to ask whether an arbitrary value of type Any has a specific field with a specific type, and then allowing them to access that member once they have determined its type and value. This way users could check for the presence of the metadata method on each extension/attachment and call it if it is found. This is similar to how duck-typing works in a language like JavaScript or Python, and comes with all the dangers and footguns inherent in that style.

  2. We could add runtime interface conformance checking. Users could do a dynamic cast to an interface, like some interface Metadata that contains a metadata():String function type, and if successful would be able to use the resulting value as a value of that interface type. This is safer than the reflection example, but has more failure points. In particular, because Cadence typing is nominal rather than structural, developers of an attachment/extension would need to explicitly declare that they implement Metadata, and any extensions/attachments that have the metadata():String method but don't declare implementation of Metadata would be fail the dynamic cast.

  3. We could allow resource developers to require that any extensions/attachments implement an interface. This would ensure that developers of an extension/attachment for their resource include all the functionality in each extension/attachment that the resource developer expects, and thus make that functionality available to the resource developer. E.g., the resource developer could require that all its attachments/extensions implement Metadata, so then X would be easily known to be {Metadata}, making all the members on that type available on all the extensions/attachments during iteration. However, this also means that if the resource developer wishes to impose a new requirement on extensions/attachments, perhaps to support a new standard similar to how Metadata works, this would break all existing extensions/attachments on that resource until they update their code.

It is unclear to me what the best path forward is here. All of these have pros and cons.

@dsainati1
Copy link
Contributor Author

dsainati1 commented Sep 29, 2022

onflow/flips#11 is a new FLIP proposing an alternative to this one: attachments, which are designed to solve the same problem in a more dynamic fashion, with less static typing. It has strengths and weaknesses of its own, and I am hoping we can compare and contrast these to better evaluate both of them.

In the interest in keeping discussion on both these FLIPs in a single place, please direct future discussion on these to https://forum.onflow.org/t/flip-cadence-extensions-attachments/3645

Copy link
Member

@turbolent turbolent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I think the proposal covers all edge cases

flips/20220817-extensibility.md Outdated Show resolved Hide resolved
flips/20220817-extensibility.md Show resolved Hide resolved
flips/20220817-extensibility.md Show resolved Hide resolved
the extension is destroyed. Like `init`, because `destroy` may be run when the extension is not attached to a base type, it may not reference
any fields or methods of its base type, and should simply destroy any resources declared on the extension itself.

Extensions may also declare two special methods: `attach() { ... }` and `remove() { ... }` that are not considered conflicting when attaching
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these/should these be special functions like init and destroy?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was the intention, yes.

Comment on lines +156 to +158
The extension type itself also can be referenced with just `E`, but no operations can be performed on a value of type `E` other than to
move it around or to attach it to a value using `extend` syntax (see below). That is to say, a method or field defined in the declaration of `E`
exists on values of type `@R with @E`, not on values of type `E`.
Copy link
Member

@turbolent turbolent Oct 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume that function calls on just E will be statically rejected? In the case where dynamic dispatch is involved, e.g. an instance of E is assigned to a variable of type {I}, we will also need to dynamically reject the direct function calls.
Maybe add that to the proposal, if it is the planned behaviour. If not, how else could we handle this?

This limitation will also be a bit restrictive. For example, given that an extension could be an NFT, it would be impossible to get metadata of it in the detached state.

Maybe extensions can just be normal type declarations, which allows them to be used in the detached state, and then an "extension declaration" can be added, which defines how the type interacts with the extended type. For example:

struct S {}

struct E {
  // usable in detached state
  fun foo() { ... } 
}

extension E for S {
  // only usable in attached state
  fun bar() { ... }
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

an instance of E is assigned to a variable of type {I}

At the moment this isn't a concern because extensions cannot implement interfaces in their detached state, so there would never be any dynamic dispatch to worry about.

However, this is also unlikely to be a relevant concern for much longer; based on the discussion in the last meeting about extensibility, I think the sentiment is that extensions should not themselves be resources with value, and should not be usable independently of the thing they are attached to. I.e. removing an extension should also destroy it. Instead the extension would be a "manager" for the valuable resource, perhaps through the indirection of a reference or capability.

flips/20220817-extensibility.md Outdated Show resolved Hide resolved
flips/20220817-extensibility.md Show resolved Hide resolved
@dsainati1 dsainati1 requested review from turbolent and SupunS October 11, 2022 14:09
@dsainati1
Copy link
Contributor Author

This has been open for a while with no further comments, and based on the discussion here (https://forum.onflow.org/t/flip-cadence-extensions-attachments/3645), it seems fairly settled that Attachments is the preferred approach for this problem. If nobody disagrees, I am going to close and reject this FLIP in a day or two.

@dsainati1
Copy link
Contributor Author

I have updated this FLIP to be Rejected, as we have elected to go with the attachments proposal instead. Thank you everyone for your input on this!

@dsainati1 dsainati1 closed this Nov 2, 2022
@dsainati1 dsainati1 reopened this Nov 2, 2022
@dsainati1 dsainati1 merged commit bd98648 into master Nov 2, 2022
@peterargue peterargue deleted the extensibility-FLIP branch January 17, 2023 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Cadence FLIP Flow Improvement Proposal
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants