Allow FTL to support external types by utilising type widening

[alecthomas]

In the following example did.PortableDID is a type from an external library. Currently we disallow this, but this pattern occurs a lot, particularly when dealing with open standards, etc.

Currently we need to do something like this:

var portableDID = ftl.Secret[map[string]any]("did_web_portable_did")

// Access the DID

func bearerDID(ctx context.Context) did.BearerDID {
	m := portableDID.Get(ctx)
	data, err := json.Marshal(m)
	if err != nil {
		panic(err)
	}
	var d did.PortableDID
	if err = json.Unmarshal(data, &d); err != nil {
		panic(err)
	}
	out, err := did.FromPortableDID(d)
	if err != nil {
		panic(err)
	}
	return out
}

Which is represented in the schema like so:

secret did_web_portable_did {String: Any}

Gross.

The proposal is that because there's no type safety anyway, we allow external types to be used directly and the FTL type extraction would find a wider type that would still represent the concrete type in Go. In this case it would be a {String: Any}, but for eg. an array of something it might be [Any].

var portableDID = ftl.Secret[did.PortableDID]("did_web_portable_did")

That is, it would still be represented in the FTL schema as {String: Any} but would automatically decode into a strong Go type. This is no less type safe than what we already have, and in some ways is more type safe because at least we have strong typing in Go. In other languages such as Kotlin the same pattern could apply, using the custom external types for that "type".

Once #1285 lands, if type widening occurs a warning would be issued.

This would also apply to request/response types, etc. Basically anywhere FTL types are used.

Potential solution

Here's a potential proposal @matt2e , @gak and @alecthomas discussed:

Problems:

1. Making this sane cross-language.
2. Making it simple to use.

Proposal:

Introduce type aliases so that type mapping annotations can be added to the named types.

typealias DID {String: Any}
  +typemap go "github.com/foo/bar.DID"
  +typemap kotlin "com.tbd.did.PortableDID"

secret did_web_portable_did DID

Then in Go this might look like the following.

//ftl:typealias
//ftl:typemap kotlin "com.tbd.did.PortableDID"
type DID did.PortableDID

// UserID represents a user ID.
//
//ftl:typealias
type UserID string

var portableDID = ftl.Secret[did.PortableDID]("did_web_portable_did")

In this example we've defined a type alias "DID", which gives FTL a named type to add annotations to, and because we know the target type we can allow users to implicitly use the "real" external type and we will take care of the type mapping for them.

Longer term in a multi-language world we would want the ability for both producers and consumers to be able to override these type mappings.

[alecthomas]

There is another ticket re. mapping to/from external types