SIP-58 - Named Tuples

content/named-tuples.md

@@ -0,0 +1,282 @@
+---
+layout: sip
+permalink: /sips/named-tuples.html
+stage: implementation
+status: waiting-for-implementation
+presip-thread: https://contributors.scala-lang.org/t/pre-sip-named-tuples/6403/164
+title: SIP-NN - Named Tuples
+---
+
+**By: Martin Odersky**
+
+## History
+
+| Date          | Version            |
+|---------------|--------------------|
+| Jan 13th 2024 | Initial Draft      |
+
+## Summary
+
+We propose to add new form of tuples where the elements are named.
+Named tuples can be types, terms, or patterns. Syntax examples:
+```scala
+type Person = (name: String, age: Int)
+val Bob: Person = (name = "Bob", age = 33)
+
+Bob match
+  case (name = n, age = 22) => ...
+```
+
+We also propose to revive SIP 43 to support patterns with named fields. Named pattern fields for case classes are analogous to named patterns for tuple elements. User-defined named pattern matching is supported since named tuples can be results of extractor methods.
+
+## Motivation
+
+ 1. Named tuples are a convenient lightweight way to return multiple results from a function. But the absence of names obscures their meaning, and makes decomposition with _1, _2 ugly and hard to read. The existing alternative is to define a class instead. This does name fields, but is more heavy-weight, both in terms of notation and generated bytecode. Named tuples give the same convenience of definition as regular tuples at far better readability.
+
+ 1. Named tuples are an almost ideal substrate on which to implement relational algebra and other database oriented operations. They are a good representation of database rows and allow the definition of generic operations such as projections and joins since they can draw on Scala 3’s existing generic machinery for tuples based on match types.
+
+ 1. Named tuples make named pattern matching trivial to implement. The discussion on SIP 43 showed that without them it’s unclear how to implement named pattern matching at all.
+
+## Proposed solution
+
+The elements of a tuple can now be named. Example:
+```scala
+type Person = (name: String, age: Int)
+val Bob: Person = (name = "Bob", age = 33)
+
+Bob match
+  case (name, age) =>
+    println(s"$name is $age years old")
+
+val persons: List[Person] = ...
+val minors = persons.filter: p =>
+  p.age < 18
+```
+Named bindings in tuples are similar to function parameters and arguments. We use `name: Type` for element types and `name = value` for element values. It is illegal to mix named and unnamed elements in a tuple, or to use the same same
+name for two different elements.
+
+Fields of named tuples can be selected by their name, as in the line `p.age < 18` above.
+
+Example:
+
+~~~ scala
+// This is an @main method
+@main def foo(x: Int): Unit =
+  println(x)
+~~~
+
+### Conformance
+
+The order of names in a named tuple matters. For instance, the type `Person` above and the type `(age: Int, name: String)` would be different, incompatible types.
+
+Values of named tuple types can also be be defined using regular tuples. For instance:
+```scala
+val x: Person = ("Laura", 25)
+
+def register(person: Person) = ...
+register(person = ("Silvain", 16))
+register(("Silvain", 16))
+```
+This follows since a regular tuple `(T_1, ..., T_n)` is treated as a subtype of a named tuple `(N_1 = T_1, ..., N_n = T_n)` with the same element types. On the other hand, named tuples do not conform to unnamed tuples, so the following is an error:
+```scala
+val x: (String, Int) = Bob           // error: type mismatch
+```
+One can convert a named tuple to an unnamed tuple with the `toTuple` method, so the following works:
+```scala
+val x: (String, Int) = Bob.toTuple // ok
+```
+
+_Question:_ Should we define an implicit conversion, either in place of this method or in addition to it?
+
+Note that conformance rules for named tuples are analogous to the rules for named parameters. One can assign parameters by position to a named parameter list.
+```scala
+  def f(param: Int) = ...
+  f(param = 1)   // OK
+  f(2)           // Also OK
+```
+But one cannot use a name to pass an argument to an unnamed parameter:
+```scala
+    val f: Int => T
+    f(2)         // OK
+    f(param = 2) // Not OK
+```
+The rules for tuples are analogous. Unnamed tuples conform to named tuple types, but the opposite does not hold.
+
+
+### Pattern Matching
+
+When pattern matching on a named tuple, the pattern may be named or unnamed.
+If the pattern is named it needs to mention only a subset of the tuple names, and these names can come in any order. So the following are all OK:
+```scala
+Bob match
+  case (name, age) => ...
+
+Bob match
+  case (name = x, age = y) => ...
+
+Bob match
+  case (age = x) => ...
+
+Bob match
+  case (age = x, name = y) => ...
+```
+
+### Expansion
+
+Named tuples are in essence just a convenient syntax for regular tuples. In the internal representation, a named tuple type is represented at compile time as a pair of two tuples. One tuple contains the names as literal constant string types, the other contains the element types. The runtime representation of a named tuples consists of just the element values, whereas the names are forgotten. This is achieved  by declaring `NamedTuple`
+in package `scala` as an opaque type as follows:
+```scala
+  opaque type NamedTuple[N <: Tuple, +V <: Tuple] >: V = V
+```
+For instance, the `Person` type would be represented as the type
+```scala
+NamedTuple[("name", "age"), (String, Int)]
+```
+`NamedTuple` is an opaque type alias of its second, value parameter. The first parameter is a string constant type which determines the name of the element. Since the type is just an alias of its value part, names are erased at runtime, and named tuples and regular tuples have the same representation.
+
+A `NamedTuple[N, V]` type is publicly known to be a supertype (but not a subtype) of its value paramater `V`, which means that regular tuples can be assigned to named tuples but not _vice versa_.
+
+The `NamedTuple` object contains a number of extension methods for named tuples hat mirror the same functions in `Tuple`. Examples are
+`apply`, `head`, `tail`, `take`, `drop`, `++`, `map`, or `zip`.
+Similar to `Tuple`, the `NamedTuple` object also contains types such as `Elem`, `Head`, `Concat`
+that describe the results of these extension methods.
+
+The translation of named tuples to instances of `NamedTuple` is fixed by the specification and therefore known to the programmer. This means that:
+
+ - All tuple operations also work with named tuples "out of the box".
+ - Macro libraries can rely on this expansion.
+
+### The FieldsOf Type
+
+The `NamedTuple` object contains a type definition
+```scala
+  type FieldsOf[T] <: AnyNamedTuple
+```
+`FieldsOf` is treated specially by the compiler. When `FieldsOf` is applied to
+an argument type that is an instance of a case class, the type expands to the named
+tuple consisting of all the fields of that case class. Here, fields means: elements of the first parameter section. For instance, assuming
+```scala
+case class City(zip: Int, name: String, population: Int)
+```
+then `FieldsOf[City]` is the named tuple
+```scala
+(zip: Int, name: String, population: Int)
+```
+The same works for enum cases expanding to case classes.
+
+### Pattern Matching with Named Fields in General
+
+We allow named patterns not just for named tuples but also for case classes. For instance:
+```scala
+city match
+  case c @ City(name = "London") => println(p.population)
+  case City(name = n, zip = 1026, population = pop) => println(pop)
+```
+
+Named constructor patterns are analogous to named tuple patterns. In both cases
+
+ - every name must match the name some field of the selector,
+ - names can come in any order,
+ - not all fields of the selector need to be matched.
+
+This revives SIP 43, with a much simpler desugaring than originally proposed.
+Named patterns are compatible with extensible pattern matching simply because
+`unapply` results can be named tuples.
+
+
+### Restrictions
+
+The following restrictions apply to named tuples and named pattern arguments:
+
+ 1. Either all elements of a tuple or constructor pattern are named or none are named. It is illegal to mix named and unnamed elements in a tuple. For instance, the following is in error:
+    ```scala
+    val illFormed1 = ("Bob", age = 33)  // error
+    ```
+ 2. Each element name in a named tuple or constructor pattern must be unique. For instance, the following is in error:
+    ```scala
+    val illFormed2 = (name = "", age = 0, name = true)  // error
+    ```
+ 3. Named tuples and case classes can be matched with either named or regular patterns. But regular tuples and other selector types can only be matched with regular tuple patterns. For instance, the following is in error:
+    ```scala
+    (tuple: Tuple) match
+        case (age = x) => // error
+    ```
+
+### Syntax Changes
+
+The syntax of Scala is extended as follows to support named tuples and
+named constructor arguments:
+```
+SimpleType        ::=  ...
+                    |  ‘(’ NameAndType {‘,’ NameAndType} ‘)’
+NameAndType       ::=  id ':' Type
+
+SimpleExpr        ::=  ...
+                    |  '(' NamedExprInParens {‘,’ NamedExprInParens} ')'
+NamedExprInParens ::=  id '=' ExprInParens
+
+Patterns          ::=  Pattern {‘,’ Pattern}
+                    |  NamedPattern {‘,’ NamedPattern}
+NamedPattern      ::=  id '=' Pattern
+```
+
+### Compatibility
+
+Named tuple types and expressions are simply desugared to types and trees already known to Scala. The desugaring happens before the checking, so does not influence Tasty generation.
+
+Pattern matching with named fields requires some small additions to Typer and the PatternMatcher phase. It does not change the Tasty format, though.
+
+Backward source compatibility is partially preserved since additions to types and patterns come with new syntax that was not expressible before. When looking at tuple expressions, we have one instance of a source incompatibility:
+
+```scala
+var age: Int
+(age = 1)
+```
+This was an assignment in parentheses before, and is a named tuple of arity one now. It is however not idiomatic Scala code, since assignments are not usually enclosed in parentheses. The problem could also be detected and diagnosed fairly straightforwardly: When faced with a unary named tuple, try to interpret it as an assignment, and if that succeeds, issue a migration error.
+
+### Open questions
+
+ 1. What is the precise set of types and operations we want to add to `NamedTuple`. This could also evolve after this SIP is completed.
+
+ 2. Should there be an implicit conversion from named tuples to ordinary tuples?
+
+## Alternatives
+
+### Structural Types
+
+We also considered to expand structural types. Structural types allow to abstract over existing classes, but require reflection or some other library-provided mechanism for element access. By contrast, named tuples have a separate representation as tuples, which can be manipulated directly. Since elements are ordered, traversals can be defined, and this allows the definition of type generic algorithms over named tuples. Structural types don’t allow such generic algorithms directly. Be could define mappings between structural types and named tuples, which could be used to implement such algorithms. These mappings would certainly become simpler if they map to/from named tuples than if they had to map to/from user-defined "HMap"s.
+
+By contrast to named tuples, structural types are unordered and have width subtyping. This comes with the price that no natural element ordering exist, and that one usually needs some kind of dictionary structure for access. We believe that the following advantages of named tuples over structural types outweigh the loss of subtyping flexibility:
+
+ - Better integration since named tuples and normal tuples share the same representation.
+ - Better efficiency, since no dictionary is needed.
+ - Natural traversal order allows the formulation of generic algorithms such as projections and joins.
+
+### Conformance
+
+A large part of Pre-SIP discussion centered around subtyping rules,. whether ordinary tuples should subtype named-tuples (as in this proposal) or _vice versa_ or maybe no subtyping at all.
+
+Looking at precedent in other languages it feels like we we do want some sort of subtyping for easy convertibility and possibly an implicit conversion in the other direction.
+
+The discussion established that both forms of subtyping are sound. My personal opinion is that the subtyping of this proposal is both more useful and safer than the one in the other direction. There is also the problem that changing the subtyping direction would be incompatible with the current structure of `Tuple` and `NamedTuple` since for instance `zip` is already an inline method on `Tuple` so it could not be overridden in `NamedTuple`. To make this work requires a refactoring of `Tuple` to use more extension methods, and the questions whether this is feasible and whether it can be made binary backwards compatible are unknown. I personally will not work on this, if others are willing to make the effort we can discuss the alternative subtyping as well.
+
+### Spread Operator
+
+An idea I was thinking of but that I did not include in this proposal highlights another potential problem with subtyping. Consider adding a _spread_ operator `*` for tuples and named tuples. if `x` is a tuple then `f(x*)` is `f` applied to all fields of `x` expanded as individual arguments. Likewise, if `y` is a named tuple, then `f(y*)` is `f` applied to all elements of `y` as named arguments.
+Now, if named tuples would be subtypes of tuples, this would actually be ambiguous since widening `y` in `y*` to a regular tuple would yield a different call. But with the subtyping direction we have, this would work fine.
+
+I believe tuple spread is a potentially useful addition that would fit in well with Scala. But it's not immediately relevant to this proposal, so is left out for now.
+
+
+## Related work
+
+This section should list prior work related to the proposal, notably:
+
+- [Pre-SIP Discussion](https://contributors.scala-lang.org/t/pre-sip-named-tuples/6403)
+
+- [SIP 43 on Pattern Matching with Named Fields](https://github.com/scala/improvement-proposals/pull/44)
+
+- [Experimental Implementation](https://github.com/lampepfl/dotty/pull/19174)
+
+## FAQ
+