Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AutoDiff] Remove '_Differentiable.zeroTangentVectorInitializer'. #35329

Merged
merged 1 commit into from
Jan 20, 2021
Merged

[AutoDiff] Remove '_Differentiable.zeroTangentVectorInitializer'. #35329

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 6 additions & 100 deletions docs/DifferentiableProgramming.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -986,44 +986,6 @@ public protocol Differentiable {
/// equivalent to exponential map, which moves `self` on the geodesic
/// surface along the given tangent vector.
mutating func move(along direction: TangentVector)

/// A closure that produces a zero tangent vector and does not capture `self`.
///
/// In some cases, the zero tangent vector of `self` is equal to
/// `TangentVector.zero`. In other cases, the zero tangent vector depends on
/// information in `self`, such as shape for an n-dimensional array type.
/// For differentiable programming, it is more memory-efficient to define a
/// custom `zeroTangentVectorInitializer` property which returns a closure
/// that captures and uses only the necessary information to create a zero
/// tangent vector. For example:
///
/// ```swift
/// struct Vector {
/// var scalars: [Float]
/// var count: Int { scalars.count }
/// init(repeating repeatedElement: Float, count: Int) { ... }
/// }
///
/// extension Vector: Differentiable {
/// typealias TangentVector = Vector
///
/// @noDerivative
/// var zeroTangentVectorInitializer: () -> TangentVector {
/// let count = self.count
/// return { TangentVector(repeating: 0, count: count) }
/// }
/// }
/// ```
///
@noDerivative
var zeroTangentVectorInitializer: () -> TangentVector { get }
}

extension Differentiable {
/// A tangent vector such that `move(along: zeroTangentVector)` will not modify
/// `self`.
@noDerivative
var zeroTangentVector: TangentVector { zeroTangentVectorInitializer() }
}
```

Expand Down Expand Up @@ -1092,13 +1054,6 @@ public extension Differentiable where Self == TangentVector {
}
```

The `zeroTangentVector` property returns a tangent vector such that calling
`move(along:)` on the vector will not modify `self`. A zero tangent vector is
often used in the initialization of mathematical optimization, where tangent
vectors are initially zero and modified iteratively. This property may be
different from `TangentVector.zero` because some tangent vectors depend on
instance properties of `self`, e.g. the `count` property in `Array`.

#### `Differentiable` conformances

Conforming a type to `Differentiable` tells Swift that changes in values of this
Expand Down Expand Up @@ -1146,13 +1101,6 @@ extension Array: Differentiable where Element: Differentiable {
self[i].move(along: Element.TangentVector(direction.elements[i]))
}
}

@noDerivative
public var zeroTangentVectorInitializer: () -> TangentVector {
{ [zeroInits = map(\.zeroTangentVectorInitializer)] in
TangentVector(zeroInits.map { $0() })
}
}
}

// struct Dictionary<Key: Hashable, Value>
Expand All @@ -1173,14 +1121,6 @@ extension Dictionary: Differentiable where Value: Differentiable {
self[i].move(along: Value.TangentVector(direction.elements[i]))
}
}

@noDerivative
public var zeroTangentVectorInitializer: () -> TangentVector {
{ [keys = self.keys] in
let pairs = zip(keys, sequence(first: .zero, next: {$0}))
return TangentVector(Dictionary(uniqueKeysWithValues: pairs))
}
}
}

// enum Optional<Wrapped>
Expand All @@ -1199,18 +1139,6 @@ extension Optional: Differentiable where Wrapped: Differentiable {
self?.move(along: value)
}
}

@noDerivative
public var zeroTangentVectorInitializer: () -> TangentVector {
switch self {
case nil:
return { TangentVector(nil) }
case let x?:
return { [zeroTanInit = x.zeroTangentVectorInitializer] in
TangentVector(zeroTanInit())
}
}
}
}
```

Expand Down Expand Up @@ -1261,13 +1189,12 @@ product manifold of the manifolds each differentiable variable's type
represents. Differentiable variables' types are required to conform to
`Differentiable` because the synthesized implementation needs to access each
differentiable variable's type's `TangentVector` associated type and invoke each
differentiable variable's implementation of `move(along:)` and
`zeroTangentVectorInitializer`. Because the synthesized implementation needs to
invoke `move(along:)` on each differentiable variable, the differentiable
variables must have a `move(along:)` which satisfies the protocol requirement
and can be invoked on the property. That is, the property must be either a
variable (`var`) or a constant (`let`) with a non-`mutating` implementation of
the `move(along:)` protocol requirement.
differentiable variable's implementation of `move(along:)`. Because the
synthesized implementation needs to invoke `move(along:)` on each differentiable
variable, the differentiable variables must have a `move(along:)` which satisfies the
protocol requirement and can be invoked on the property. That is, the property
must be either a variable (`var`) or a constant (`let`) with a non-`mutating`
implementation of the `move(along:)` protocol requirement.

The synthesized `TangentVector` has the same effective access level as the
original type declaration. Properties in the synthesized `TangentVector` have
Expand All @@ -1282,12 +1209,6 @@ example, synthesized `TangentVector`s always adopt the `AdditiveArithmetic` and
The synthesized `move(along:)` method calls `move(along:)` for each pair of a
differentiable variable and its corresponding property in `TangentVector`.

The synthesized `zeroTangentVectorInitializer` property returns a closure that
captures and calls each stored property's `zeroTangentVectorInitializer`
closure. When memberwise derivation is not possible (e.g. for custom
user-defined `TangentVector` types), `zeroTangentVectorInitializer` is
synthesized as a `{ TangentVector.zero }` closure.

```swift
struct Foo<T: Differentiable, U: Differentiable>: @memberwise Differentiable {
// `x` and `y` are the "differentiable variables".
Expand All @@ -1306,13 +1227,6 @@ struct Foo<T: Differentiable, U: Differentiable>: @memberwise Differentiable {
// x.move(along: direction.x)
// y.move(along: direction.y)
// }
//
// var zeroTangentVectorInitializer: () -> TangentVector {
// { [xTanInit = x.zeroTangentVectorInitializer,
// yTanInit = y.zeroTangentVectorInitializer] in
// TangentVector(x: xTanInit(), y: yTanInit())
// }
// }
}
```

Expand Down Expand Up @@ -1404,14 +1318,6 @@ struct Point<T: Real>: @memberwise Differentiable, @memberwise AdditiveArithmeti
// The compiler synthesizes:
//
// typealias TangentVector = Self
//
// @noDerivative
// var zeroTangentVectorInitializer: () -> TangentVector {
// { [xTanInit = x.zeroTangentVectorInitializer,
// yTanInit = y.zeroTangentVectorInitializer] in
// TangentVector(x: xTanInit(), y: yTanInit())
// }
// }
}
```

Expand Down
1 change: 0 additions & 1 deletion include/swift/AST/KnownIdentifiers.def
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,6 @@ IDENTIFIER(move)
IDENTIFIER(pullback)
IDENTIFIER(TangentVector)
IDENTIFIER(zero)
IDENTIFIER(zeroTangentVectorInitializer)

#undef IDENTIFIER
#undef IDENTIFIER_
Expand Down
Loading