Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V0.10.0 working #69

Draft
wants to merge 75 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
5fdbe83
Some sketch in simplifying `tensor.Tensor`
chewxy Mar 10, 2020
1f10192
Added Scalar type to prepare for v0.10.0
chewxy Mar 16, 2020
bda53da
Merge branch 'master' into v0.10.0-working
chewxy Jun 18, 2020
44e71b4
Exported StandardEngine
chewxy Jun 23, 2020
5cea23d
Merge branch 'master' into v0.10.0-working
chewxy Dec 20, 2020
0ef7b6b
Merge remote-tracking branch 'origin/v0.10.0-working' into v0.10.0-wo…
chewxy Dec 29, 2020
3e49a5e
Added an unsafe checker.
chewxy Dec 29, 2020
10669eb
Merge branch 'master' into v0.10.0-working
chewxy Jan 17, 2021
027a850
Merge branch 'master' into v0.10.0-working
chewxy Jan 17, 2021
266b215
Removed `standardEngine` requirement from the Tensor definition
chewxy Jan 18, 2021
f5f224b
Merge remote-tracking branch 'origin/v0.10.0-working' into v0.10.0-wo…
chewxy Jan 18, 2021
f79a6ba
Removed requirement for fmt.Stringer and fmt.Formatter
chewxy Jan 18, 2021
b90b340
Merge branch 'master' into v0.10.0-working
chewxy Apr 4, 2021
54b4849
Initial work to separate out View into its own type...
chewxy Apr 4, 2021
a45f584
Finished moving View out. Now time to deal with the transposey views
chewxy Apr 5, 2021
2b2c322
Merge branch 'master' into v0.10.0-working
chewxy Apr 18, 2021
72da5ac
Started to move out Dtype to its own package.
chewxy Jul 9, 2021
70f5515
More work to move Dtype related stuff into its own package
chewxy Jul 9, 2021
64da890
More work on moving Dtype to its own package
chewxy Jul 9, 2021
40696d4
Finished move of Dtype to its own package
chewxy Jul 9, 2021
f473c71
Moved Shapes out and use the shapes package
chewxy Jul 16, 2021
9284c4c
Merge remote-tracking branch 'origin/master' into v0.10.0-working
chewxy Jul 16, 2021
9c85a54
Merge remote-tracking branch 'origin/v0.10.0-working' into v0.10.0-wo…
chewxy Jul 16, 2021
3b361cb
Fixed iterator example
chewxy Jul 16, 2021
790423c
More work to remove Shape stuff from this package
chewxy Jul 16, 2021
aadb03f
Deifne equality for DenseTensor
chewxy Jul 21, 2021
b5d3255
Fixed MatMul API to handle different Tensor types
chewxy Jul 21, 2021
b9b9610
Expanded OpOpt's methods to allow modification
chewxy Jul 21, 2021
e350632
Added axial iterator
chewxy Jul 26, 2021
fc40f1f
Added max example
chewxy Jul 26, 2021
52da427
Added prelim sketch of Scatter
chewxy Jul 26, 2021
0276550
Added some checks for gather scatter
chewxy Jul 26, 2021
1392c0c
Fixed the def'n of StandardEngine, added StandardEngine2
chewxy Jul 28, 2021
45f2901
Moved funcopts into its own file because it should be in its own file.
chewxy Jul 28, 2021
945bb67
Added more support for context.Context. Now the API functions can be …
chewxy Jul 28, 2021
5708fa5
Generated tests for context func opts
chewxy Jul 28, 2021
843e31e
Unexported OpOpt.
chewxy Jul 28, 2021
8bb1cc3
Moved native iterators into package `tensor` proper.
chewxy Jul 29, 2021
309a2ff
Moved native select into package `tensor` as well.
chewxy Jul 29, 2021
17d8af0
Added lazy native select
chewxy Jul 29, 2021
693ae40
renamed the files
chewxy Jul 29, 2021
3b675c0
Renamed LazyNativeSelectF64 to BatchedNativeSelectF64
chewxy Jul 29, 2021
ff02820
Playing with the API a bit to see if I can make it nicer
chewxy Jul 29, 2021
6448fd0
Playing with API a bit more.
chewxy Jul 29, 2021
0eb6ae0
I am finally satisfied with the API. Added a lot more documentation
chewxy Jul 29, 2021
f4ff98c
fixed the WithBacking funcopt to actually play nicer with CUDA.
chewxy Aug 4, 2021
f478e34
Added a temporary range based iter...
chewxy Aug 4, 2021
3028c48
Fixed a bug in the ElNe API
chewxy Aug 5, 2021
e7ffa20
Added some funcopts to Inner to allow for context handling
chewxy Aug 18, 2021
d000a37
Added some more support for context.Context
chewxy Aug 19, 2021
9f3ca0f
Changed almost all the engine interface definitions to also include c…
chewxy Aug 19, 2021
c9298a2
Converted transpose and concat to use context.
chewxy Aug 19, 2021
4a15ca2
added some notes clarifying the API design of the tensor package. Loo…
chewxy Aug 19, 2021
8f8a52d
Fixed up Stack, Repeat and Diag
chewxy Aug 20, 2021
4f78ddc
Fixed the linear algebra methods to include context.Context
chewxy Aug 20, 2021
18e8bf8
Fixed reduction methods to use context
chewxy Aug 20, 2021
7a3ac9b
Fixed argmethods to use context
chewxy Aug 20, 2021
aebd7fd
Fixed up a missing FMA fix for handling contexts
chewxy Aug 20, 2021
ac15a99
Fixed most of the APIs. Some of them like the reduction-related packa…
chewxy Aug 20, 2021
4736352
Fixed up argmethods. Now to work on actually fixing the API
chewxy Aug 20, 2021
8f474cd
Fixed the API for Inner
chewxy Sep 2, 2021
fb09a60
Fixed the StdEng's definition of Trace to actually implement Tracer
chewxy Sep 3, 2021
423bfda
Changed the shape definition of Outer such that it only checks for th…
chewxy Sep 9, 2021
e89d039
Merge remote-tracking branch 'origin/master' into v0.10.0-working
chewxy Oct 4, 2021
6d947b5
Fixed minmaxbetween for 0.10.0
chewxy Oct 4, 2021
27f372d
Made the Slice type = shapes.Slice
chewxy Mar 21, 2022
c4c3263
Fixed error messages to have a consistent error message for NYIs
chewxy May 4, 2022
7e74991
Merge remote-tracking branch 'origin/master' into v0.10.0-working
chewxy May 4, 2022
5d33203
Fixed a bunch of issues due to merging from origin/master.
chewxy May 4, 2022
460aba2
Added documentation for Transpose and T()
chewxy May 4, 2022
d0c11c8
Added examples for SliceInto
chewxy May 4, 2022
2916935
Merge remote-tracking branch 'origin/master' into v0.10.0-working
Jul 18, 2022
02d9dc7
Merge remote-tracking branch 'origin/master' into v0.10.0-working
chewxy Jul 22, 2022
bf9b9e7
Some corrections to the tests of selbyidx. Prepping for the larger ch…
chewxy Sep 26, 2023
da1342f
Merge remote-tracking branch 'origin/v0.10.0-working' into v0.10.0-wo…
Sep 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 17 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,16 @@ Package `tensor` is a package that provides efficient, generic (by some definiti
The main purpose of this package is to support the operations required by [Gorgonia](https://gorgonia.org/gorgonia).

## Introduction ##
In the data analysis world, [Numpy](http://http://www.numpy.org/) and [Matlab](https://www.mathworks.com/products/matlab.html) currently reign supreme. Both tools rely heavily on having performant n-dimensional arrays, or tensors. **There is an obvious need for multidimensional arrays in Go**.
In the data analysis world, [Numpy](http://http://www.numpy.org/) and [Matlab](https://www.mathworks.com/products/matlab.html) currently reign supreme. Both tools rely heavily on having performant n-dimensional arrays, or tensors. **There is an obvious need for multidimensional arrays in Go**.

While slices are cool, a large majority of scientific and numeric computing work relies heavily on matrices (two-dimensional arrays), three dimensional arrays and so on. In Go, the typical way of getting multidimensional arrays is to use something like `[][]T`. Applications that are more math heavy may opt to use the very excellent Gonum [`matrix` package](https://github.com/gonum/matrix). What then if we want to go beyond having a `float64` matrix? What if we wanted a 3-dimensional `float32` array?

It comes to reason then there should be a data structure that handles these things. The `tensor` package fits in that niche.
It comes to reason then there should be a data structure that handles these things. The `tensor` package fits in that niche.

### Basic Idea: Tensor ###
A tensor is a multidimensional array. It's like a slice, but works in multiple dimensions.

With slices, there are usage patterns that are repeated enough that warrant abstraction - `append`, `len`, `cap`, `range` are abstractions used to manipulate and query slices. Additionally slicing operations (`a[:1]` for example) are also abstractions provided by the language. Andrew Gerrand wrote a very good write up on [Go's slice usage and internals](https://blog.golang.org/go-slices-usage-and-internals).
With slices, there are usage patterns that are repeated enough that warrant abstraction - `append`, `len`, `cap`, `range` are abstractions used to manipulate and query slices. Additionally slicing operations (`a[:1]` for example) are also abstractions provided by the language. Andrew Gerrand wrote a very good write up on [Go's slice usage and internals](https://blog.golang.org/go-slices-usage-and-internals).

Tensors come with their own set of usage patterns and abstractions. Most of these have analogues in slices, enumerated below (do note that certain slice operation will have more than one tensor analogue - this is due to the number of options available):

Expand All @@ -26,7 +26,7 @@ Tensors come with their own set of usage patterns and abstractions. Most of thes
| `a[0]` | `T.At(x,y)` |
| `append(a, ...)`| `T.Stack(...)`, `T.Concat(...)` |
| `copy(dest, src)`| `T.CopyTo(dest)`, `tensor.Copy(dest, src)` |
| `for _, v := range a` | `for i, err := iterator.Next(); err == nil; i, err = iterator.Next()` |
| `for _, v := range a` | `for i, err := iterator.Next(); err == nil; i, err = iterator.Next()` |

Some operations for a tensor does not have direct analogues to slice operations. However, they stem from the same idea, and can be considered a superset of all operations common to slices. They're enumerated below:

Expand Down Expand Up @@ -77,7 +77,7 @@ fmt.Printf("a:\n%v\n", a)

To create a 3-Tensor is just as easy - just put the correct shape and you're good to go:

```go
```go
// Creating a (2,3,4) 3-Tensor of float32
b := New(WithBacking(Range(Float32, 0, 24)), WithShape(2, 3, 4))
fmt.Printf("b:\n%1.1f\n", b)
Expand Down Expand Up @@ -133,6 +133,12 @@ fmt.Printf("b:\n%v", b)
There is a whole laundry list of methods and functions available at the [godoc](https://godoc.org/gorgonia.org/tensor) page


## API Notes ##

This package has a notion of "layers" in its API. This section clarifies the different patterns seen in the API.




## Design of `*Dense` ##

Expand All @@ -142,7 +148,7 @@ The design of the `*Dense` tensor is quite simple in concept. However, let's sta

The data structure for `*Dense` is similar, but a lot more complex. Much of the complexity comes from the need to do accounting work on the data structure as well as preserving references to memory locations. This is how the `*Dense` is defined:

```go
```go
type Dense struct {
*AP
array
Expand All @@ -168,7 +174,7 @@ type array struct {
}
```

`*storage.Header` is the same structure as `reflect.SliceHeader`, except it stores a `unsafe.Pointer` instead of a `uintptr`. This is done so that eventually when more tests are done to determine how the garbage collector marks data, the `v` field may be removed.
`*storage.Header` is the same structure as `reflect.SliceHeader`, except it stores a `unsafe.Pointer` instead of a `uintptr`. This is done so that eventually when more tests are done to determine how the garbage collector marks data, the `v` field may be removed.

The `storage.Header` field of the `array` (and hence `*Dense`) is there to provide a quick and easy way to translate back into a slice for operations that use familiar slice semantics, of which much of the operations are dependent upon.

Expand Down Expand Up @@ -205,17 +211,17 @@ The alternative designs can be seen in the [ALTERNATIVE DESIGNS document](https:

Example:

```go
```go

x := New(WithBacking([]string{"hello", "world", "hello", "world"}), WithShape(2,2))
x = New(WithBacking([]int{1,2,3,4}), WithShape(2,2))
```

The above code will not cause a compile error, because the structure holding the underlying array (of `string`s and then of `int`s) is a `*Dense`.
The above code will not cause a compile error, because the structure holding the underlying array (of `string`s and then of `int`s) is a `*Dense`.

One could argue that this sidesteps the compiler's type checking system, deferring it to runtime (which a number of people consider dangerous). However, tools are being developed to type check these things, and until Go does support typechecked generics, unfortunately this will be the way it has to be.

Currently, the tensor package supports limited type of genericity - limited to a tensor of any primitive type.
Currently, the tensor package supports limited type of genericity - limited to a tensor of any primitive type.

# How This Package is Developed #
Much of the code in this package is generated. The code to generate them is in the directory `genlib2`. `genlib2` requires [`goimports`](https://godoc.org/golang.org/x/tools/cmd/goimports) binary to be available in the $PATH.
Expand Down Expand Up @@ -246,7 +252,7 @@ See also: CONTRIBUTING.md

## Contributors and Significant Contributors ##

All contributions are welcome. However, there is a new class of contributor, called Significant Contributors.
All contributions are welcome. However, there is a new class of contributor, called Significant Contributors.

A Significant Contributor is one who has shown *deep understanding* of how the library works and/or its environs. Here are examples of what constitutes a Significant Contribution:

Expand Down
4 changes: 2 additions & 2 deletions ap.go
Original file line number Diff line number Diff line change
Expand Up @@ -365,9 +365,9 @@ func (ap *AP) unlock() { ap.fin = false }
func (ap *AP) calcStrides() []int {
switch {
case ap.o.IsRowMajor():
return ap.shape.CalcStrides()
return CalcStrides(ap.shape)
case ap.o.IsColMajor():
return ap.shape.CalcStridesColMajor()
return CalcStridesColMajor(ap.shape)
}
panic("unreachable")
}
Expand Down
2 changes: 1 addition & 1 deletion ap_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ func TestAccessPatternS(t *testing.T) {
var err error

for _, sts := range sliceTests {
ap = MakeAP(sts.shape, sts.shape.CalcStrides(), 0, 0)
ap = MakeAP(sts.shape, CalcStrides(sts.shape), 0, 0)
if apS, ndStart, ndEnd, err = ap.S(sts.shape.TotalSize(), sts.slices...); err != nil {
t.Errorf("%v errored: %v", sts.name, err)
continue
Expand Down
Loading
Loading