Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] cl_exp_tensor #1006

Draft
wants to merge 26 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
7d5e350
Sketch something for cl_khr_tensor
linehill Oct 25, 2023
1f0be1e
Apply suggestions from code review
linehill Nov 2, 2023
a801aaf
* Add brief introduction.
linehill Nov 2, 2023
b890c30
Add contributors
linehill Nov 2, 2023
fafb30b
* Fix name for add kernel creator
linehill Nov 2, 2023
740f3f2
* cl_tensor_type -> cl_tensor _datatype.
linehill Nov 2, 2023
701daa3
* clEnqueue(Read,Write)Tensor -> clEnqueue(TranslateFrom,TranslateTo)…
linehill Nov 2, 2023
aa9ead7
Refactor command buffer temporary property out of tensor
linehill Nov 2, 2023
88a0a84
Fix cl_tensor_type -> cl_tensor_datatype
linehill Nov 2, 2023
37fe006
Add an open question
linehill Nov 2, 2023
0a43252
Add CL_INVALID_TENSOR error code
linehill Nov 2, 2023
d10d149
Require either buffer or host_ptr to be non-NULL
linehill Nov 2, 2023
f40eeda
Regenerate html for cl_exp_tensor
linehill Nov 2, 2023
ef90030
cl_khr_tensor.* -> cl_exp_tensor.*
linehill Nov 9, 2023
ac8499f
* Add overview
linehill Nov 15, 2023
274e76e
Apply suggestions from code review
linehill Nov 15, 2023
2d0687a
* Add command buffer counterparts for tensor translation commands
linehill Nov 16, 2023
e7e99fe
* translate -> import/export
linehill Nov 17, 2023
c1c0221
Update date
linehill Nov 23, 2023
a9c5402
Update cl_exp_tensor, version 0.2.0
linehill Aug 9, 2024
4586eef
Update extensions/cl_exp_tensor.asciidoc
linehill Aug 15, 2024
1911600
Apply suggestions from code review
linehill Aug 15, 2024
af6d58c
Address some feedback, fix formatting
linehill Aug 15, 2024
294b1a1
Add people who gave feedback in the version history
linehill Aug 15, 2024
2293467
Update html render (temporary)
linehill Aug 15, 2024
01a4158
Update extensions/cl_exp_tensor.asciidoc
linehill Aug 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 41 additions & 16 deletions extensions/cl_exp_tensor.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,8 @@ CL_TENSOR_DTYPE_UINT16_EXP 0x????
CL_TENSOR_DTYPE_UINT32_EXP 0x????
CL_TENSOR_DTYPE_UINT64_EXP 0x????

CL_TENSOR_DTYPE_FP8_EXP 0x????
CL_TENSOR_DTYPE_FP8E4M3_EXP 0x????
CL_TENSOR_DTYPE_FP8E5M2_EXP 0x????
CL_TENSOR_DTYPE_FP16_EXP 0x????
CL_TENSOR_DTYPE_FP32_EXP 0x????
CL_TENSOR_DTYPE_FP64_EXP 0x????
Expand Down Expand Up @@ -795,7 +796,17 @@ and true values, respectively.]
| CL_TENSOR_DTYPE_UINT16_EXP | 16-bit unsigned integer. | cl_ushort.
| CL_TENSOR_DTYPE_UINT32_EXP | 32-bit unsigned integer. | cl_uint.
| CL_TENSOR_DTYPE_UINT64_EXP | 64-bit unsigned integer. | cl_ulong.
| CL_TENSOR_DTYPE_FP8_EXP | Half precision floating-point. | cl_char.

| CL_TENSOR_DTYPE_FP8E4M3_EXP | 8-bit floating point with a sign bit,
4 exponent bits, 3 mantissa bits and a exponent bias of 7.
| cl_char.

| CL_TENSOR_DTYPE_FP8E5M2_EXP | 8-bit floating point with a sign bit,
5 exponent bits, 2 mantissa bits and a exponent bias of 15.
| cl_char.

// Reference: https://arxiv.org/pdf/2209.05433

| CL_TENSOR_DTYPE_FP16_EXP | Half precision floating-point. | cl_half.
| CL_TENSOR_DTYPE_BFLOAT16_EXP | 16-bit brain floating-point. | cl_ushort
| CL_TENSOR_DTYPE_FP32_EXP | Single precision floating-point. | cl_float.
Expand All @@ -812,20 +823,29 @@ and true values, respectively.]
|===
| *layout type* | *tensor layout type* | *Description*

| CL_TENSOR_LAYOUT_OPAQUE_EXP | N/A | The tensor doesn't have application
defined memory layout. Driver controls the tensors layout. To read
or write elements of the tensor
| CL_TENSOR_LAYOUT_OPAQUE_EXP | N/A a| The tensor doesn't have
application defined memory layout. Driver controls the tensors
layout. To read or write elements of the tensor, the application
must:

* use *clEnqueueExportToTensor* and *clEnqueueImportFromTensor* (or their
command buffer variants) or
* use *clEnqueueCopyTensor* to copy elements to / from another tensor
object with an application-defined memory layout.

| CL_TENSOR_LAYOUT_BLAS_EXP
|<<cl-tensor-layout-blas,cl_tensor_layout_blas_exp>>
| A type that describes a packed memory layout similar ones used in BLAS APIs.

| CL_TENSOR_LAYOUT_BLAS_EXP
| CL_TENSOR_LAYOUT_BLAS_PITCHED_EXP
|<<cl-tensor-layout-blas,cl_tensor_layout_blas_pitched_exp>>
| A type that describe memory layout similar ones used in BLAS APIs.

| CL_TENSOR_LAYOUT_ML_EXP | <<cl-tensor-layout-ml,cl_tensor_layout_ml_exp>> |
A convenience layout type over `CL_TENSOR_LAYOUT_BLAS_EXP`.
| CL_TENSOR_LAYOUT_ML_EXP | <<cl-tensor-layout-blas,cl_tensor_layout_ml_exp>> |

The tensor layout is specified with an enumerator. Each enumerator
corresponds to a predefined configuration of
*cl_tensor_layout_blas_exp* structure.

|===

Expand Down Expand Up @@ -878,15 +898,15 @@ typedef struct cl_tensor_layout_ml_exp {
rank is greater than two.

// ^ This condition is meant to ensure that the tensor elements at different
// coordinates don't alias.
// coordinates don't alias in memory.

* _ml_type_ defines the memory layout via enumerators which corresponds to
predefined configurations of `cl_tensor_layout_blas_exp` structure
as listed in <<tensor-layout-ml-types,ML tensor layout type>> table.

The memory layout descriptions map tensor coordinates to buffer's
memory byte locations respect to buffer's base address as followed in
pseudo C:
memory byte locations respect to buffer's base address as in the
followed in pseudo C code example:
linehill marked this conversation as resolved.
Show resolved Hide resolved

[source,c]
----
Expand Down Expand Up @@ -1040,14 +1060,19 @@ clEnqueueMapBuffer(

== Version History

[cols="5,15,15,70"]
[cols="5,10,15,40"]
[grid="rows"]
[options="header"]
|====
| Version | Date | Author | Changes
| 0.1.0 | 2023-11-23 | Henry Linjamäki | *Initial revision*

| 0.2.0 | 2024-8-14 | Henry Linjamäki |
| Version | Date | Author | Changes
| 0.1.0 | 2023-11-23 | Henry Linjamäki | *Initial revision*

| 0.2.0 | 2024-8-14 |
Henry Linjamäki +
Pekka Jääskeläinen +
Michal Babej +
Freddie Witherden
a|

* Rework document structure match to the cl_khr_extension_template.

Expand Down
Loading