Skip to content

Commit

Permalink
Remove inverse transform transpose
Browse files Browse the repository at this point in the history
  • Loading branch information
jackhaughton-argon committed Apr 5, 2018
1 parent 05ca78b commit 897fa45
Show file tree
Hide file tree
Showing 2 changed files with 1,920 additions and 130 deletions.
41 changes: 10 additions & 31 deletions 08.decoding.process.md
Original file line number Diff line number Diff line change
Expand Up @@ -4408,20 +4408,6 @@ CurrFrame.

The reconstruction and dequantization process is defined as follows:

The variable transpose is set equal to ( Tx_Width[ txSz ] > Tx_Height[ txSz ] ).

The variable txSize is derived as follows:

* If transpose is equal to 1, txSize is set equal to Transpose_Tx_Size[ txSz ].

* Otherwise (transpose is equal to 0), txSize is set equal to txSz.

The variable txType is derived as follows:

* If transpose is equal to 1, txType is set equal to Transpose_Tx_Type[ PlaneTxType ].

* Otherwise (transpose is equal to 0), txType is set equal to PlaneTxType.

The variable dqDenom is derived as follows:

* If txSz is equal to TX_32X32, TX_16X32, TX_32X16, TX_16X64, or TX_64X16, dqDenom is set equal to 2.
Expand All @@ -4446,11 +4432,11 @@ The variable tw is set equal to Min( 32, w ).

The variable th is set equal to Min( 32, h ).

The variable flipUD is derived as follows. If txType is equal to one of FLIPADST_DCT,
The variable flipUD is derived as follows. If PlaneTxType is equal to one of FLIPADST_DCT,
FLIPADST_ADST, V_FLIPADST, or FLIPADST_FLIPADST, flipUD is set equal to 1. Otherwise,
flipUD is set equal to 0.

The variable flipLR is derived as follows. If txType is equal to one of DCT_FLIPADST,
The variable flipLR is derived as follows. If PlaneTxType is equal to one of DCT_FLIPADST,
ADST_FLIPADST, H_FLIPADST, or FLIPADST_FLIPADST, flipLR is set equal to 1. Otherwise,
flipLR is set equal to 0.

Expand Down Expand Up @@ -4480,7 +4466,7 @@ The following ordered steps apply:
f. Dequant[ i ][ j ] is set equal to Clip3( - ( 1 \<\< ( 7 + BitDepth ) ), ( 1 \<\< ( 7 + BitDepth ) ) - 1, dq2 ).

2. Invoke the 2D inverse transform block process defined in [section 7.12.2][]
with the variables txSize, txType and transpose as input. The inverse transform outputs are stored
with the variable txSz as input. The inverse transform outputs are stored
in the Residual buffer.

3. For i = 0..(h-1), for j = 0..(w-1), the following applies:
Expand All @@ -4490,7 +4476,7 @@ The following ordered steps apply:
* The variable yy is set equal to flipUD ? ( h - i - 1 ) : i.

* CurrFrame[ plane ][ y + yy ][ x + xx ] is set equal to
Clip1( CurrFrame[ plane ][ y + yy ][ x + xx ] + ( transpose ? Residual[ j ][ i ] : Residual[ i ][ j ] ) ).
Clip1( CurrFrame[ plane ][ y + yy ][ x + xx ] + Residual[ i ][ j ] ).

It is a requirement of bitstream conformance that the values written into the
Residual array in step 2 are representable by a signed integer with 8 + BitDepth bits.
Expand Down Expand Up @@ -5042,13 +5028,7 @@ The process to invoke depends on n as follows:
This process performs a 2D inverse transform for an array of coefficients stored in the
2D array Dequant. The output is placed in the 2D array Residual.

The inputs to this process are:

* a variable txSz that specifies the transform size,

* a variable txType that specifies the transform type,

* a variable transpose, which, if equal to 1, signifies that the transform should be transposed.
The input to this process is a variable txSz that specifies the transform size.

Set the variable log2W equal to Tx_Width_Log2[ txSz ].

Expand All @@ -5070,8 +5050,7 @@ The row transforms with i = 0..(h-1) are applied as follows:

* T[ j ] is derived as follows for j = 0..(w-1):

* If i and j are both less than 32, T[ j ] is set equal to ( transpose ? Dequant[ j ][ i ]
\: Dequant[ i ][ j ] ).
* If i and j are both less than 32, T[ j ] is set equal to Dequant[ i ][ j ].

* Otherwise, T[ j ] is set equal to 0.

Expand All @@ -5081,11 +5060,11 @@ The row transforms with i = 0..(h-1) are applied as follows:
* If Lossless is equal to 1, invoke the Inverse WHT process as specified in
[section 7.12.1.10][] with shift equal to 2.

* Otherwise, if txType is equal to one of DCT_DCT, ADST_DCT, FLIPADST_DCT or H_DCT,
* Otherwise, if PlaneTxType is equal to one of DCT_DCT, ADST_DCT, FLIPADST_DCT or H_DCT,
invoke the inverse DCT process as specified in [section 7.12.1.3][] with the
input variable n equal to log2W and the input variable r equal to rowClampRange.

* Otherwise, if txType is equal to one of DCT_ADST, ADST_ADST, DCT_FLIPADST, FLIPADST_FLIPADST,
* Otherwise, if PlaneTxType is equal to one of DCT_ADST, ADST_ADST, DCT_FLIPADST, FLIPADST_FLIPADST,
ADST_FLIPADST, FLIPADST_ADST, H_ADST, or H_FLIPADST, invoke the inverse ADST process as
specified in [section 7.12.1.9][] with input variable n equal to log2W and the input variable
r equal to rowClampRange.
Expand All @@ -5105,12 +5084,12 @@ The column transforms with j = 0..(w-1) are applied as follows:
* If Lossless is equal to 1, invoke the Inverse WHT process as specified in
[section 7.12.1.10][] with shift equal to 0.

* Otherwise, if txType is equal to one of DCT_DCT, DCT_ADST, DCT_FLIPADST or
* Otherwise, if PlaneTxType is equal to one of DCT_DCT, DCT_ADST, DCT_FLIPADST or
V_DCT, invoke the inverse DCT process as specified in [section 7.12.1.3][]
with the input variable n equal to log2H and the input variable r equal to
colClampRange.

* Otherwise, if txType is equal to one of ADST_DCT, ADST_ADST, FLIPADST_DCT,
* Otherwise, if PlaneTxType is equal to one of ADST_DCT, ADST_ADST, FLIPADST_DCT,
FLIPADST_FLIPADST, ADST_FLIPADST, FLIPADST_ADST, V_ADST, or V_FLIPADST,
invoke the inverse ADST process as specified in [section 7.12.1.9][] with input variable n equal to
log2H and the input variable r equal to
Expand Down
Loading

0 comments on commit 897fa45

Please sign in to comment.