Quiz 2.4 Part 3

[gracecluvohio]

Hi,
Here's the part 3 quiz question for reference:
Suppose we have two matrices,

A = [[1, 2], [3, 4]]

and

B = [[5, 6], [7, 8]].

If we carry out the matrix multiplication C = AB, which results in a 2×2 matrix, what is the output value in the upper right?

In the part 3 video, it said we had to transpose the weight matrix. So for the question, we would be multiplying the rows of the input matrix and the columns of the transposed weight matrix, which would look like this:
input matrix:
1 2
3 4

weight matrix:
5 7
6 8

So then shouldn't the output value in the upper right be the dot product between the input matrix first row, [1,2] and weight matrix second row, [7,8], which is 23? However, the answer seems to imply that I don't need to transpose the weight matrix because the answer is 22, which is the dot product between [1,2] and [6,8] (the second row of the untransposed weight matrix).

[rasbt]

Great question! In the video, I was specifically referring to the case where we multiply the inputs $X$ with the weights $W$. So we have $X.W^T$ (which it is how it's implemented in PyTorch's Linear layer based on how the weights are stored in $W$).

For the quiz, I was just thinking of a regular matrix multiplications between to matrices $A.B$, not $A.B^T$. Sorry if this was confusing!

[gracecluvohio]

That makes a lot more sense, thank you!