松明

pytorchドット積



Pytorch Dot Product



aとbの次元は同じである必要はなく、サブ次元と同じにすることもできます。

import torch a = torch.Tensor([0.5,2]).reshape( 2) b = torch.Tensor(range(1, 25)).reshape(3,2,1,4) print(a[:,None]) print(b) # print(b.mul(a[None,:,None,None,])) print(b.mul(a[:,None,None])) # print(b.mul(a[None,:,None,None,]))

注意を払う：



次のコードはエラーを報告し、寸法に一貫性がありません。

import torch a = torch.Tensor(range(6)).reshape(2, 3) b = torch.Tensor(range(1, 7)).reshape(2, 3) batch = len(a) length = len(a[0]) c = torch.zeros(batch, batch, length) start = time.time() for i in range(batch): for j in range(batch): c[i][j] = a[i] * b[j] It can be seen that in fact each line of a is multiplied by each line of b as an element of c, and finally c is a three-dimensional array. But the problem is that the execution speed of the Python interpreter is slow, so I made improvements and expanded a and b respectively according to dimension 1 and dimension, and then multiplied it, the result is the same. a_batch = torch.stack([a] * batch, dim=1) b_batch = torch.stack([b] * batch, dim=0) c_batch = a_batch * b_batch The speed is obviously much faster, but it is more memory intensive and cumbersome. Looking back at the solution of a student's code, using the None extension to solve this problem conveniently. a_2 = a[:, None, :] b_2 = b[None, :, :] c_2 = a_2 * b_2 In fact, the dimension size of a_2 and b_2 is 1 in the dimension of None instead of the number of my stack. The respective dimensions of the output are as follows a.shape torch.Size([2, 3]) a_batch.shape torch.Size([2, 2, 3]) a_2.shape torch.Size([2, 1, 3]) b.shape torch.Size([2, 3]) b_batch.shape torch.Size([2, 2, 3]) b_2.shape torch.Size([1, 2, 3]) The doubt is that the csdn blog I read before said that the matrix dot multiplication of pytorch requires that the dimensions of the two matrices are the same. However, why can the dimensions a_2 and b_2 be multiplied? So check the official documentation of pytorch. That is to see the documentation of torch.mul https://pytorch.org/docs/stable/torch.html?highlight=mul#torch.mul, visible It can be seen that in fact, when the dimensions are different, if the matrix is ​​broadcastable, you can also multiply, see the definition of broadcastable https://pytorch.org/docs/stable/notes/broadcasting.html#broadcasting-semantics Chinese translation is If the following rules hold, the two tensors are 'broadcastable': Each tensor has at least one dimension. When iterating the sizes, starting from the tail size, the size must be equal, or one of the sizes is 1, or the size does not exist. Official example >>> x=torch.empty(5,7,3) >>> y=torch.empty(5,7,3) # same shapes are always broadcastable (i.e. the above rules always hold) >>> x=torch.empty((0,)) >>> y=torch.empty(2,2) # x and y are not broadcastable, because x does not have at least 1 dimension # can line up trailing dimensions >>> x=torch.empty(5,3,4,1) >>> y=torch.empty( 3,1,1) # x and y are broadcastable. # 1st trailing dimension: both have size 1 # 2nd trailing dimension: y has size 1 # 3rd trailing dimension: x size == y size # 4th trailing dimension: y dimension doesn't exist # but: >>> x=torch.empty(5,2,4,1) >>> y=torch.empty( 3,1,1) # x and y are not broadcastable, because in the 3rd trailing dimension 2 != 3

転載：



https://blog.csdn.net/zjhao666/article/details/97756377 

|_+_|