Exercise 5&6 00_fundamentals - random seeds (solution spoiler)

[danielLutziger]

I have a question regarding the seeds in torch. I have attached 3 different versions of the code, together with their output. What I understand is, that when I use torch.manual_seed(1234) after every line, then the same tensor is generated, as expected. However, if I just use it in the beginning then I would expect t1 to be created with seed 1234 and t2 to be random. This does not seem to be the case, due to the torch.cuda.manual_seed(1234), which seems to set t2. If I however remove the torch.manual_seed(1234) then I would expect the first tensor to be the content of t2, however, it then seems to be completely random.

I didn't get the correlation from the docs (manual seed docs (where I looked)
cuda docs (where I looked))

I attached the 3 code snippets below and marked the tensors I would expect to be the same in bold.

# ex 5
torch.cuda.manual_seed(1234)

# ex 6

device = "cuda" if torch.cuda.is_available() else "cpu"
torch.manual_seed(1234)
t1 = torch.rand(2,3).to(device)
torch.manual_seed(1234)
t2 = torch.rand(2,3).to(device)
t1, t2

# as expected
(tensor([[0.0290, 0.4019, 0.2598],
         [0.3666, 0.0583, 0.7006]], device='cuda:0'),
 tensor([[0.0290, 0.4019, 0.2598],
         [0.3666, 0.0583, 0.7006]], device='cuda:0'))

# ex 5
torch.cuda.manual_seed(1234)

# ex 6

device = "cuda" if torch.cuda.is_available() else "cpu"
torch.manual_seed(1234)
t1 = torch.rand(2,3).to(device)
t2 = torch.rand(2,3).to(device)
t1, t2

# first one as expected, 2nd one "random", but why does the sample solution have the exact same 2nd one, thus not random?
(tensor([[0.0290, 0.4019, 0.2598],
         [0.3666, 0.0583, 0.7006]], device='cuda:0'),
 tensor([[0.0518, 0.4681, 0.6738],
         [0.3315, 0.7837, 0.5631]], device='cuda:0'))

# ex 5
torch.cuda.manual_seed(1234)

# ex 6

device = "cuda" if torch.cuda.is_available() else "cpu"
t1 = torch.rand(2,3).to(device)
t2 = torch.rand(2,3).to(device)
t1, t2

# both are random, why isnt the first one the same as the 2nd one from above?
(tensor([[0.8436, 0.4265, 0.9561],
         [0.0770, 0.4108, 0.0014]], device='cuda:0'),
 tensor([[0.5414, 0.6419, 0.2976],
         [0.7077, 0.4189, 0.0655]], device='cuda:0'))

[danielLutziger]

Nevermind... I got it. The seeding is applied once, then all the data is generated with that seed, meaning that in the first example the seed is basically reset. In the second example the seed is not set and the data generation just continues with the initialized seed. In the last example, no seed is set.