It seems that PyTorch has updated the underlying sampling algorithm for torch.multinomial in modern 2.x versions. Due to this, despite having the exact same structure for the 2D bi-gram tensor N and using the exact same code/seed from the video, the first generation yields an index of 3 (character 'c') instead of 13 (character 'm').
The math for generating the initial random uniform decimals is the same(which is why you would get same results for the example he shows using torch.rand), but the way PyTorch 2.x maps those decimals to discrete bins in multinomial has changed since the video was recorded (which used PyTorch 1.12).
The Fix:
To perfectly replicate the deterministic outputs from the tutorial, use a virtual environment with Python 3.10 and install the older dependencies. I have tested this and the outputs match the video exactly. Copy paste the following into requirements.txt should do the trick.
torch==1.12.1
torchvision==0.13.1
torchaudio==0.12.1
numpy<2.0
matplotlib
ipykernel
tensorboard
It seems that PyTorch has updated the underlying sampling algorithm for torch.multinomial in modern 2.x versions. Due to this, despite having the exact same structure for the 2D bi-gram tensor N and using the exact same code/seed from the video, the first generation yields an index of 3 (character 'c') instead of 13 (character 'm').
The math for generating the initial random uniform decimals is the same(which is why you would get same results for the example he shows using torch.rand), but the way PyTorch 2.x maps those decimals to discrete bins in multinomial has changed since the video was recorded (which used PyTorch 1.12).
The Fix:
To perfectly replicate the deterministic outputs from the tutorial, use a virtual environment with Python 3.10 and install the older dependencies. I have tested this and the outputs match the video exactly. Copy paste the following into requirements.txt should do the trick.
torch==1.12.1
torchvision==0.13.1
torchaudio==0.12.1
numpy<2.0
matplotlib
ipykernel
tensorboard