Complete RNG State Management in PyTorch

Reproducibility in PyTorch requires understanding that multiple random number generators operate independently. Setting torch.manual_seed() is just the beginning.

The Three RNG Sources

• . **PyTorch CPU RNG** - torch.manual_seed()
• . **PyTorch CUDA RNG** - torch.cuda.manual_seed_all()
• . **NumPy RNG** - numpy.random.seed()
• . **Python RNG** - random.seed()

Each operates independently. A model might use torch for tensor operations, numpy for data augmentation, and Python's random for shuffling.

Capturing Complete State

def capture_rng_state():
    return {
        'torch': torch.get_rng_state(),
        'cuda': torch.cuda.get_rng_state_all(),
        'numpy': numpy.random.get_state(),
        'python': random.getstate(),

def restore_rng_state(state): torch.set_rng_state(state['torch']) torch.cuda.set_rng_state_all(state['cuda']) numpy.random.set_state(state['numpy']) random.setstate(state['python']) ```

When to Capture

Capture state immediately before the operation you want to reproduce. Restore it before each repetition.

This is the foundation of deterministic-nodes' RNG locking mechanism.