Introduction to Computing

This chapter covers how programs execute on modern hardware, from the fetch-decode-execute cycle to distributed GPU clusters.

• How Programs Execute: The fetch-decode-execute cycle, registers, memory hierarchy, and latency numbers every programmer should know
• CPU and GPU Architecture: CPU vs GPU design, SIMT execution, thread hierarchy, memory types, and bandwidth vs latency
• From Assembly to PyTorch: The abstraction stack: how torch.matmul becomes GPU instructions
• High-Performance Computing: MPI, NCCL, distributed training topology, and profiling with nsys