2. Getting Started
  3. Introduction
  4. 🧩 Puzzles Usage Guide
  6. Part I: GPU Fundamentals
  7. Puzzle 1: Map
    1. 🔰 Raw Memory Approach
    2. 💡 Preview: Modern Approach with LayoutTensor
  9. Puzzle 2: Zip
  10. Puzzle 3: Guards
  11. Puzzle 4: 2D Map
    1. 🔰 Raw Memory Approach
    2. 📚 Learn about LayoutTensor
    3. 🚀 Modern 2D Operations
  13. Puzzle 5: Broadcast
    1. 🔰 Raw Memory Approach
    2. 📐 LayoutTensor Version
  15. Puzzle 6: Blocks
  16. Puzzle 7: 2D Blocks
    1. 🔰 Raw Memory Approach
    2. 📐 LayoutTensor Version
  18. Puzzle 8: Shared Memory
    1. 🔰 Raw Memory Approach
    2. 📐 LayoutTensor Version
  21. Part II: GPU Algorithms
  22. Puzzle 9: Pooling
    1. 🔰 Raw Memory Approach
    2. 📐 LayoutTensor Version
  24. Puzzle 10: Dot Product
    1. 🔰 Raw Memory Approach
    2. 📐 LayoutTensor Version
  26. Puzzle 11: 1D Convolution
    1. 🔰 Simple Version
    2. ⭐ Complete Version
  28. Puzzle 12: Prefix Sum
    1. 🔰 Simple Version
    2. ⭐ Complete Version
  30. Puzzle 13: Axis Sum
  31. Puzzle 14: Matrix Multiplication (MatMul)
    1. 🔰 Naive Version with Global Memory
    2. 📚 Learn about Roofline Model
    3. 🤝 Shared Memory Version
    4. 📐 Tiled Version
  34. Part III: Interfacing with Python via MAX Graph Custom Ops
  35. Puzzle 15: 1D Convolution Op
  36. Puzzle 16: Softmax Op
  37. Puzzle 17: Attention Op
  38. 🎯 Bonus Challenges
  40. Part IV: Advanced GPU Algorithms
  41. Puzzle 18: 2D Convolution Op
  42. Puzzle 19: 3D Average Pooling
    1. 📚 Learn about 3D Memory Layout
    2. Basic Version
    3. LayoutTensor Version
  44. Puzzle 20: 3D Convolution
    1. 📚 Learn about 3D Convolution
    2. Basic Version
    3. Optimized Version
  46. Puzzle 21: 3D Tensor Multiplication
    1. 📚 Learn about Tensor Operations
    2. Basic Version
    3. LayoutTensor Version
  48. Puzzle 22: Multi-Head Self-Attention
    1. 📚 Learn about Attention Mechanisms
    2. Basic Version
    3. Optimized Version
  51. Part V: Performance Optimization Puzzles
  52. Puzzle 23: Memory Coalescing
    1. 📚 Learn about Memory Access Patterns
    2. Basic Version
    3. Optimized Version
  54. Puzzle 24: Bank Conflicts
    1. 📚 Learn about Shared Memory Banks
    2. Version 1: With Conflicts
    3. Version 2: Conflict-Free
  56. Puzzle 25: Warp-Level Optimization
    1. 📚 Learn about Warp Primitives
    2. Version 1: Shared Memory Reduction
    3. Version 2: Warp Shuffle Reduction
  59. Part VI: Real-world Application Puzzles
  60. Puzzle 26: Image Processing Pipeline
    1. 📚 Learn about Kernel Fusion
    2. Version 1: Separate Kernels
    3. Version 2: Fused Pipeline
  62. Puzzle 27: Neural Network Layers
    1. 📚 Learn about Layer Fusion
    2. Version 1: Basic Implementation
    3. Version 2: Optimized Implementation
  64. Puzzle 28: Multi-Level Tiling
    1. 📚 Learn about Cache Hierarchies
    2. Version 1: Single-Level MatMul
    3. Version 2: Multi-Level MatMul
  67. Part VII: Debug & Profile Puzzles
  68. Puzzle 29: Race Condition Detective
    1. 📚 Learn about Race Conditions
    2. Version 1: Find the Bug
    3. Version 2: Fix the Bug
  70. Puzzle 30: Memory Optimization
    1. 📚 Learn about Memory Management
    2. Version 1: Memory Leaks
    3. Version 2: Memory Planning
  73. Part VIII: Modern GPU Features
  74. Puzzle 31: Dynamic Parallelism
    1. 📚 Learn about Nested Parallelism
    2. Version 1: Flat Implementation
    3. Version 2: Nested Launch
  76. Puzzle 32: Tensor Core Programming
    1. 📚 Learn about Tensor Cores
    2. Version 1: Regular MatMul
    3. Version 2: Tensor Core MatMul
  78. Puzzle 33: Multi-GPU Programming
    1. 📚 Learn about Device Communication
    2. Version 1: Single GPU
    3. Version 2: Multi-GPU