`warp.broadcast()` One-to-Many Communication

For warp-level coordination we can use broadcast() to share data from one lane to all other lanes within a warp. This powerful primitive enables efficient block-level computations, conditional logic coordination, and one-to-many communication patterns without shared memory or explicit synchronization.

Key insight: The broadcast() operation leverages SIMT execution to let lane 0 share its computed value with all other lanes in the same warp, enabling efficient coordination patterns and collective decision-making.

What are broadcast operations? Broadcast operations are communication patterns where one thread computes a value and shares it with all other threads in a group. This is essential for coordination tasks like computing block-level statistics, making collective decisions, or sharing configuration parameters across all threads in a warp.

Key concepts

In this puzzle, you’ll learn:

Warp-level broadcasting with broadcast()
One-to-many communication patterns
Collective computation strategies
Conditional coordination across lanes
Combined broadcast-shuffle operations

The broadcast operation enables lane 0 to share its value with all other lanes. The source lane is always lane 0—there is no parameter to change it: \[\Large \text{broadcast}(\text{value}) = \text{value from lane 0, given to every lane}\]

This transforms complex coordination patterns into simple warp-level operations, enabling efficient collective computations without explicit synchronization.

The broadcast concept

Traditional coordination requires complex shared memory patterns:

# Traditional approach - complex and error-prone
shared_memory[lane] = local_computation()
barrier()  # Expensive synchronization
if lane == 0:
    shared_memory[0] = compute_from_shared_memory()
barrier()  # Another expensive synchronization
var final_result = shared_memory[0]  # All threads read

Problems with traditional approach:

Memory overhead: Requires shared memory allocation
Synchronization: Multiple expensive barrier operations
Complex logic: Managing shared memory indices and access patterns
Error-prone: Easy to introduce race conditions

With broadcast(), coordination becomes elegant:

# Warp broadcast approach - simple and safe
var collective_value = 0
if lane == 0:
    collective_value = compute_block_statistic()
collective_value = broadcast(collective_value)  # Share with all lanes
var result = use_collective_value(collective_value)

Benefits of broadcast:

Zero memory overhead: No shared memory required
No barrier needed: broadcast() lowers to a single warp shuffle instruction, which synchronizes the participating lanes itself
Simple pattern: One lane computes, all lanes receive
Composable: Easy to combine with other warp operations

1. Basic broadcast

Implement a basic broadcast pattern where lane 0 computes a block-level statistic and shares it with all lanes.

Requirements:

Lane 0 should compute the sum of the first 4 elements in the current block
This computed value must be shared with all other lanes in the warp using broadcast()
Each lane should then add this shared value to its own input element

Test data: Input [1, 2, 3, 4, 5, 6, 7, 8, ...] should produce output [11, 12, 13, 14, 15, 16, 17, 18, ...]

Challenge: How do you coordinate so that only one lane does the block-level computation, but all lanes can use the result in their individual operations?

Configuration

Vector size: SIZE = WARP_SIZE (32 or 64 depending on GPU)
Grid configuration: (1, 1) blocks per grid
Block configuration: (WARP_SIZE, 1) threads per block
Data type: DType.float32
Layout: row_major[SIZE]() (1D row-major)

Code to complete

def basic_broadcast[
    size: Int
](
    output: TileTensor[mut=True, dtype, LayoutType, MutAnyOrigin],
    input: TileTensor[mut=False, dtype, LayoutType, MutAnyOrigin],
):
    """
    Basic broadcast: Lane 0 computes a block-local value, broadcasts it to all lanes.
    Each lane then uses this broadcast value in its own computation.
    """
    var global_i = block_dim.x * block_idx.x + thread_idx.x
    var lane = Int(lane_id())

    if global_i < size:
        var broadcast_value: output.ElementType = 0.0

        # FILL IN (roughly 10 lines)

View full file: problems/p25/p25.mojo

Tips

1. Understanding broadcast mechanics

The broadcast(value) operation takes the value from lane 0 and distributes it to all lanes in the warp.

Key insight: Only lane 0’s value matters for the broadcast. Other lanes’ values are ignored, but all lanes receive lane 0’s value.

Visualization:

Before broadcast: Lane 0 has val_0, Lane 1 has val_1, Lane 2 has val_2, ...
After broadcast:  Lane 0 has val_0, Lane 1 has val_0, Lane 2 has val_0, ...

Think about: How can you ensure only lane 0 computes the value you want to broadcast?

2. Lane-specific computation

Design your algorithm so that lane 0 performs the special computation while other lanes wait.

Pattern to consider:

var shared_value = initial_value
if lane == 0:
    # Only lane 0 computes
    shared_value = special_computation()
# All lanes participate in broadcast
shared_value = broadcast(shared_value)

Critical questions:

What should other lanes’ values be before the broadcast?
How do you ensure lane 0 has the correct value to broadcast?

3. Collective usage

After broadcasting, all lanes have the same value and can use it in their individual computations.

Think about: How does each lane combine the broadcast value with its own local data?

Test the basic broadcast:

pixi run p25 --broadcast-basic

pixi run -e amd p25 --broadcast-basic

pixi run -e apple p25 --broadcast-basic

uv run poe p25 --broadcast-basic

Expected output when solved:

WARP_SIZE:  32
SIZE:  32
output: HostBuffer([11.0, 12.0, 13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.0, 29.0, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0])
expected: HostBuffer([11.0, 12.0, 13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.0, 29.0, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0])
Basic broadcast test: passed
Puzzle 25 complete ✅

Solution

def basic_broadcast[
    size: Int
](
    output: TileTensor[mut=True, dtype, LayoutType, MutAnyOrigin],
    input: TileTensor[mut=False, dtype, LayoutType, MutAnyOrigin],
):
    """
    Basic broadcast: Lane 0 computes a block-local value, broadcasts it to all lanes.
    Each lane then uses this broadcast value in its own computation.
    """
    var global_i = block_dim.x * block_idx.x + thread_idx.x
    var lane = Int(lane_id())

    if global_i < size:
        # Step 1: Lane 0 computes special value (sum of first 4 elements in this block)
        var broadcast_value: output.ElementType = 0.0
        if lane == 0:
            var block_start = block_idx.x * block_dim.x
            var sum: output.ElementType = 0.0
            for i in range(4):
                if block_start + i < size:
                    sum += input[block_start + i]
            broadcast_value = sum

        # Step 2: Broadcast lane 0's value to all lanes in this warp
        broadcast_value = broadcast(broadcast_value)

        # Step 3: All lanes use broadcast value in their computation
        output[global_i] = broadcast_value + input[global_i]

This solution demonstrates the fundamental broadcast pattern for warp-level coordination.

Algorithm breakdown:

if global_i < size:
    # Step 1: Lane 0 computes special value
    var broadcast_value: output.ElementType = 0.0
    if lane == 0:
        # Only lane 0 performs this computation
        var block_start = block_idx.x * block_dim.x
        var sum: output.ElementType = 0.0
        for i in range(4):
            if block_start + i < size:
                sum += input[block_start + i]
        broadcast_value = sum

    # Step 2: Share lane 0's value with all lanes
    broadcast_value = broadcast(broadcast_value)

    # Step 3: All lanes use the broadcast value
    output[global_i] = broadcast_value + input[global_i]

SIMT execution trace:

Step 1: Lane-specific computation
  Lane 0: Computes sum of input[0] + input[1] + input[2] + input[3] = 1+2+3+4 = 10
  Lane 1: broadcast_value remains 0.0 (not lane 0)
  Lane 2: broadcast_value remains 0.0 (not lane 0)
  ...
  Lane 31: broadcast_value remains 0.0 (not lane 0)

Step 2: broadcast(broadcast_value) executes
  Lane 0: Keeps its value → broadcast_value = 10.0
  Lane 1: Receives lane 0's value → broadcast_value = 10.0
  Lane 2: Receives lane 0's value → broadcast_value = 10.0
  ...
  Lane 31: Receives lane 0's value → broadcast_value = 10.0

Step 3: Individual computation with broadcast value
  Lane 0: output[0] = 10.0 + input[0] = 10.0 + 1.0 = 11.0
  Lane 1: output[1] = 10.0 + input[1] = 10.0 + 2.0 = 12.0
  Lane 2: output[2] = 10.0 + input[2] = 10.0 + 3.0 = 13.0
  ...
  Lane 31: output[31] = 10.0 + input[31] = 10.0 + 32.0 = 42.0

Why broadcast is superior:

Coordination efficiency: Single operation coordinates all lanes
Memory efficiency: No shared memory allocation required
Barrier-free: The shuffle instruction synchronizes the lanes it reads from, so no barrier() is needed
Scalable pattern: Works identically regardless of warp size

Performance characteristics:

Instruction count: One shuffle instruction - broadcast(val) is shuffle_idx(val, 0)
Bandwidth: 0 bytes (register-to-register communication)
Coordination: Every lane in the warp receives the value

2. Conditional broadcast

Implement conditional coordination where lane 0 analyzes block data and makes a decision that affects all lanes.

Requirements:

Lane 0 should analyze the first 8 elements in the current block and find their maximum value
This maximum value must be broadcast to all other lanes using broadcast()
Each lane should then apply conditional logic: if their element is above half the maximum, double it; otherwise, halve it

Test data: Input [3, 1, 7, 2, 9, 4, 6, 8, ...] (repeating pattern) should produce output [1.5, 0.5, 14.0, 1.0, 18.0, 2.0, 12.0, 16.0, ...]

Challenge: How do you coordinate block-level analysis with element-wise conditional transformations across all lanes?

Configuration

Vector size: SIZE = WARP_SIZE (32 or 64 depending on GPU)
Grid configuration: (1, 1) blocks per grid
Block configuration: (WARP_SIZE, 1) threads per block

Code to complete

def conditional_broadcast[
    size: Int
](
    output: TileTensor[mut=True, dtype, LayoutType, MutAnyOrigin],
    input: TileTensor[mut=False, dtype, LayoutType, MutAnyOrigin],
):
    """
    Conditional broadcast: Lane 0 makes a decision based on block-local data, broadcasts it to all lanes.
    All lanes apply different logic based on the broadcast decision.
    """
    var global_i = block_dim.x * block_idx.x + thread_idx.x
    var lane = Int(lane_id())

    if global_i < size:
        var decision_value: output.ElementType = 0.0

        # FILL IN (roughly 10 lines)

        var current_input = input[global_i]
        var threshold = decision_value / 2.0
        if current_input >= threshold:
            output[global_i] = current_input * 2.0  # Double if >= threshold
        else:
            output[global_i] = current_input / 2.0  # Halve if < threshold

Tips

1. Analysis and decision-making

Lane 0 needs to analyze multiple data points and make a decision that will guide all other lanes.

Key questions:

How can lane 0 efficiently analyze multiple elements?
What kind of decision should be broadcast to coordinate lane behavior?
How do you handle boundary conditions when analyzing data?

Pattern to consider:

var decision = default_value
if lane == 0:
    # Analyze block-local data
    decision = analyze_and_decide()
decision = broadcast(decision)

2. Conditional execution coordination

After receiving the broadcast decision, all lanes need to apply different logic based on the decision.

Think about:

How do lanes use the broadcast value to make local decisions?
What operations should be applied in each conditional branch?
How do you ensure consistent behavior across all lanes?

Conditional pattern:

if (local_data meets_broadcast_criteria):
    # Apply one transformation
else:
    # Apply different transformation

3. Data analysis strategies

Consider efficient ways for lane 0 to analyze multiple data points.

Approaches to consider:

Finding maximum/minimum values
Computing averages or sums
Detecting patterns or thresholds
Making binary decisions based on data characteristics

Test the conditional broadcast:

pixi run p25 --broadcast-conditional

pixi run -e amd p25 --broadcast-conditional

pixi run -e apple p25 --broadcast-conditional

uv run poe p25 --broadcast-conditional

Expected output when solved:

WARP_SIZE:  32
SIZE:  32
output: HostBuffer([1.5, 0.5, 14.0, 1.0, 18.0, 2.0, 12.0, 16.0, 1.5, 0.5, 14.0, 1.0, 18.0, 2.0, 12.0, 16.0, 1.5, 0.5, 14.0, 1.0, 18.0, 2.0, 12.0, 16.0, 1.5, 0.5, 14.0, 1.0, 18.0, 2.0, 12.0, 16.0])
expected: HostBuffer([1.5, 0.5, 14.0, 1.0, 18.0, 2.0, 12.0, 16.0, 1.5, 0.5, 14.0, 1.0, 18.0, 2.0, 12.0, 16.0, 1.5, 0.5, 14.0, 1.0, 18.0, 2.0, 12.0, 16.0, 1.5, 0.5, 14.0, 1.0, 18.0, 2.0, 12.0, 16.0])
Conditional broadcast test: passed
Puzzle 25 complete ✅

Solution

def conditional_broadcast[
    size: Int
](
    output: TileTensor[mut=True, dtype, LayoutType, MutAnyOrigin],
    input: TileTensor[mut=False, dtype, LayoutType, MutAnyOrigin],
):
    """
    Conditional broadcast: Lane 0 makes a decision based on block-local data, broadcasts it to all lanes.
    All lanes apply different logic based on the broadcast decision.
    """
    var global_i = block_dim.x * block_idx.x + thread_idx.x
    var lane = Int(lane_id())

    if global_i < size:
        # Step 1: Lane 0 analyzes block-local data and makes decision (find max of first 8 in block)
        var decision_value: output.ElementType = 0.0
        if lane == 0:
            var block_start = block_idx.x * block_dim.x
            decision_value = input[block_start] if block_start < size else 0.0
            for i in range(1, min(8, min(WARP_SIZE, size - block_start))):
                if block_start + i < size:
                    var current_val = input[block_start + i]
                    if current_val > decision_value:
                        decision_value = current_val

        # Step 2: Broadcast decision to all lanes in this warp
        decision_value = broadcast(decision_value)

        # Step 3: All lanes apply conditional logic based on broadcast decision
        var current_input = input[global_i]
        var threshold = decision_value / 2.0
        if current_input >= threshold:
            output[global_i] = current_input * 2.0  # Double if >= threshold
        else:
            output[global_i] = current_input / 2.0  # Halve if < threshold

This solution demonstrates advanced broadcast patterns for conditional coordination across lanes.

Complete algorithm analysis:

if global_i < size:
    # Step 1: Lane 0 analyzes block data and makes decision
    var decision_value: output.ElementType = 0.0
    if lane == 0:
        # Find maximum among first 8 elements in block
        var block_start = block_idx.x * block_dim.x
        decision_value = input[block_start] if block_start < size else 0.0
        for i in range(1, min(8, min(WARP_SIZE, size - block_start))):
            if block_start + i < size:
                var current_val = input[block_start + i]
                if current_val > decision_value:
                    decision_value = current_val

    # Step 2: Broadcast decision to coordinate all lanes
    decision_value = broadcast(decision_value)

    # Step 3: All lanes apply conditional logic based on broadcast
    var current_input = input[global_i]
    var threshold = decision_value / 2.0
    if current_input >= threshold:
        output[global_i] = current_input * 2.0  # Double if >= threshold
    else:
        output[global_i] = current_input / 2.0  # Halve if < threshold

Decision-making execution trace:

Input data: [3.0, 1.0, 7.0, 2.0, 9.0, 4.0, 6.0, 8.0, ...]

Step 1: Lane 0 finds maximum of first 8 elements
  Lane 0 analysis:
    Start with input[0] = 3.0
    Compare with input[1] = 1.0 → keep 3.0
    Compare with input[2] = 7.0 → update to 7.0
    Compare with input[3] = 2.0 → keep 7.0
    Compare with input[4] = 9.0 → update to 9.0
    Compare with input[5] = 4.0 → keep 9.0
    Compare with input[6] = 6.0 → keep 9.0
    Compare with input[7] = 8.0 → keep 9.0
    Final decision_value = 9.0

Step 2: Broadcast decision_value = 9.0 to all lanes
  All lanes now have: decision_value = 9.0, threshold = 4.5

Step 3: Conditional execution per lane
  Lane 0: input[0] = 3.0 < 4.5 → output[0] = 3.0 / 2.0 = 1.5
  Lane 1: input[1] = 1.0 < 4.5 → output[1] = 1.0 / 2.0 = 0.5
  Lane 2: input[2] = 7.0 ≥ 4.5 → output[2] = 7.0 * 2.0 = 14.0
  Lane 3: input[3] = 2.0 < 4.5 → output[3] = 2.0 / 2.0 = 1.0
  Lane 4: input[4] = 9.0 ≥ 4.5 → output[4] = 9.0 * 2.0 = 18.0
  Lane 5: input[5] = 4.0 < 4.5 → output[5] = 4.0 / 2.0 = 2.0
  Lane 6: input[6] = 6.0 ≥ 4.5 → output[6] = 6.0 * 2.0 = 12.0
  Lane 7: input[7] = 8.0 ≥ 4.5 → output[7] = 8.0 * 2.0 = 16.0
  ...pattern repeats for remaining lanes

Mathematical foundation: This implements a threshold-based transformation: \[\Large f(x) = \begin{cases} 2x & \text{if } x \geq \tau \\ \frac{x}{2} & \text{if } x < \tau \end{cases}\]

Where \(\tau = \frac{\max(\text{block_data})}{2}\) is the broadcast threshold.

Coordination pattern benefits:

Centralized analysis: One lane analyzes, all lanes benefit
Consistent decisions: All lanes use the same threshold
Adaptive behavior: Threshold adapts to block-local data characteristics
Efficient coordination: Single broadcast coordinates complex conditional logic

Applications:

Adaptive algorithms: Adjusting parameters based on local data characteristics
Quality control: Applying different processing based on data quality metrics
Load balancing: Distributing work based on block-local complexity analysis

3. Broadcast-shuffle coordination

Implement advanced coordination combining both broadcast() and shuffle_down() operations.

Requirements:

Lane 0 should compute the average of the first 4 elements in the block and broadcast this scaling factor to all lanes
Each lane should use shuffle_down(offset=1) to get their next neighbor’s value
For most lanes: multiply the scaling factor by (current_value + next_neighbor_value)
For the last lane in the warp: multiply the scaling factor by just current_value (no valid neighbor)

Test data: Input follows pattern [2, 4, 6, 8, 1, 3, 5, 7, ...] (first 4 elements: 2,4,6,8 then repeating 1,3,5,7)

Lane 0 computes scaling factor: (2+4+6+8)/4 = 5.0
Expected output: [30.0, 50.0, 70.0, 45.0, 20.0, 40.0, 60.0, 40.0, ...]

Challenge: How do you coordinate multiple warp primitives so that one lane’s computation affects all lanes, while each lane also accesses its neighbor’s data?

Configuration

Vector size: SIZE = WARP_SIZE (32 or 64 depending on GPU)
Grid configuration: (1, 1) blocks per grid
Block configuration: (WARP_SIZE, 1) threads per block

Code to complete

def broadcast_shuffle_coordination[
    size: Int
](
    output: TileTensor[mut=True, dtype, LayoutType, MutAnyOrigin],
    input: TileTensor[mut=False, dtype, LayoutType, MutAnyOrigin],
):
    """
    Combine broadcast() and shuffle_down() for advanced warp coordination.
    Lane 0 computes block-local scaling factor, broadcasts it to all lanes in the warp.
    Each lane uses shuffle_down() for neighbor access and applies broadcast factor.
    """
    var global_i = block_dim.x * block_idx.x + thread_idx.x
    var lane = Int(lane_id())

    if global_i < size:
        var scale_factor: output.ElementType = 0.0

        # FILL IN (roughly 14 lines)

Tips

1. Multi-primitive coordination

This puzzle requires orchestrating both broadcast and shuffle operations in sequence.

Think about the flow:

One lane computes a value for the entire warp
This value is broadcast to all lanes
Each lane uses shuffle to access neighbor data
The broadcast value influences how neighbor data is processed

Coordination pattern:

# Phase 1: Broadcast coordination
var shared_param = compute_if_lane_0()
shared_param = broadcast(shared_param)

# Phase 2: Shuffle neighbor access
current_val = input[global_i]
neighbor_val = shuffle_down(current_val, offset)

# Phase 3: Combined computation
result = combine(current_val, neighbor_val, shared_param)

2. Parameter computation strategy

Consider what kind of block-level parameter would be useful for scaling neighbor operations.

Questions to explore:

What statistic should lane 0 compute from the block data?
How should this parameter influence the neighbor-based computation?
What happens at warp boundaries when shuffle operations are involved?

3. Combined operation design

Think about how to meaningfully combine broadcast parameters with shuffle-based neighbor access.

Pattern considerations:

Should the broadcast parameter scale the inputs, outputs, or computation?
How do you handle boundary cases where shuffle returns undefined data?
What’s the most efficient order of operations?

Test the broadcast-shuffle coordination:

pixi run p25 --broadcast-shuffle-coordination

pixi run -e amd p25 --broadcast-shuffle-coordination

pixi run -e apple p25 --broadcast-shuffle-coordination

uv run poe p25 --broadcast-shuffle-coordination

Expected output when solved:

WARP_SIZE:  32
SIZE:  32
output: HostBuffer([30.0, 50.0, 70.0, 45.0, 20.0, 40.0, 60.0, 40.0, 20.0, 40.0, 60.0, 40.0, 20.0, 40.0, 60.0, 40.0, 20.0, 40.0, 60.0, 40.0, 20.0, 40.0, 60.0, 40.0, 20.0, 40.0, 60.0, 40.0, 20.0, 40.0, 60.0, 35.0])
expected: HostBuffer([30.0, 50.0, 70.0, 45.0, 20.0, 40.0, 60.0, 40.0, 20.0, 40.0, 60.0, 40.0, 20.0, 40.0, 60.0, 40.0, 20.0, 40.0, 60.0, 40.0, 20.0, 40.0, 60.0, 40.0, 20.0, 40.0, 60.0, 40.0, 20.0, 40.0, 60.0, 35.0])
Broadcast + shuffle coordination test: passed
Puzzle 25 complete ✅

Solution

def broadcast_shuffle_coordination[
    size: Int
](
    output: TileTensor[mut=True, dtype, LayoutType, MutAnyOrigin],
    input: TileTensor[mut=False, dtype, LayoutType, MutAnyOrigin],
):
    """
    Combine broadcast() and shuffle_down() for advanced warp coordination.
    Lane 0 computes block-local scaling factor, broadcasts it to all lanes in the warp.
    Each lane uses shuffle_down() for neighbor access and applies broadcast factor.
    """
    var global_i = block_dim.x * block_idx.x + thread_idx.x
    var lane = Int(lane_id())

    if global_i < size:
        # Step 1: Lane 0 computes block-local scaling factor
        var scale_factor: output.ElementType = 0.0
        if lane == 0:
            # Compute average of first 4 elements in this block's data
            var block_start = block_idx.x * block_dim.x
            var sum: output.ElementType = 0.0
            for i in range(4):
                if block_start + i < size:
                    sum += input[block_start + i]
            scale_factor = sum / 4.0

        # Step 2: Broadcast scaling factor to all lanes in this warp
        scale_factor = broadcast(scale_factor)

        # Step 3: Each lane gets current and next values
        var current_val = input[global_i]
        var next_val = shuffle_down(current_val, 1)

        # Step 4: Apply broadcast factor with neighbor coordination
        if lane < WARP_SIZE - 1 and global_i < size - 1:
            # Combine current + next, then scale by broadcast factor
            output[global_i] = (current_val + next_val) * scale_factor
        else:
            # Last lane in warp or last element: only current value, scaled by broadcast factor
            output[global_i] = current_val * scale_factor

This solution demonstrates the most advanced warp coordination pattern, combining broadcast and shuffle primitives.

Complete algorithm analysis:

if global_i < size:
    # Step 1: Lane 0 computes block-local scaling factor
    var scale_factor: output.ElementType = 0.0
    if lane == 0:
        var block_start = block_idx.x * block_dim.x
        var sum: output.ElementType = 0.0
        for i in range(4):
            if block_start + i < size:
                sum += input[block_start + i]
        scale_factor = sum / 4.0

    # Step 2: Broadcast scaling factor to all lanes
    scale_factor = broadcast(scale_factor)

    # Step 3: Each lane gets current and next values via shuffle
    var current_val = input[global_i]
    var next_val = shuffle_down(current_val, 1)

    # Step 4: Apply broadcast factor with neighbor coordination
    if lane < WARP_SIZE - 1 and global_i < size - 1:
        output[global_i] = (current_val + next_val) * scale_factor
    else:
        output[global_i] = current_val * scale_factor

Multi-primitive execution trace:

Input data: [2, 4, 6, 8, 1, 3, 5, 7, ...]

Phase 1: Lane 0 computes scaling factor
  Lane 0 computes: (input[0] + input[1] + input[2] + input[3]) / 4
                 = (2 + 4 + 6 + 8) / 4 = 20 / 4 = 5.0
  Other lanes: scale_factor remains 0.0

Phase 2: Broadcast scale_factor = 5.0 to all lanes
  All lanes now have: scale_factor = 5.0

Phase 3: Shuffle operations for neighbor access
  Lane 0: current_val = input[0] = 2, next_val = shuffle_down(2, 1) = input[1] = 4
  Lane 1: current_val = input[1] = 4, next_val = shuffle_down(4, 1) = input[2] = 6
  Lane 2: current_val = input[2] = 6, next_val = shuffle_down(6, 1) = input[3] = 8
  Lane 3: current_val = input[3] = 8, next_val = shuffle_down(8, 1) = input[4] = 1
  ...
  Lane 31: current_val = input[31], next_val = undefined

Phase 4: Combined computation with broadcast scaling
  Lane 0: output[0] = (2 + 4) * 5.0 = 6 * 5.0 = 30.0
  Lane 1: output[1] = (4 + 6) * 5.0 = 10 * 5.0 = 50.0
  Lane 2: output[2] = (6 + 8) * 5.0 = 14 * 5.0 = 70.0
  Lane 3: output[3] = (8 + 1) * 5.0 = 9 * 5.0 = 45.0
  Lane 4: output[4] = (1 + 3) * 5.0 = 4 * 5.0 = 20.0
  ...
  Lane 31: output[31] = input[31] * 5.0

  The last lane in the warp takes the else branch: it has no valid neighbor to
  the right, so it scales its own value instead of a pair sum.

Communication pattern analysis: This algorithm implements a hierarchical coordination pattern:

Vertical coordination (broadcast): Lane 0 → All lanes
Horizontal coordination (shuffle): Lane i → Lane i+1
Combined computation: Uses both broadcast and shuffle data

Mathematical foundation: \[\Large \text{output}[i] = \begin{cases} (\text{input}[i] + \text{input}[i+1]) \cdot \beta & \text{if lane} i < \text{WARP_SIZE} - 1 \\ \text{input}[i] \cdot \beta & \text{if lane } i = \text{WARP_SIZE} - 1 \end{cases}\]

Where \(\beta = \frac{1}{4}\sum_{k=0}^{3} \text{input}[\text{block_start} + k]\) is the broadcast scaling factor.

Advanced coordination benefits:

Multi-level communication: Combines global (broadcast) and local (shuffle) coordination
Adaptive scaling: Block-level parameters influence neighbor operations
Efficient composition: Two primitives work together seamlessly
Complex algorithms: Enables sophisticated parallel algorithms

Real-world applications:

Adaptive filtering: Block-level noise estimation with neighbor-based filtering
Dynamic load balancing: Global work distribution with local coordination
Multi-scale processing: Global parameters controlling local stencil operations

Summary

Here is what the core pattern of this section looks like

var shared_value = initial_value
if lane == 0:
    shared_value = compute_block_statistic()
shared_value = broadcast(shared_value)
var result = use_shared_value(shared_value, local_data)

Key benefits:

One-to-many coordination: Single lane computes, all lanes benefit
No barriers to place: The shuffle instruction carries the synchronization
Composable patterns: Easily combines with shuffle and other warp operations

Applications: Block statistics, collective decisions, parameter sharing, adaptive algorithms.