Skip to content

Adding nica2a preset #248

Open
pierreantoineH wants to merge 3 commits intoROCm:candidatefrom
pierreantoineH:candidate_a2anicpreset
Open

Adding nica2a preset #248
pierreantoineH wants to merge 3 commits intoROCm:candidatefrom
pierreantoineH:candidate_a2anicpreset

Conversation

@pierreantoineH
Copy link
Copy Markdown

Description

  • Adding nica2a preset (NIC all-to-all over GPUs via NIC executors, multi-node): stride/device grouping and NIC planes.

    • STRIDE — Step size for stride orbits on rank-major devices (gcd with total device count); no traffic between different orbits.
    • GROUP_SIZE — Devices per subgroup inside each stride orbit (natural rank-major order); must divide orbit size (default: all devices per rank × GPUs).
    • NIC_A2A_SCOPEintra: transfers only within the same device subgroup; inter: only between different subgroups (same stride orbit only).
    • NIC_A2A_NO_SAME_RANK — When non-zero, omit transfers where source and destination are the same rank.
    • NUM_NIC_PLANES — Split NIC endpoints into this many disjoint planes (rank-major index modulo planes); traffic only between NICs in the same plane.

Motivation

Need to stress scale-out network independently from intra node communication with Alltoall pattern

Technical Details

The pattern is defined to use all NICs, can be used in the case of several NICs per GPU .
The stride startegy differ from the one defined in PodAlltoall . Here natural order inside each stride orbit is used. Syntax close to the one of PodAlltoall.
The scope allows to considers running inside the same groups but the need was also to run only between groups.
Also an option in terms of number of planes have been added (can also be used in rail topology) to be able to restrict a2a pattern to have only peers connected through scale-out network.

Test Plan

Tested up to 6 nodes varying different variables.

Test Result

Expected patterns used in each case

…ulti-node): stride/device grouping and NIC planes.

  - `STRIDE` — Step size for stride orbits on rank-major devices (`gcd` with total device count); no traffic between different orbits.
  - `GROUP_SIZE` — Devices per subgroup inside each stride orbit (natural rank-major order); must divide orbit size (default: all devices per rank × GPUs).
  - `NIC_A2A_SCOPE` — `intra`: transfers only within the same device subgroup; `inter`: only between different subgroups (same stride orbit only).
  - `NIC_A2A_NO_SAME_RANK` — When non-zero, omit transfers where source and destination are the same rank.
  - `NUM_NIC_PLANES` — Split NIC endpoints into this many disjoint planes (rank-major index modulo planes); traffic only between NICs in the same plane.
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new nica2a preset to TransferBench to generate NIC-executor-based all-to-all traffic across GPUs with configurable device stride/grouping, intra/inter-group scope, same-rank filtering, and NIC plane partitioning—intended to stress scale-out networking independently of intra-node GPU communication.

Changes:

  • Register new nica2a preset in the preset map.
  • Implement NicAllToAllPreset with STRIDE/GROUP_SIZE orbit grouping, NIC plane filtering, and scope controls.
  • Document the new preset and its environment variables in the changelog.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File Description
src/client/Presets/Presets.hpp Adds nica2a preset entry and includes the new preset header.
src/client/Presets/NicAllToAll.hpp Implements the NIC all-to-all preset logic, transfer generation, and reporting table.
CHANGELOG.md Documents the new preset and its configuration knobs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@AtlantaPepsi
Copy link
Copy Markdown
Contributor

I'll test the new changes later today and tomorrow

{
{"a2a", {AllToAllPreset, "Tests parallel transfers between all pairs of GPU devices"}},
{"a2a_n", {AllToAllRdmaPreset, "Tests parallel transfers between all pairs of GPU devices using Nearest NIC RDMA transfers"}},
{"nica2a", {NicAllToAllPreset, "All-to-all GPU traffic over NIC transfers using each NIC's closest GPU endpoint"}},
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: spacing

Suggested change
{"nica2a", {NicAllToAllPreset, "All-to-all GPU traffic over NIC transfers using each NIC's closest GPU endpoint"}},
{"nica2a", {NicAllToAllPreset, "All-to-all GPU traffic over NIC transfers using each NIC's closest GPU endpoint"}},

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants