Root cause: Subnet manager assigns different PKEYs to ports - test-1 (LID 0x44): Full membership PKEYs (0xa001, 0x9001) - test-2 (LID 0x7d): Limited membership PKEY 0x7fff only This causes MTU mismatch (4092 vs 2044) and NCCL failure. Cluster admin needs to configure opensm partitions to fix. Ethernet is working and stable. Co-Authored-By: Claude <noreply@anthropic.com>