2 Commits

Author SHA1 Message Date
52b8680d95 Enable InfiniBand for TensorFlow distributed training
- Set NCCL IB env vars before TF import (IB_DISABLE=0, GPU Direct RDMA)
- Use NCCL_DEBUG_FILE approach for transport logging (same as PyTorch DAG)
- Add IB pre-flight diagnostics (dev/infiniband, sys/class/infiniband, ibv_devinfo)
- Add per-epoch logging with timing for all workers
- Add NCCL transport summary after training (NET/IB, GPU Direct, Channel info)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-04-16 16:47:23 +03:00
526f201f41 init commit 2026-01-12 16:25:19 +03:00