George Stykalin fdb3e590a6 Redirect NCCL stderr to stdout, add verbose per-batch logging ...

- Redirect stderr to stdout via dup2 so Airflow captures NCCL TRACE
  output for all ranks (not just rank 0)
- Log per-batch loss for all ranks
- Log per-epoch summary with local/global avg loss and timing
- Log total training time

Co-Authored-By: Claude <noreply@anthropic.com>

2026-04-16 16:24:59 +03:00

..

connection-check-default.py

update graphs

2026-01-26 16:13:01 +03:00

connection-check.py

update dags

2026-01-26 15:49:45 +03:00

test-train-pytorch.py

Redirect NCCL stderr to stdout, add verbose per-batch logging

2026-04-16 16:24:59 +03:00

test-train-tensorflow.py

init commit

2026-01-12 16:25:19 +03:00

train.py

update dags

2026-04-15 12:10:19 +03:00