GitHub / NVIDIA/TransformerEngine issues and pull requests
#7119 - Changed the way pyright translates `tuple[()]` into a specialized `Se…
Pull Request -
State: closed - Opened by erictraut almost 2 years ago
- 1 comment
#3550 - Preserve NaN through half-precision MXFP8 amax reductions
Issue -
State: open - Opened by sylvesterkaczmarek 2 days ago
#3549 - Fix FusedSGD zero_grad argument forwarding
Pull Request -
State: open - Opened by wujingyue 2 days ago
- 1 comment
Labels: community-contribution
#3548 - KDA attention
Pull Request -
State: open - Opened by ksivaman 2 days ago
- 1 comment
Labels: 2.20
#3547 - [PyTorch] Compile Linear and Bias operations with forward fusion
Pull Request -
State: open - Opened by pggPL 2 days ago
#3546 - [PyTorch] Enable LayerNormLinear UB AG overlap under no_grad
Pull Request -
State: open - Opened by ravimajeti 3 days ago
- 1 comment
Labels: community-contribution
#3545 - [PyTorch] Keep DistributedWeight objects on ctx across saved-tensor hooks
Pull Request -
State: open - Opened by xrennvidia 3 days ago
- 1 comment
Labels: community-contribution
#3544 - feat: vmm slot for local cuda graph offload
Pull Request -
State: open - Opened by Xdydy 3 days ago
- 1 comment
Labels: community-contribution
#3543 - vmm support for cuda graph offload
Pull Request -
State: closed - Opened by Xdydy 3 days ago
- 1 comment
Labels: community-contribution
#3542 - Use PyTorch CUTLASS for BF16 grouped GEMM on SM100
Pull Request -
State: open - Opened by wujingyue 3 days ago
- 2 comments
Labels: community-contribution
#3541 - Fix distributed Newton-Schulz correctness
Pull Request -
State: closed - Opened by vcherepanov-nv 3 days ago
- 1 comment
Labels: 2.20
#3540 - [PyTorch][CP] Support compact metadata with FA4
Pull Request -
State: open - Opened by sudhakarsingh27 3 days ago
- 2 comments
#3539 - GDP attention support
Pull Request -
State: closed - Opened by ksivaman 3 days ago
- 3 comments
Labels: 2.20
#3538 - fix: add missing NVTE_BHSD labels to fused_attn to_string
Pull Request -
State: open - Opened by andrewwhitecdw 3 days ago
- 1 comment
Labels: community-contribution
#3537 - Prototype of green context + VMM localization
Pull Request -
State: open - Opened by WanZzzzzz 3 days ago
Labels: community-contribution
#3536 - test(attention): anchor the context-parallel suite to an independent reference
Pull Request -
State: open - Opened by nvegesna-netizen 4 days ago
- 2 comments
Labels: community-contribution
#3535 - [Docs] Refresh README
Pull Request -
State: open - Opened by sbhavani 4 days ago
- 1 comment
#3534 - [PyTorch] [torch.compile] torch.compile support for LayerNormLinear and LayerNormMLP
Pull Request -
State: open - Opened by pggPL 4 days ago
- 1 comment
#3533 - [PyTorch] Retain NVFP4 RNG tensors through quantization
Pull Request -
State: open - Opened by Connor-XY 4 days ago
- 1 comment
Labels: community-contribution
#3532 - fix(attention): stop handing FlashAttention 4 the -1 window sentinel
Pull Request -
State: open - Opened by nvegesna-netizen 4 days ago
- 1 comment
Labels: community-contribution
#3531 - [PyTorch] Remove stale xfail for selective attention recompute
Pull Request -
State: closed - Opened by negvet 4 days ago
- 2 comments
Labels: 2.20
#3530 - [PyTorch] Extend no-load-balance CP to the a2a comm type
Pull Request -
State: open - Opened by Rudin6 5 days ago
- 1 comment
Labels: community-contribution
#3529 - [PyTorch] Allow F16 GDN inside FP8 autocast
Pull Request -
State: closed - Opened by layalir 5 days ago
- 2 comments
Labels: community-contribution
#3528 - FA4 returns an all-zero output for causal attention on SM100 after flash-attention #2490
Issue -
State: open - Opened by nvegesna-netizen 5 days ago
- 3 comments
#3527 - feat(attention): cuDNN FROST attention backend for head_dim in (256, 512], with context parallelism
Pull Request -
State: open - Opened by nvegesna-netizen 5 days ago
- 3 comments
Labels: community-contribution, 2.21.0
#3524 - [JAX] Remove duplicate score-mod test run
Pull Request -
State: closed - Opened by KshitijLakhani 5 days ago
- 4 comments
Labels: 2.20
#3523 - Remove the view() calls from Linear/LNLinear/LNMLP modules
Pull Request -
State: open - Opened by ptrendx 5 days ago
- 4 comments
#3522 - [JAX] Support single-process-multi-device EP via XLA-borrowed communicator
Pull Request -
State: open - Opened by phu0ngng 5 days ago
- 3 comments
#3521 - [PyTorch] GDN2 support and linear attention refactor
Pull Request -
State: closed - Opened by ksivaman 5 days ago
- 4 comments
Labels: 2.20
#3517 - [PyTorch] Support distributed weights in GroupedLinear's grouped-tensor path
Pull Request -
State: open - Opened by fanshiqing 6 days ago
- 2 comments
Labels: org-contribution
#3516 - [JAX] Align fused attn backward output gradient sharding and other test strengthening
Pull Request -
State: open - Opened by KshitijLakhani 6 days ago
- 2 comments
Labels: bug, attention, 2.20
#3513 - Abstract CUDA hardcodes into configurable te_device_type / te_platform
Pull Request -
State: open - Opened by lxd-cumt 7 days ago
- 1 comment
Labels: community-contribution
#3512 - Abstract CUDA hardcodes into configurable te_device_type / te_platform
Pull Request -
State: closed - Opened by lxd-cumt 7 days ago
- 1 comment
Labels: community-contribution
#3511 - Insert assert header
Pull Request -
State: closed - Opened by aybchan 9 days ago
- 2 comments
Labels: org-contribution
#3510 - [PyTorch] Add head-parallel FA4 backward for cuDNN CP attention
Pull Request -
State: open - Opened by bzantium 10 days ago
- 2 comments
Labels: community-contribution, attention
#3508 - [Draft] Port cuDNN frontend attention to Python API
Pull Request -
State: closed - Opened by vcherepanov-nv 10 days ago
- 2 comments
#3506 - Fix THD P2P pad detection and tail-zero gating
Pull Request -
State: open - Opened by RPalmr 10 days ago
- 3 comments
Labels: community-contribution
#3503 - MOE Sequential Block with Dispatch and Combine as Basic Ops
Pull Request -
State: open - Opened by vthumbe1503 11 days ago
- 6 comments
Labels: 2.20
#3502 - [Pytorch] Replace TorchScript fallbacks with eager execution
Pull Request -
State: closed - Opened by sudhakarsingh27 11 days ago
- 4 comments
Labels: 3.0
#3499 - Fix hybrid extra state size mismatch
Pull Request -
State: open - Opened by negvet 12 days ago
- 2 comments
#3498 - [JAX] Fix undefined sr_rng_state that breaks --dry-run in two encoder examples
Pull Request -
State: open - Opened by Anai-Guo 12 days ago
- 1 comment
Labels: community-contribution
#3497 - Replace RuntimeError with warnings in ScaledSReLU
Pull Request -
State: closed - Opened by ghadiaravi13 12 days ago
- 1 comment
Labels: community-contribution
#3496 - [PyTorch] torch.compile for an OperationFuser group holding one operation
Pull Request -
State: closed - Opened by pggPL 13 days ago
- 3 comments
#3495 - [PyTorch] Use torch's register_custom_class API for opaque quantizers when available
Pull Request -
State: closed - Opened by mmarcinkiewicz 13 days ago
- 2 comments
Labels: community-contribution
#3494 - [Docs] Add Mixture of Experts guide
Pull Request -
State: open - Opened by pggPL 13 days ago
- 1 comment
Labels: documentation
#3493 - [PyTorch] [torch.compile] Prepare LayerNormLinear and LayerNormMLP for torch.compile
Pull Request -
State: closed - Opened by pggPL 13 days ago
- 5 comments
#3492 - Fix hybrid fp8 test guard
Pull Request -
State: closed - Opened by negvet 14 days ago
- 2 comments
#3491 - Update Linear docstring referring torch.Linear
Pull Request -
State: closed - Opened by negvet 14 days ago
- 1 comment
Labels: documentation
#3490 - [PyT] Disable FA3 for training when head_dim_qk != head_dim_v
Pull Request -
State: open - Opened by yuweih205 14 days ago
- 1 comment
Labels: community-contribution
#3489 - Add native transport for dynamic context parallelism
Pull Request -
State: closed - Opened by xiaoyao0115 15 days ago
- 1 comment
Labels: community-contribution
#3488 - Support row-only MXFP8 distributed master-weight casts
Pull Request -
State: open - Opened by xiuhu17 15 days ago
- 1 comment
Labels: community-contribution
#3487 - [PyTorch][Attention] THD P2P context-parallel regression when padded cu_seqlens are value-equal but not object-identical
Issue -
State: open - Opened by cuichenx 16 days ago
Labels: attention
#3486 - [JAX] Interpolate the tensor-sequence-parallelism warning
Pull Request -
State: open - Opened by Anai-Guo 16 days ago
- 3 comments
Labels: community-contribution
#3485 - [PyTorch] Fix NaN expert weight gradients at num_groups == 1 with SReLU
Pull Request -
State: closed - Opened by GarlGuo 16 days ago
- 3 comments
Labels: community-contribution
#3484 - [Bug] NaN expert weight gradients at num_groups == 1 with a padded token buffer for SReLU
Issue -
State: closed - Opened by GarlGuo 16 days ago
- 1 comment
Labels: bug
#3483 - [Common] Fix Grouped MXFP8 work mapping and TMA synchronization
Pull Request -
State: closed - Opened by Oleg-Goncharov 16 days ago
- 3 comments
#3481 - [Bug] Backend selection picks FA3 for training with head_dim_qk=192 / v_head_dim=128, but FA3 backward cannot run it
Issue -
State: open - Opened by yuweih205 17 days ago
- 4 comments
Labels: attention
#3480 - [PyTorch] Split FusedAttnFunc into single-argument forward/backward helpers
Pull Request -
State: open - Opened by pggPL 17 days ago
- 2 comments
Labels: 2.20
#3474 - [Bug] Illegal memory access in grouped MXFP8 quantize on GB200 (sm_100) introduced by #3337; TMA store completion handoff missing in group_quantize_mxfp8.cuh
Issue -
State: open - Opened by janbernloehr 17 days ago
- 6 comments
#3473 - [PyTorch] Enable fused activation recompute for ScaledTanhSReLU
Pull Request -
State: open - Opened by wanyingw 18 days ago
- 1 comment
Labels: community-contribution
#3472 - [PyTorch] torch.compile support for FusedAttention
Pull Request -
State: open - Opened by pggPL 18 days ago
- 1 comment
Labels: 2.21.0
#3471 - [PyTorch] DeepSeekV3Layer: full MoE transformer layer (MLA + DeepSeek MoE)
Pull Request -
State: open - Opened by pggPL 18 days ago
#3469 - [PyTorch] Fix FP8 illegal memory access in single-process multi-GPU execution
Pull Request -
State: open - Opened by SuperGoodGame 18 days ago
- 2 comments
Labels: community-contribution
#3468 - Automatically omit unused columnwise primary weights for backward overrides
Pull Request -
State: open - Opened by xiuhu17 18 days ago
- 3 comments
Labels: community-contribution
#3467 - [Common] row-scaled nvfp4 path: add single-launch group fused amax
Pull Request -
State: open - Opened by cael-ling 18 days ago
- 1 comment
Labels: community-contribution
#3462 - [JAX] Make MoEBlock aware of Dense TP axes
Pull Request -
State: open - Opened by jberchtold-nvidia 19 days ago
- 1 comment
#3459 - [Common] Add non-TMA MXFP8 cast-only kernels for specialized rowwise-only and row+colwise
Pull Request -
State: closed - Opened by tdophung 19 days ago
- 5 comments
Labels: 2.20
#3457 - [JAX] Add SiTU-GLU support to JAX
Pull Request -
State: open - Opened by jberchtold-nvidia 19 days ago
- 3 comments
#3456 - [PyTorch] Schedule delayed-scaling updates after backward
Pull Request -
State: open - Opened by pggPL 20 days ago
- 3 comments
#3454 - [Common] row-scaled nvfp4 path: fuse row/col amax into a single TMA-tiled kernel
Pull Request -
State: open - Opened by cael-ling 20 days ago
- 1 comment
Labels: community-contribution
#3452 - [JAX] Add EP borrowed-comm path via XLA FFI collectives extension
Pull Request -
State: open - Opened by phu0ngng 20 days ago
- 2 comments
#3443 - Cutlass Upgrade to 4.8
Pull Request -
State: closed - Opened by vthumbe1503 21 days ago
- 4 comments
Labels: 2.20
#3437 - Tied weights do not accumulate with delayed wgrad
Issue -
State: open - Opened by wujingyue 24 days ago
- 2 comments
#3436 - Remove redundant runtime checks for activation recompute
Pull Request -
State: open - Opened by ghadiaravi13 24 days ago
- 1 comment
Labels: community-contribution
#3435 - Fix linter error
Pull Request -
State: open - Opened by fheinecke 24 days ago
- 1 comment
#3434 - Use `dlopen` to load NCCL EP
Pull Request -
State: open - Opened by fheinecke 24 days ago
- 1 comment
Labels: 2.19
#3433 - test: skip cuBLASMp tests when unsupported
Pull Request -
State: closed - Opened by vcherepanov-nv 24 days ago
- 1 comment
Labels: 2.19
#3432 - Add SM90 gate to distributed softcap score-mod test
Pull Request -
State: closed - Opened by vcherepanov-nv 24 days ago
- 1 comment
Labels: 2.19
#3431 - Fix TE EP MOE tests aborting insted of skipping when < 4 GPUs are available
Pull Request -
State: closed - Opened by fheinecke 25 days ago
- 2 comments
Labels: 2.19
#3430 - [Common] Split grouped activation build
Pull Request -
State: open - Opened by harryzhou2000 25 days ago
- 1 comment
Labels: org-contribution
#3429 - [JAX] Fix GEMM partitioning to reduce over nested contracting mesh axes
Pull Request -
State: open - Opened by phu0ngng 25 days ago
- 3 comments
#3428 - [JAX] Limit max logit attr to forward
Pull Request -
State: closed - Opened by KshitijLakhani 25 days ago
- 5 comments
Labels: 2.19
#3427 - [PyTorch] Reduce CUDA graph memory retention
Pull Request -
State: open - Opened by buptzyb 26 days ago
- 2 comments
Labels: community-contribution
#3426 - [PyTorch] Fix mutable QB bounds in CUDA graphs
Pull Request -
State: open - Opened by harryzhou2000 26 days ago
- 1 comment
Labels: community-contribution, org-contribution
#3425 - [JAX] Fix classification of sync-tagged collective start instructions
Pull Request -
State: closed - Opened by KshitijLakhani 26 days ago
- 2 comments
Labels: 2.19
#3424 - [PyTorch] Fix Test's Reference Implementation of Clamped Swiglu
Pull Request -
State: closed - Opened by vthumbe1503 27 days ago
- 3 comments
#3423 - Support paged stashing for GroupedLinear activations
Pull Request -
State: open - Opened by lhb8125 27 days ago
- 3 comments
Labels: community-contribution
#3422 - [Common] Default EP comm kernels to 32 SMs; keep shuffle/preprocess on all SMs
Pull Request -
State: closed - Opened by phu0ngng 27 days ago
- 2 comments
#3421 - Fix: fused QUproj + RoPE + Quant
Pull Request -
State: open - Opened by ghadiaravi13 27 days ago
- 1 comment
Labels: community-contribution
#3420 - [PyTorch] Add native transport for dynamic context parallelism
Pull Request -
State: open - Opened by xiaoyao0115 28 days ago
Labels: community-contribution
#3418 - [PyTorch] Bound FusedAdam tensor-handle usage
Pull Request -
State: open - Opened by zupengwang 29 days ago
- 1 comment
Labels: community-contribution
#3417 - [Common] Fix TMA synchronization in quantization kernels
Pull Request -
State: open - Opened by Oleg-Goncharov 30 days ago
- 4 comments
#3416 - Fix include after ptx move
Pull Request -
State: closed - Opened by mmarcinkiewicz 30 days ago
- 1 comment
Labels: community-contribution
#3415 - [PyTorch][Test-Fix] Scale test_multi_process_ep outer timeout with launcher pass count
Pull Request -
State: closed - Opened by phu0ngng about 1 month ago
- 3 comments
Labels: 2.19
#3414 - Add the support of variable dtypes of params and grads in capturable Fused Adam.
Pull Request -
State: closed - Opened by ptrendx about 1 month ago
- 2 comments
#3413 - Fix PyTorch EP tests sharing a single timeout
Pull Request -
State: closed - Opened by fheinecke about 1 month ago
- 4 comments
#3412 - [JAX] Fix counting of synced and wrapped (async, fusion) collectives in HLO
Pull Request -
State: closed - Opened by aybchan about 1 month ago
- 3 comments
Labels: org-contribution, 2.19
#3411 - Ring Attention: free unused kv comm buffers
Pull Request -
State: open - Opened by francesco-bertolotti about 1 month ago
- 1 comment
Labels: community-contribution
#3410 - Reduce Grouped MLP Fuser CPU Overhead
Pull Request -
State: open - Opened by zhongbozhu about 1 month ago
- 5 comments
Labels: community-contribution
#3408 - [Common] Benchmarkable Unit Tests
Pull Request -
State: open - Opened by denera about 1 month ago
- 2 comments