xrsrke/pipegoose issues and pull requests

#27 - Support parallelizing arbitrary transformer torch modules

Issue - State: closed - Opened by xrsrke over 1 year ago

#27 - Support parallelizing arbitrary transformer torch modules

Issue - State: closed - Opened by xrsrke over 1 year ago

#27 - Support parallelizing arbitrary transformer torch modules

Issue - State: closed - Opened by xrsrke over 1 year ago

#26 - Support TPU

Issue - State: closed - Opened by xrsrke over 1 year ago

#25 - Lazy initialization of massive models

Issue - State: open - Opened by xrsrke over 1 year ago - 1 comment
Labels: help wanted

#25 - Lazy initialization of massive models

Issue - State: open - Opened by xrsrke over 1 year ago - 1 comment
Labels: help wanted

#25 - Lazy initialization of massive models

Issue - State: open - Opened by xrsrke over 1 year ago - 1 comment
Labels: help wanted

#24 - Checkpointing

Issue - State: closed - Opened by xrsrke over 1 year ago - 4 comments
Labels: good first issue

#24 - Checkpointing

Issue - State: closed - Opened by xrsrke over 1 year ago - 4 comments
Labels: good first issue

#24 - Checkpointing

Issue - State: closed - Opened by xrsrke over 1 year ago - 4 comments
Labels: good first issue

#23 - WIP: Trainer

Pull Request - State: open - Opened by isamu-isozaki over 1 year ago - 7 comments

#23 - WIP: Trainer

Pull Request - State: open - Opened by isamu-isozaki over 1 year ago - 7 comments

#22 - Sequence Parallelism

Issue - State: open - Opened by xrsrke over 1 year ago - 1 comment

#22 - Sequence Parallelism

Issue - State: open - Opened by xrsrke over 1 year ago - 1 comment

#22 - Sequence Parallelism

Issue - State: open - Opened by xrsrke over 1 year ago - 1 comment

#21 - Callbacks for Distributed Optimizer

Issue - State: closed - Opened by xrsrke over 1 year ago

#21 - Callbacks for Distributed Optimizer

Issue - State: closed - Opened by xrsrke over 1 year ago

#21 - Callbacks for Distributed Optimizer

Issue - State: closed - Opened by xrsrke over 1 year ago

#20 - ZeRO-1

Issue - State: closed - Opened by xrsrke over 1 year ago

#20 - ZeRO-1

Issue - State: closed - Opened by xrsrke over 1 year ago

#20 - ZeRO-1

Issue - State: closed - Opened by xrsrke over 1 year ago

#19 - Mixture of Experts

Issue - State: open - Opened by xrsrke over 1 year ago

#19 - Mixture of Experts

Issue - State: open - Opened by xrsrke over 1 year ago

#19 - Mixture of Experts

Issue - State: open - Opened by xrsrke over 1 year ago

#18 - Trainer

Issue - State: open - Opened by xrsrke over 1 year ago - 4 comments
Labels: good first issue

#18 - Trainer

Issue - State: open - Opened by xrsrke over 1 year ago - 4 comments
Labels: good first issue

#18 - Trainer

Issue - State: open - Opened by xrsrke over 1 year ago - 4 comments
Labels: good first issue

#17 - Implement new tensor parallelism technique

Issue - State: open - Opened by xrsrke over 1 year ago
Labels: help wanted

#17 - Implement new tensor parallelism technique

Issue - State: open - Opened by xrsrke over 1 year ago
Labels: help wanted

#17 - Implement new tensor parallelism technique

Issue - State: open - Opened by xrsrke over 1 year ago
Labels: help wanted

#16 - Setup documentation

Issue - State: open - Opened by xrsrke over 1 year ago
Labels: help wanted, good first issue

#16 - Setup documentation

Issue - State: open - Opened by xrsrke over 1 year ago
Labels: help wanted, good first issue

#16 - Setup documentation

Issue - State: open - Opened by xrsrke over 1 year ago
Labels: help wanted, good first issue

#15 - Reproducible in 3D Parallelism

Issue - State: closed - Opened by xrsrke over 1 year ago
Labels: help wanted, good first issue

#15 - Reproducible in 3D Parallelism

Issue - State: closed - Opened by xrsrke over 1 year ago
Labels: help wanted, good first issue

#15 - Reproducible in 3D Parallelism

Issue - State: closed - Opened by xrsrke over 1 year ago
Labels: help wanted, good first issue

#14 - Mixed precision training in FP16

Issue - State: open - Opened by xrsrke over 1 year ago
Labels: help wanted

#13 - Fused Optimizer

Issue - State: open - Opened by xrsrke over 1 year ago - 8 comments

#13 - Fused Optimizer

Issue - State: open - Opened by xrsrke over 1 year ago - 8 comments

#11 - Deparallelize tensor parallelism

Issue - State: closed - Opened by xrsrke over 1 year ago - 4 comments
Labels: help wanted

#10 - Kernel Fusion using torch.jit

Issue - State: open - Opened by xrsrke over 1 year ago - 3 comments
Labels: good first issue

#10 - Kernel Fusion using torch.jit

Issue - State: open - Opened by xrsrke over 1 year ago - 3 comments
Labels: good first issue

#9 - Dataloader and Sampler for 3D Parallelism

Issue - State: closed - Opened by xrsrke over 1 year ago
Labels: good first issue

#9 - Dataloader and Sampler for 3D Parallelism

Issue - State: closed - Opened by xrsrke over 1 year ago
Labels: good first issue

#8 - Port CUDA Kernels

Issue - State: open - Opened by xrsrke over 1 year ago - 6 comments
Labels: help wanted, good first issue

#8 - Port CUDA Kernels

Issue - State: open - Opened by xrsrke over 1 year ago - 6 comments
Labels: help wanted, good first issue

#7 - Implement new pipeline parallelism technique

Issue - State: open - Opened by xrsrke over 1 year ago
Labels: High Priority

#6 - Model partitioning for pipeline parallelism

Issue - State: open - Opened by xrsrke over 1 year ago - 1 comment

#5 - Bucket small tensors and collective operations into larger ones

Issue - State: open - Opened by xrsrke over 1 year ago

#4 - Gradient Checkpointing

Issue - State: open - Opened by xrsrke over 1 year ago - 1 comment

#3 - add test

Pull Request - State: closed - Opened by xrsrke over 1 year ago

#2 - Fsdp

Pull Request - State: closed - Opened by xrsrke over 1 year ago

#1 - WIP

Pull Request - State: closed - Opened by xrsrke over 1 year ago

GitHub / xrsrke/pipegoose issues and pull requests