eleutherai/oslo issues and pull requests

#215 - TP does not work on Flan-T5

Issue - State: open - Opened by shreyansh26 about 1 year ago

#214 - is there any plan to support llama model?

Issue - State: open - Opened by yuyaxiong about 1 year ago

#213 - Feature/auto model test

Pull Request - State: closed - Opened by koliaok over 1 year ago - 1 comment

#212 - Give num_concurrent as argument

Pull Request - State: closed - Opened by jinwonkim93 over 1 year ago

#211 - [Fix] Refactor ZeRO Directory Structure

Pull Request - State: closed - Opened by yhna940 over 1 year ago - 2 comments
Labels: ZeRO

#210 - Auto model test integration

Pull Request - State: closed - Opened by koliaok over 1 year ago - 1 comment

#209 - [Fix] rpc not working on aws

Pull Request - State: closed - Opened by jinwonkim93 over 1 year ago - 2 comments

#208 - [Add] copyright for ZeRO3

Pull Request - State: closed - Opened by yhna940 over 1 year ago
Labels: ZeRO

#207 - [Feature] Enhance Distributed Data Parallel wrapper with ZeRO

Pull Request - State: closed - Opened by yhna940 over 1 year ago - 1 comment
Labels: ZeRO

#206 - [Fix] Change the logger of ZeRO3

Pull Request - State: closed - Opened by yhna940 over 1 year ago
Labels: ZeRO

#205 - [Add] ZeRO3 unit test

Pull Request - State: closed - Opened by yhna940 over 1 year ago
Labels: ZeRO

#204 - Ohwi/pp use message queue

Pull Request - State: closed - Opened by ohwi over 1 year ago - 2 comments

#203 - To apply FlashAttention

Issue - State: open - Opened by dyanos over 1 year ago - 1 comment

#202 - Add requirement for lightseq2

Pull Request - State: closed - Opened by jinwonkim93 over 1 year ago

#201 - [Feature] new interface for zero

Pull Request - State: closed - Opened by jinwonkim93 over 1 year ago

#200 - BART PreTraining code

Pull Request - State: closed - Opened by hmy831004 over 1 year ago

#199 - Add DP on Trainer

Pull Request - State: closed - Opened by tree-park over 1 year ago

#198 - Apply Multi-head Attention Layer of LightSeq2 on BERT

Pull Request - State: closed - Opened by dyanos over 1 year ago - 6 comments

#197 - Applied Multi-head Attention Layer of LightSeq2 on BERT

Pull Request - State: closed - Opened by dyanos over 1 year ago - 1 comment

#196 - [Feature] Add GradScaler for ZeroOptim

Pull Request - State: open - Opened by nijkah over 1 year ago - 3 comments

#195 - [Del] grad handle free storage func

Pull Request - State: closed - Opened by yhna940 over 1 year ago
Labels: ZeRO

#194 - Remove typos related to distributed tensor

Pull Request - State: closed - Opened by KKIEEK over 1 year ago

#193 - Revert "Resolve merge conflicts for ZeRO3"

Pull Request - State: closed - Opened by KKIEEK over 1 year ago

#192 - Resolve merge conflicts for ZeRO3

Pull Request - State: closed - Opened by KKIEEK over 1 year ago - 1 comment

#191 - Task Variable Naming change

Pull Request - State: closed - Opened by hmy831004 over 1 year ago

#190 - [Add] Distributed Logger Utility for PyTorch

Pull Request - State: closed - Opened by yhna940 over 1 year ago

#189 - [Del] Remove DistributedTensor

Pull Request - State: closed - Opened by yhna940 over 1 year ago
Labels: Bug

#188 - Trainer updates additional functions

Pull Request - State: closed - Opened by tree-park over 1 year ago - 1 comment

#187 - [Fix] Minor Issues in Data Parallel and Chunk Utils

Pull Request - State: closed - Opened by yhna940 over 1 year ago
Labels: ZeRO

#186 - TODO : Zero optimizer deparallel

Issue - State: open - Opened by hyeinhyun over 1 year ago
Labels: Data Parallelism

#185 - [Enhance] Support ZeRO3

Pull Request - State: closed - Opened by yhna940 over 1 year ago - 2 comments
Labels: ZeRO

#184 - [Refact] Mv dist tensor level

Pull Request - State: closed - Opened by yhna940 over 1 year ago

#183 - [zero] Suggests a minor change to confusing variable names in the ZeRO optimizer

Pull Request - State: closed - Opened by yhna940 over 1 year ago
Labels: ZeRO

#182 - Change save_pretrained and from_parallelized annotation

Pull Request - State: closed - Opened by koliaok over 1 year ago - 2 comments
Labels: Bug

#181 - [Enhance] add zero3 opt

Pull Request - State: closed - Opened by yhna940 over 1 year ago - 1 comment
Labels: ZeRO

#180 - fixed save_pretrained

Pull Request - State: closed - Opened by jason9693 over 1 year ago - 5 comments

#179 - Revert ViT TP2D logic

Pull Request - State: closed - Opened by KKIEEK over 1 year ago

#178 - [Enhance] Improve Zero3 Implementation: Search Utility, Consolidation, and In-Place Dist Tensor Conversion

Pull Request - State: closed - Opened by yhna940 over 1 year ago - 1 comment
Labels: Design, ZeRO

#177 - [Fix] minor bug for single output in _DistributedDataParallel

Pull Request - State: closed - Opened by yhna940 over 1 year ago - 2 comments
Labels: Bug

#176 - [MOD] BART TASK Update

Pull Request - State: closed - Opened by hmy831004 over 1 year ago - 5 comments

#175 - TODO : Documentation for Expert Parallel

Issue - State: open - Opened by scsc0511 over 1 year ago
Labels: Expert Parallelism

#174 - Remove torch version 2.0.0 dependency in

Pull Request - State: closed - Opened by koliaok over 1 year ago - 1 comment
Labels: Bug, Data Parallelism

#173 - [Fix] Support gradient accumulation for DDP

Pull Request - State: closed - Opened by KKIEEK over 1 year ago - 1 comment

#172 - Make decoder-only models to be able to generate with `inputs_embeds`

Pull Request - State: closed - Opened by ingyuseong over 1 year ago - 1 comment

#171 - Add restarting model from saved model and fix bug

Pull Request - State: closed - Opened by tree-park over 1 year ago

#170 - adding tutorial docs for DataParallel, ZeroOptimizer

Pull Request - State: closed - Opened by jinwonkim93 over 1 year ago - 3 comments

#169 - Wrong import in zero

Pull Request - State: closed - Opened by jinwonkim93 over 1 year ago - 3 comments

#168 - Prototyping FSDP

Pull Request - State: closed - Opened by KKIEEK over 1 year ago - 2 comments

#167 - [Fix] zero optimizer w/ tensor parallel test

Pull Request - State: closed - Opened by yhna940 over 1 year ago - 4 comments
Labels: ZeRO

#166 - import ParallelMode

Pull Request - State: closed - Opened by bzantium over 1 year ago

#165 - TODO: Test Code Enhancement

Issue - State: open - Opened by bzantium over 1 year ago

#164 - [Fix] zero param check

Pull Request - State: closed - Opened by yhna940 over 1 year ago - 2 comments
Labels: Bug, ZeRO

#163 - [Add] Heterogeneous Memory Manager for ZeRO3

Pull Request - State: closed - Opened by yhna940 over 1 year ago
Labels: ZeRO

#162 - Add Apache License 2.0 copyright notice to ZeRO

Pull Request - State: closed - Opened by yhna940 over 1 year ago
Labels: Documentation, ZeRO

#161 - [Fix] Handle case when ModelOutput doesn't contain loss

Pull Request - State: closed - Opened by KKIEEK over 1 year ago

#160 - [Fix] specify import of zero optimizer

Pull Request - State: closed - Opened by yhna940 over 1 year ago
Labels: ZeRO

#159 - Fix Tp param

Pull Request - State: closed - Opened by yhna940 over 1 year ago
Labels: Bug, ZeRO

#158 - [Doc] add data parallel and zero

Pull Request - State: closed - Opened by jinwonkim93 over 1 year ago

#157 - [Fix] Handle case with mapping or sequence output

Pull Request - State: closed - Opened by KKIEEK over 1 year ago

#156 - Fix typo from 2p5D to 2P5D in docs

Pull Request - State: closed - Opened by KKIEEK over 1 year ago - 2 comments

#155 - [Enhance] Support ViT for TensorParallel

Pull Request - State: closed - Opened by KKIEEK over 1 year ago - 8 comments

#154 - TODO: Write dp documentation and tutorials

Issue - State: closed - Opened by jinwonkim93 over 1 year ago

#153 - [Fix] add set_params_to_ignore in init file

Pull Request - State: closed - Opened by jinwonkim93 over 1 year ago - 1 comment

#152 - Fix TP embedding layers

Issue - State: open - Opened by jason9693 over 1 year ago

#151 - Hotfix/tp docs : image modified

Pull Request - State: closed - Opened by jason9693 over 1 year ago

#150 - Hotfix/tp docs - latex and tutorial modified

Pull Request - State: closed - Opened by jason9693 over 1 year ago

#149 - Docs/tp docs

Pull Request - State: closed - Opened by jason9693 over 1 year ago

#148 - [Deps] Add transformers package

Pull Request - State: closed - Opened by KKIEEK over 1 year ago - 1 comment

#147 - Fix circular import for ZeRO optimizer

Pull Request - State: closed - Opened by KKIEEK over 1 year ago

#146 - [Fix] Support long tensor for DDP backward

Pull Request - State: closed - Opened by KKIEEK over 1 year ago - 1 comment

#145 - [Refactoring] trivial ZeRO structural changes

Pull Request - State: closed - Opened by yhna940 over 1 year ago
Labels: ZeRO

#144 - Fix DDP backward

Pull Request - State: closed - Opened by nijkah over 1 year ago

#143 - [Add] unit test for zero optimizer

Pull Request - State: closed - Opened by yhna940 over 1 year ago - 1 comment
Labels: ZeRO

#142 - [Clean] Clean DDP Code

Pull Request - State: closed - Opened by nijkah over 1 year ago - 1 comment

#141 - [Refactor] Refactor backward interface in DP

Pull Request - State: closed - Opened by nijkah over 1 year ago - 1 comment

#140 - Update copyright

Pull Request - State: closed - Opened by StellaAthena over 1 year ago - 1 comment

#139 - [#101] impleent save/load feature for expert_parallel

Pull Request - State: closed - Opened by scsc0511 over 1 year ago - 1 comment

#138 - Distributed data parallel

Pull Request - State: closed - Opened by jinwonkim93 over 1 year ago - 2 comments

#137 - [ADD] support Distributed Data Parallel

Pull Request - State: closed - Opened by jinwonkim93 almost 2 years ago - 9 comments
Labels: Data Parallelism

#136 - Docs/120 init docs

Pull Request - State: closed - Opened by hyunwoongko almost 2 years ago

#135 - Docs/120 init docs

Pull Request - State: closed - Opened by hyunwoongko almost 2 years ago

#134 - Docs/120 init docs

Pull Request - State: closed - Opened by hyunwoongko almost 2 years ago

#133 - [Add] support ZeRO Optimizer

Pull Request - State: closed - Opened by yhna940 almost 2 years ago - 8 comments
Labels: Data Parallelism, ZeRO

#132 - Add save and eval features on trainer

Pull Request - State: closed - Opened by tree-park almost 2 years ago - 1 comment

#131 - TODO: CUDA Kernel for Transformer Model

Issue - State: closed - Opened by dyanos almost 2 years ago - 9 comments

#130 - TODO : Zero stage 1,2 backward

Issue - State: closed - Opened by hyeinhyun almost 2 years ago
Labels: Data Parallelism

#129 - Add inference code

Pull Request - State: closed - Opened by hyunwoongko almost 2 years ago

#128 - Feature/126 refactor all test scripts

Pull Request - State: closed - Opened by hyunwoongko almost 2 years ago

#127 - [#126] refactor all test scripts

Pull Request - State: closed - Opened by hyunwoongko almost 2 years ago

#126 - Refactor all test scripts

Issue - State: closed - Opened by hyunwoongko almost 2 years ago

#125 - remove class_replace argument from wrappers

Pull Request - State: closed - Opened by bzantium almost 2 years ago

#124 - Remove class replace which is unnecessary from TP wrapper.

Issue - State: closed - Opened by bzantium almost 2 years ago

#123 - [#115] Lazy parallelism

Pull Request - State: closed - Opened by hyunwoongko almost 2 years ago

#119 - [fix] T5 fix TP bugs

Pull Request - State: closed - Opened by bzantium almost 2 years ago

#115 - Lazy Parallelization

Issue - State: closed - Opened by hyunwoongko almost 2 years ago
Labels: Pipeline Parallelism, Design

#114 - [MOD] initialize when oslo config not setting

Pull Request - State: closed - Opened by hmy831004 almost 2 years ago - 5 comments

#101 - TODO: Save Deparallelized Expert Parallel Model

Issue - State: closed - Opened by scsc0511 about 2 years ago

#100 - Renaming is fully_sharded_data_parallel.py

Pull Request - State: closed - Opened by josemlopez about 2 years ago

#99 - fix _FullyShardedDataParallelMapping when running test_fsdp.py

Issue - State: closed - Opened by josemlopez about 2 years ago - 1 comment

#98 - [#82] implement deparallelization feature for expert_parallel

Pull Request - State: closed - Opened by scsc0511 about 2 years ago - 4 comments

GitHub / eleutherai/oslo issues and pull requests