eleutherai/oslo issues and pull requests

#97 - PatricStar for Zero

Issue - State: open - Opened by dongsungkim about 2 years ago

#96 - Integration ZeroDDP and ShardedModelv2 from colossal AI

Issue - State: open - Opened by dongsungkim about 2 years ago - 1 comment

#95 - [hotfix] fixed tp deparallelized deadlock

Pull Request - State: closed - Opened by jason9693 about 2 years ago - 1 comment

#94 - Feature/99 colo ddp integration

Pull Request - State: closed - Opened by dongsungkim about 2 years ago - 4 comments

#93 - coloDDP integration

Issue - State: closed - Opened by dongsungkim about 2 years ago - 1 comment

#92 - Feature/91 deparallelize pp

Pull Request - State: closed - Opened by ohwi about 2 years ago - 4 comments

#91 - TODO: Deparallelize Pipeline Parallel

Issue - State: closed - Opened by ohwi about 2 years ago
Labels: Pipeline Parallelism

#90 - Update trainer to modify code by additional requirements

Pull Request - State: closed - Opened by tree-park about 2 years ago - 1 comment

#89 - fix _TensorParallelMappingForHuggingFace error

Pull Request - State: closed - Opened by dongsungkim about 2 years ago - 4 comments

#88 - No _TensorParallelMappingForHuggingFace

Issue - State: closed - Opened by dongsungkim about 2 years ago - 1 comment

#87 - Feature/86 add mapping for oslo model

Pull Request - State: closed - Opened by loopinf about 2 years ago

#86 - add mapping for oslo model

Issue - State: closed - Opened by loopinf about 2 years ago

#85 - Feature/84 apply vocab parallel cross entropy

Pull Request - State: closed - Opened by bzantium about 2 years ago - 1 comment

#84 - Apply vocab parallel cross entropy for oslo models

Issue - State: closed - Opened by bzantium about 2 years ago

#83 - Feature/2 tp functional

Pull Request - State: closed - Opened by jason9693 about 2 years ago

#82 - TODO : Deparallelize expert parallel

Issue - State: closed - Opened by scsc0511 about 2 years ago
Labels: Expert Parallelism

#81 - implement vocab parallel cross entropy for TP 1d, 2d, 2p5d, 3d and te…

Pull Request - State: closed - Opened by bzantium about 2 years ago

#80 - Implement vocab parallel cross entropy loss

Issue - State: closed - Opened by bzantium about 2 years ago

#79 - Implement vocab parall crossentropy loss

Issue - State: closed - Opened by loopinf about 2 years ago
Labels: Tensor Parallelism

#78 - [#77] refactoring transformers wrap

Pull Request - State: closed - Opened by minqukanq about 2 years ago

#77 - Refactoring transformers wrap

Issue - State: closed - Opened by minqukanq about 2 years ago

#76 - [#74] wrapping transformers layers

Pull Request - State: closed - Opened by minqukanq about 2 years ago

#75 - [#74] wrap transformers layer

Pull Request - State: closed - Opened by minqukanq about 2 years ago

#74 - wrap transformers layer

Issue - State: closed - Opened by minqukanq about 2 years ago

#73 - (chore) add cuda checks, cuda_ext

Pull Request - State: closed - Opened by ghost about 2 years ago

#72 - Add Trainer new version to main branch

Pull Request - State: closed - Opened by tree-park about 2 years ago - 5 comments

#71 - [#70] pass multiple optim params

Pull Request - State: closed - Opened by minqukanq about 2 years ago

#70 - pass optimizer parameters

Issue - State: closed - Opened by minqukanq about 2 years ago - 2 comments

#69 - [#67] Rename SP

Pull Request - State: closed - Opened by hyunwoongko about 2 years ago

#68 - [#67] Rename SP

Pull Request - State: closed - Opened by hyunwoongko about 2 years ago

#67 - Rename SequenceDataParallel to SequenceParallel

Issue - State: closed - Opened by hyunwoongko about 2 years ago
Labels: Sequence Parallel

#66 - FSDP returns different loss value with zero stage 2 and 3

Issue - State: closed - Opened by dongsungkim about 2 years ago - 1 comment
Labels: Data Parallelism

#65 - (chore) add small formattings & versioning to setup.py, add README.md

Pull Request - State: closed - Opened by ghost about 2 years ago - 2 comments

#64 - [#5] redesign zero module

Pull Request - State: closed - Opened by minqukanq about 2 years ago - 2 comments

#63 - [#62] feat: Use fused_scale_mask_softmax functionality on GPT2

Pull Request - State: closed - Opened by loopinf about 2 years ago - 1 comment

#62 - fused_scale_mask_softmax on GPT2 model

Issue - State: closed - Opened by loopinf about 2 years ago
Labels: Kernal Fusion

#61 - [#42] pp data comm func

Pull Request - State: closed - Opened by ohwi about 2 years ago

#60 - [#59] add distributedSampler to DDP and modularise the code

Pull Request - State: closed - Opened by dongsungkim about 2 years ago

#59 - Add datadistribedSampler to DDP

Issue - State: closed - Opened by dongsungkim about 2 years ago

#58 - feat: Activate fused_scale_mask_softmax function

Pull Request - State: closed - Opened by loopinf about 2 years ago

#57 - [#44] switch: code base of expert parallel from colossalai to deepspeed

Pull Request - State: closed - Opened by scsc0511 about 2 years ago

#56 - [#55] Make sequence parallel splitting automatic

Pull Request - State: closed - Opened by hyunwoongko about 2 years ago
Labels: Sequence Parallel

#55 - Make sequence parallel splitting automatic

Issue - State: closed - Opened by hyunwoongko about 2 years ago
Labels: Sequence Parallel

#54 - remove checking devices of parameters

Pull Request - State: closed - Opened by dongsungkim about 2 years ago
Labels: Bug, Data Parallelism, Sequence Parallel

#53 - Change bert model to use `_fused_scale_mask_softmax` functions

Issue - State: closed - Opened by loopinf about 2 years ago - 2 comments
Labels: Kernal Fusion

#52 - [#50] Modify PP to functional

Pull Request - State: closed - Opened by hyunwoongko about 2 years ago

#51 - [#50] Modify PP to functional

Pull Request - State: closed - Opened by hyunwoongko about 2 years ago

#50 - Modify PP to functional

Issue - State: closed - Opened by hyunwoongko about 2 years ago
Labels: Pipeline Parallelism, Design

#49 - SP parameter device type error

Issue - State: closed - Opened by dongsungkim about 2 years ago
Labels: Bug, Data Parallelism, Sequence Parallel

#48 - TODO: Test TP + PP

Issue - State: open - Opened by hyunwoongko about 2 years ago
Labels: Pipeline Parallelism, Tensor Parallelism

#47 - [#42] pp data comm func (WIP)

Pull Request - State: closed - Opened by ohwi about 2 years ago

#46 - Feature/32 fix bug f linear

Pull Request - State: closed - Opened by loopinf about 2 years ago - 1 comment

#45 - fix: restore some import functions

Pull Request - State: closed - Opened by loopinf about 2 years ago - 1 comment

#44 - TODO : switch code base of expert parallel from colossalai to deepspeed

Issue - State: closed - Opened by scsc0511 about 2 years ago - 1 comment
Labels: Expert Parallelism

#43 - fused_bias_gelu is missing when call BertModel

Issue - State: closed - Opened by loopinf about 2 years ago - 1 comment
Labels: Bug, Kernal Fusion

#42 - TODO: clean data communication functions for PP

Issue - State: open - Opened by ohwi about 2 years ago
Labels: Pipeline Parallelism

#41 - Expert parallel

Pull Request - State: closed - Opened by scsc0511 about 2 years ago - 4 comments

#40 - [#39] Fix sorting error when allocating params

Pull Request - State: closed - Opened by bzantium about 2 years ago - 1 comment

#39 - Fix sorting error at allocate_param function

Issue - State: closed - Opened by bzantium about 2 years ago
Labels: Bug

#38 - Expert parallel

Pull Request - State: closed - Opened by scsc0511 about 2 years ago

#37 - sort dictionary correctly

Pull Request - State: closed - Opened by bzantium about 2 years ago

#36 - rename tensor_parallel: parallel_nd to nd

Pull Request - State: closed - Opened by bzantium about 2 years ago - 1 comment

#35 - [#34] Fix some code bugs

Pull Request - State: closed - Opened by hyunwoongko about 2 years ago

#34 - Fix some code errors

Issue - State: closed - Opened by hyunwoongko about 2 years ago
Labels: Design

#33 - Add description how to use fused_scale_softmax

Issue - State: closed - Opened by loopinf about 2 years ago - 1 comment
Labels: Kernal Fusion, Documentation

#32 - Error on test_modeling_bert.py

Issue - State: closed - Opened by loopinf about 2 years ago
Labels: Bug

#31 - add GPT-NeoX mapping

Pull Request - State: closed - Opened by bzantium about 2 years ago

#30 - [#29] Add FSDP mapping

Pull Request - State: closed - Opened by hyunwoongko about 2 years ago

#29 - Add FullyShardedDataParallelMapping

Issue - State: closed - Opened by hyunwoongko about 2 years ago
Labels: Data Parallelism

#28 - [#27] Make activation checkpointing module

Pull Request - State: closed - Opened by hyunwoongko about 2 years ago

#27 - Use oslo activation checkpointing rather than torch activation checkpointing

Issue - State: closed - Opened by hyunwoongko about 2 years ago
Labels: Activation Checkpointing

#26 - [#24] Modify many structures for 2.1

Pull Request - State: closed - Opened by hyunwoongko about 2 years ago

#25 - comment out unnecessary initialized import

Pull Request - State: closed - Opened by loopinf about 2 years ago

#24 - Modify many structures for 2.1

Issue - State: closed - Opened by hyunwoongko about 2 years ago
Labels: Design

#23 - fix parameter error in logger.set_contruction_data_and_log : missing …

Pull Request - State: closed - Opened by dongsungkim about 2 years ago

#22 - [#21] Refactor tasks module

Pull Request - State: closed - Opened by hyunwoongko about 2 years ago

#21 - TODO: refactor tasks module

Issue - State: closed - Opened by hyunwoongko about 2 years ago
Labels: Design

#20 - Pipeline parallelism ohwi

Pull Request - State: closed - Opened by ohwi about 2 years ago

#19 - [#7] Redesign DP and SP

Pull Request - State: closed - Opened by hyunwoongko about 2 years ago

#18 - Fix slurm local world size

Issue - State: closed - Opened by hyunwoongko about 2 years ago
Labels: Bug

#17 - [#10] fix many bugs

Pull Request - State: closed - Opened by hyunwoongko about 2 years ago

#16 - [#10] Fix many bugs

Pull Request - State: closed - Opened by hyunwoongko about 2 years ago

#15 - wand module not found when running test_mlm.py

Issue - State: open - Opened by josemlopez about 2 years ago - 1 comment
Labels: Bug

#14 - 'TrainingArguments' object has no attribute 'parallel_mode' when running mBart test

Issue - State: open - Opened by josemlopez about 2 years ago - 1 comment
Labels: Bug

#13 - No module named oslo.transformers.data when running mBart test.

Issue - State: closed - Opened by josemlopez about 2 years ago - 1 comment
Labels: Bug

#12 - No module named 'oslo.torch.experimental' when running mbart test

Issue - State: closed - Opened by josemlopez about 2 years ago - 1 comment
Labels: Bug

#11 - Syntax error in mappings_utils.py when installing OSLO

Issue - State: closed - Opened by josemlopez about 2 years ago - 1 comment
Labels: Bug

#10 - Error when installing OSLO

Issue - State: closed - Opened by josemlopez about 2 years ago - 1 comment
Labels: Bug

#9 - [#8] Add TODO template

Pull Request - State: closed - Opened by hyunwoongko over 2 years ago

#8 - TODO: Add TODO template.

Issue - State: closed - Opened by hyunwoongko over 2 years ago
Labels: Management

#7 - TODO: Redesign DDP module

Issue - State: closed - Opened by hyunwoongko over 2 years ago
Labels: Data Parallelism, Design

#6 - Milestone: Make `oslo-examples` repository

Issue - State: open - Opened by hyunwoongko over 2 years ago
Labels: Milestone

#5 - TODO: Redesign ZeRO modules

Issue - State: closed - Opened by hyunwoongko over 2 years ago - 1 comment
Labels: Data Parallelism, Design

#4 - TODO: Make DP + EP available

Issue - State: closed - Opened by hyunwoongko over 2 years ago - 1 comment
Labels: Expert Parallelism

#3 - Milestone: OSLO 2.1

Issue - State: closed - Opened by hyunwoongko over 2 years ago
Labels: Milestone

#2 - TODO: Modify wrapper design to functional

Issue - State: closed - Opened by hyunwoongko over 2 years ago
Labels: Design

#1 - TODO: Fix pipeline parallelism bugs

Issue - State: closed - Opened by hyunwoongko over 2 years ago - 1 comment
Labels: Bug, Pipeline Parallelism

GitHub / eleutherai/oslo issues and pull requests