OpenRLHF/OpenRLHF issues and pull requests

#100 - HfDeepSpeedConfig must be kept during AutoModel.from_pretrained if using ZeRO-3

Issue - State: closed - Opened by wuxibin89 about 1 year ago - 1 comment
Labels: envs

#99 - basemodel and qlora add.

Pull Request - State: closed - Opened by John-Ge about 1 year ago

#98 - Add GPT-4 evaluation scripts

Issue - State: closed - Opened by hijkzzz about 1 year ago - 1 comment
Labels: enhancement

#97 - Do you have a plan for applying Reinforced Self-Training (ReST)?

Issue - State: closed - Opened by missflash about 1 year ago - 1 comment
Labels: enhancement

#96 - Add flash-attention2.0 support

Pull Request - State: closed - Opened by suc16 about 1 year ago
Labels: enhancement

#95 - PPO OOM

Issue - State: closed - Opened by catqaq about 1 year ago - 4 comments
Labels: envs

#94 - 开启ppo-ptx会出现梯度重复计算的报错

Issue - State: closed - Opened by skepsun about 1 year ago - 9 comments

#93 - Support more prompt template in datasets

Issue - State: closed - Opened by hijkzzz about 1 year ago
Labels: enhancement

#92 - 更大的模型

Issue - State: closed - Opened by wanghao-007 about 1 year ago - 2 comments

#91 - 有几个问题

Issue - State: closed - Opened by skepsun about 1 year ago - 2 comments

#90 - available for reward model: OpenAssistant / reward-model-deberta-v3-large-v2

Pull Request - State: closed - Opened by RanchiZhao about 1 year ago - 1 comment

#88 - feat: add wandb logger in ppo trainer

Pull Request - State: closed - Opened by dabney777 about 1 year ago - 2 comments

#87 - Vocabulary overflow Issue with [PAD] for SFT

Issue - State: closed - Opened by leeeizhang about 1 year ago - 4 comments

#86 - feat: add Wandb logger

Pull Request - State: closed - Opened by dabney777 about 1 year ago - 3 comments

#85 - fix ds cpuadam bug

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#84 - fix cpu adam bug

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#83 - Support pretrain and post-pretrain

Issue - State: closed - Opened by catqaq about 1 year ago - 1 comment
Labels: enhancement, P1

#82 - refactor eval

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#81 - Dev add eval/ceval

Pull Request - State: closed - Opened by catqaq about 1 year ago - 1 comment

#80 - Dev

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#79 - revert init on gpu

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#78 - Dev

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#77 - fix update_timesteps

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#76 - support dataset with subfold

Pull Request - State: closed - Opened by wwxFromTju about 1 year ago - 1 comment

#75 - set seed

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#74 - Dev

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#73 - update license

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#72 - add ppo examples and fix container

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#71 - update docker version

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#70 - fix local rank

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#69 - fix gpus_per_node in scripts and readme

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#68 - [#52] Support Multi-nodes training on Slurm

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#67 - Support llama2 flash attention

Issue - State: closed - Opened by hijkzzz about 1 year ago - 2 comments
Labels: enhancement

#66 - Support DPO

Issue - State: closed - Opened by hijkzzz about 1 year ago - 2 comments
Labels: enhancement

#65 - Support checkpoint to prevent training from collapse

Issue - State: closed - Opened by hijkzzz about 1 year ago - 9 comments
Labels: enhancement

#64 - updata readme

Pull Request - State: closed - Opened by pikaqqqqqq about 1 year ago - 2 comments

#63 - pydantic.error_wrappers.ValidationError: 3 validation errors for DeepSpeedZeroConfig zero_hpz_partition_size extra fields not permitted (type=value_error.extra)

Issue - State: closed - Opened by pikaqqqqqq about 1 year ago - 1 comment

#62 - Support Evaluation Tools

Issue - State: closed - Opened by hijkzzz about 1 year ago - 3 comments
Labels: enhancement

#61 - fix readme error and add citation

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#60 - [QUESTION] huggingface login in readme

Issue - State: closed - Opened by suc16 about 1 year ago - 1 comment

#59 - update readme

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#58 - Support Lora & QLora

Issue - State: closed - Opened by hijkzzz about 1 year ago - 4 comments
Labels: enhancement, P0

#56 - Add better docs and usage examples

Issue - State: closed - Opened by hijkzzz about 1 year ago - 2 comments
Labels: documentation, enhancement

#55 - Support Adam Optmizer offload and reload to GPU

Issue - State: closed - Opened by hijkzzz about 1 year ago
Labels: enhancement

#54 - Support wandb logs

Issue - State: closed - Opened by hijkzzz about 1 year ago - 1 comment
Labels: enhancement

#53 - Support Decision Transformer

Issue - State: closed - Opened by hijkzzz about 1 year ago
Labels: enhancement

#52 - Support Multi-nodes training on Slurm

Issue - State: closed - Opened by hijkzzz about 1 year ago - 1 comment

#51 - Support Multiple Reward Models

Issue - State: closed - Opened by hijkzzz about 1 year ago - 3 comments
Labels: enhancement

#50 - Support Rejection Sampling

Issue - State: closed - Opened by hijkzzz about 1 year ago - 2 comments
Labels: enhancement, P0

#49 - add: save huggingface checkpoint

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#48 - Support running on Ray as distributed RLHF framework.

Issue - State: closed - Opened by jovany-wang about 1 year ago - 1 comment
Labels: enhancement

#47 - Introduce LINT tools

Issue - State: closed - Opened by jovany-wang about 1 year ago - 2 comments
Labels: enhancement

#46 - remove cuda_launch_blocking

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#45 - fix oom

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#44 - Dev

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#43 - new datasets

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#42 - fix scripts

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#41 - fix prompt data name

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#40 - fix

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#39 - fix train_ppo args

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#38 - dataset

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#37 - add orca datasets

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#36 - add log task

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#35 - use llama2 pretrain

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#34 - replace base model

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#33 - polish readme

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#32 - add more datasets and fix some bugs/readme.md

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#31 - fix readme

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#30 - fix args

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#29 - use micro_batch_size and batch_size; fix ds bugs

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#28 - fix args store_true

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#27 - fix llama2

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#26 - polish readme

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#25 - fix train_ppo

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#24 - fix train_rm

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#23 - fix deepspeed and continaer

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#22 - fix

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#21 - fix

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#20 - add english readme

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#19 - fix typo F.pad + device

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#18 - isort python modules

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#17 - add dpo train.py

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#16 - fix ds offload

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#15 - fix readme.md

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#14 - polish dirs

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#13 - remove dschat

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#12 - refactor examples

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#11 - experience makder and buffer and flash attn

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#10 - add datasets

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#9 - Refactoring code structure

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#8 - add readme.md

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#7 - remove duplicated README.md

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#6 - modify target (fix author)

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#5 - modify target

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

#4 - update readme: Pro intro, dev plan and division of labor

Pull Request - State: closed - Opened by OpenLLMAI about 1 year ago

#3 - Dev

Pull Request - State: closed - Opened by OpenLLMAI about 1 year ago

#2 - add OpenLLaMA2 docs

Pull Request - State: closed - Opened by OpenLLMAI about 1 year ago
Labels: documentation

#1 - init project

Pull Request - State: closed - Opened by hijkzzz about 1 year ago

GitHub / OpenRLHF/OpenRLHF issues and pull requests