Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / openlmlab/lomo issues and pull requests

#37 - 公式4疑问

Issue - State: closed - Opened by yaorong1996 over 1 year ago - 1 comment

#37 - 公式4疑问

Issue - State: closed - Opened by yaorong1996 over 1 year ago - 1 comment

#37 - 公式4疑问

Issue - State: closed - Opened by yaorong1996 over 1 year ago - 1 comment

#36 - How to calculate the used GPU memory for each part as in the paper?

Issue - State: open - Opened by liming-ai over 1 year ago - 2 comments

#36 - How to calculate the used GPU memory for each part as in the paper?

Issue - State: open - Opened by liming-ai over 1 year ago - 2 comments

#36 - How to calculate the used GPU memory for each part as in the paper?

Issue - State: open - Opened by liming-ai over 1 year ago - 2 comments

#35 - LOMO+QLoRA简单更改后的报错

Issue - State: closed - Opened by 00drdelius over 1 year ago - 7 comments

#35 - LOMO+QLoRA简单更改后的报错

Issue - State: closed - Opened by 00drdelius over 1 year ago - 7 comments

#35 - LOMO+QLoRA简单更改后的报错

Issue - State: closed - Opened by 00drdelius over 1 year ago - 7 comments

#33 - LORA+LOMO distributed learning

Issue - State: closed - Opened by JiaxiangRen over 1 year ago - 2 comments

#32 - type object 'torch._C._distributed_c10d.ReduceOp' has no attribute 'AVG'

Issue - State: closed - Opened by season1blue over 1 year ago - 4 comments

#31 - Key Error: LOCAL_RANK

Issue - State: closed - Opened by snykral over 1 year ago - 1 comment

#31 - Key Error: LOCAL_RANK

Issue - State: closed - Opened by snykral over 1 year ago - 1 comment

#31 - Key Error: LOCAL_RANK

Issue - State: closed - Opened by snykral over 1 year ago - 1 comment

#30 - about torch.stack(self.grad_norms)

Issue - State: open - Opened by jinzitian over 1 year ago - 3 comments

#30 - about torch.stack(self.grad_norms)

Issue - State: open - Opened by jinzitian over 1 year ago - 3 comments

#30 - about torch.stack(self.grad_norms)

Issue - State: open - Opened by jinzitian over 1 year ago - 3 comments

#28 - llama-33B/llama-65B均报OOM，8*V100跑不起来怎么回事呢？

Issue - State: open - Opened by alisyzhu over 1 year ago - 7 comments

#28 - llama-33B/llama-65B均报OOM，8*V100跑不起来怎么回事呢？

Issue - State: open - Opened by alisyzhu over 1 year ago - 7 comments

#28 - llama-33B/llama-65B均报OOM，8*V100跑不起来怎么回事呢？

Issue - State: open - Opened by alisyzhu over 1 year ago - 7 comments

#27 - Some confusion about the method of the paper

Issue - State: open - Opened by JorunoJobana over 1 year ago - 3 comments

#27 - Some confusion about the method of the paper

Issue - State: open - Opened by JorunoJobana over 1 year ago - 3 comments

#27 - Some confusion about the method of the paper

Issue - State: open - Opened by JorunoJobana over 1 year ago - 3 comments

#26 - Memory consumption first grows up then falls down.

Issue - State: open - Opened by zhenqin96 over 1 year ago - 3 comments

#26 - Memory consumption first grows up then falls down.

Issue - State: open - Opened by zhenqin96 over 1 year ago - 3 comments

#26 - Memory consumption first grows up then falls down.

Issue - State: open - Opened by zhenqin96 over 1 year ago - 3 comments

#25 - Performance Model after Full Fine-tuning by LOMOTrainer

Issue - State: open - Opened by dat-browny over 1 year ago - 9 comments

#25 - Performance Model after Full Fine-tuning by LOMOTrainer

Issue - State: open - Opened by dat-browny over 1 year ago - 9 comments

#25 - Performance Model after Full Fine-tuning by LOMOTrainer

Issue - State: open - Opened by dat-browny over 1 year ago - 9 comments

#23 - wandb permission

Issue - State: closed - Opened by season1blue over 1 year ago - 4 comments

#23 - wandb permission

Issue - State: closed - Opened by season1blue over 1 year ago - 4 comments

#22 - 看了下lomo的代码实现，训练的速度会很慢吗？

Issue - State: closed - Opened by egptee over 1 year ago - 1 comment

#22 - 看了下lomo的代码实现，训练的速度会很慢吗？

Issue - State: closed - Opened by egptee over 1 year ago - 1 comment

#21 - 更充分实验，与Adam的实验效果进行比较

Issue - State: open - Opened by yangjianxin1 over 1 year ago - 6 comments

#21 - 更充分实验，与Adam的实验效果进行比较

Issue - State: open - Opened by yangjianxin1 over 1 year ago - 6 comments

#20 - Is LOMO capable of pre-training a LLM from scratch as well?

Issue - State: open - Opened by YuxingLu613 over 1 year ago - 2 comments

#20 - Is LOMO capable of pre-training a LLM from scratch as well?

Issue - State: open - Opened by YuxingLu613 over 1 year ago - 2 comments

#18 - 数据集问题

Issue - State: open - Opened by wanghao-007 over 1 year ago - 14 comments

#18 - 数据集问题

Issue - State: open - Opened by wanghao-007 over 1 year ago - 14 comments

#17 - 4070ti有机会训练一下吗

Issue - State: closed - Opened by EveningLin over 1 year ago - 1 comment

#17 - 4070ti有机会训练一下吗

Issue - State: closed - Opened by EveningLin over 1 year ago - 1 comment

#15 - I can not find the weights after training

Issue - State: closed - Opened by LeeJodie over 1 year ago - 1 comment

#15 - I can not find the weights after training

Issue - State: closed - Opened by LeeJodie over 1 year ago - 1 comment

#14 - the model weight seems not been updated

Issue - State: closed - Opened by henryxiao1997 over 1 year ago - 4 comments

#14 - the model weight seems not been updated

Issue - State: closed - Opened by henryxiao1997 over 1 year ago - 4 comments

#13 - Testing with P100 on Kaggle

Issue - State: closed - Opened by Iambestfeed over 1 year ago - 7 comments

#13 - Testing with P100 on Kaggle

Issue - State: closed - Opened by Iambestfeed over 1 year ago - 7 comments

#12 - 是否支持量化的模型呀？

Issue - State: closed - Opened by laoda513 over 1 year ago - 4 comments

#12 - 是否支持量化的模型呀？

Issue - State: closed - Opened by laoda513 over 1 year ago - 4 comments

#10 - 4张3090能训练llama13B么，我做了尝试但是失败了

Issue - State: closed - Opened by cc2017111 over 1 year ago - 4 comments

#9 - What is the difference from official PyTorch DDP hooks?

Issue - State: open - Opened by wangkuiyi over 1 year ago - 1 comment

#8 - Train with other datasets collator/loader

Issue - State: closed - Opened by CamaradaLares over 1 year ago - 1 comment

#7 - can you provide the running config of 65b models?

Issue - State: closed - Opened by cyz14 over 1 year ago - 7 comments

#5 - [DOC] add comments in lomo.py and add dependencies in the readme

Pull Request - State: closed - Opened by QipengGuo over 1 year ago - 1 comment

#4 - Gradient accumulation

Issue - State: closed - Opened by EladDv over 1 year ago - 2 comments

#3 - The implementation of LOMO is not released?

Issue - State: closed - Opened by Amshaker over 1 year ago

#2 - time cost of 7b model training compared to AdamW

Issue - State: closed - Opened by dawnranger over 1 year ago - 2 comments

#1 - add downstream experiments

Pull Request - State: closed - Opened by ayyyq over 1 year ago