epfml/llm-baselines issues and pull requests

#24 - Muon

Pull Request - State: closed - Opened by Andron00e 3 months ago

#23 - Merge from SOAP

Pull Request - State: closed - Opened by Andron00e 3 months ago

#22 - Refactoring + reproducing AdEMAMix

Pull Request - State: closed - Opened by mpagli 3 months ago

#21 - A bunch of new optimizers and schedules

Pull Request - State: open - Opened by Andron00e 4 months ago

#20 - Displaying grad-norm + support wandb with teams

Pull Request - State: closed - Opened by mpagli 4 months ago

#19 - Eval on a fix subset + better lr decay

Pull Request - State: closed - Opened by mpagli 4 months ago

#18 - add methods

Issue - State: open - Opened by Andron00e 4 months ago - 13 comments
Labels: enhancement

#17 - Modified

Pull Request - State: open - Opened by implicitfaith 6 months ago

#16 - np.memmap memory leak and correct val sampling

Pull Request - State: open - Opened by haeggee 8 months ago

#15 - add fineweb dataset

Issue - State: closed - Opened by martinjaggi 8 months ago - 2 comments

#14 - Memory requirements + baseline configs

Issue - State: open - Opened by fabian-sp 8 months ago

#13 - Checkpointing and retrieval

Pull Request - State: closed - Opened by NicolasRR 10 months ago - 4 comments

#12 - license

Issue - State: closed - Opened by fakerybakery 10 months ago

#11 - Create LICENSE

Pull Request - State: closed - Opened by haeggee 10 months ago - 2 comments

#10 - WikiText Data

Issue - State: closed - Opened by thorinf 10 months ago

#9 - implement torch dataloader

Pull Request - State: closed - Opened by haeggee 11 months ago - 2 comments

#8 - Update utils.py - Fixed save_checkpoint with scheduler set to None

Pull Request - State: open - Opened by peacefulotter about 1 year ago

#7 - try with pytorch compile for speedup?

Issue - State: closed - Opened by martinjaggi almost 2 years ago - 1 comment

#6 - add openwebtext2 support

Pull Request - State: closed - Opened by mpagli almost 2 years ago

#5 - add_wandb_key

Pull Request - State: closed - Opened by Olivia-fsm almost 2 years ago

#4 - Modifying few things from mkrima PR