huggingface/nanotron issues and pull requests

#234 - Ci move

Pull Request - State: closed - Opened by glegendre01 about 2 months ago

#233 - Learning rate restart broken with Nanoset?

Issue - State: open - Opened by Pclanglais about 2 months ago - 5 comments

#222 - lighteval support after checkpoint, UX refactor

Pull Request - State: open - Opened by eliebak 3 months ago - 5 comments

#160 - Enable masking when tp=1

Pull Request - State: closed - Opened by YongjunHe 6 months ago

#100 - Fix some bugs

Pull Request - State: closed - Opened by jordane95 8 months ago - 1 comment

#100 - Fix some bugs

Pull Request - State: closed - Opened by jordane95 8 months ago - 1 comment

#99 - [Features] support gradient checkpointing for memory saving

Issue - State: open - Opened by zguo0525 8 months ago - 1 comment

#99 - [Features] support gradient checkpointing for memory saving

Issue - State: open - Opened by zguo0525 8 months ago - 1 comment

#98 - [Refactor] Refactoring Expert Parallelism

Pull Request - State: closed - Opened by NouamaneTazi 8 months ago - 1 comment

#98 - [Refactor] Refactoring Expert Parallelism

Pull Request - State: closed - Opened by NouamaneTazi 8 months ago - 1 comment

#97 - [Quick fix] fix circular import in logging

Pull Request - State: closed - Opened by NouamaneTazi 8 months ago

#97 - [Quick fix] fix circular import in logging

Pull Request - State: closed - Opened by NouamaneTazi 8 months ago

#96 - Bump v0.4 + Quick refactos

Pull Request - State: closed - Opened by NouamaneTazi 8 months ago

#96 - Bump v0.4 + Quick refactos

Pull Request - State: closed - Opened by NouamaneTazi 8 months ago

#95 - [DoReMi] Small refactors

Pull Request - State: closed - Opened by xrsrke 8 months ago

#95 - [DoReMi] Small refactors

Pull Request - State: closed - Opened by xrsrke 8 months ago

#94 - [Feature] Refactor ParallelContext.world_rank_matrix

Pull Request - State: closed - Opened by 0xkerem 8 months ago - 7 comments

#94 - [Feature] Refactor ParallelContext.world_rank_matrix

Pull Request - State: closed - Opened by 0xkerem 8 months ago - 7 comments

#93 - [Docs] Add unit tests as a requirement

Pull Request - State: closed - Opened by xrsrke 8 months ago - 1 comment

#93 - [Docs] Add unit tests as a requirement

Pull Request - State: closed - Opened by xrsrke 8 months ago - 1 comment

#92 - [Bug] Fix clipping gradients's test

Issue - State: closed - Opened by xrsrke 8 months ago
Labels: bug, good first issue, help wanted, High Priority

#92 - [Bug] Fix clipping gradients's test

Issue - State: closed - Opened by xrsrke 8 months ago
Labels: bug, good first issue, help wanted, High Priority

#91 - [Feature] All GPUs within the same TP group load training data from shared memory

Issue - State: open - Opened by xrsrke 9 months ago
Labels: enhancement, help wanted, Low Priority

#91 - [Feature] All GPUs within the same TP group load training data from shared memory

Issue - State: open - Opened by xrsrke 9 months ago
Labels: enhancement, help wanted, Low Priority

#90 - [Unit Test] Add unit tests for DistributedTrainer

Issue - State: open - Opened by xrsrke 9 months ago - 5 comments
Labels: good first issue, help wanted, High Priority

#90 - [Unit Test] Add unit tests for DistributedTrainer

Issue - State: open - Opened by xrsrke 9 months ago - 5 comments
Labels: good first issue, help wanted, High Priority

#89 - [Unit Test] Add unit test for DoReMi's trainer

Issue - State: open - Opened by xrsrke 9 months ago
Labels: Medium Priority

#89 - [Unit Test] Add unit test for DoReMi's trainer

Issue - State: open - Opened by xrsrke 9 months ago
Labels: Medium Priority

#88 - [Feature] Use CUDA event for measuring elasped time

Issue - State: open - Opened by xrsrke 9 months ago
Labels: enhancement, good first issue, help wanted

#88 - [Feature] Use CUDA event for measuring elasped time

Issue - State: open - Opened by xrsrke 9 months ago
Labels: enhancement, good first issue, help wanted

#87 - [Feature] Asyncronous Serialization

Issue - State: open - Opened by xrsrke 9 months ago
Labels: enhancement, good first issue, help wanted

#87 - [Feature] Asyncronous Serialization

Issue - State: open - Opened by xrsrke 9 months ago
Labels: enhancement, good first issue, help wanted

#86 - [Feature] Kernel Fusion of Layer Norm and GeLU

Issue - State: open - Opened by xrsrke 9 months ago
Labels: enhancement, help wanted

#86 - [Feature] Kernel Fusion of Layer Norm and GeLU

Issue - State: open - Opened by xrsrke 9 months ago
Labels: enhancement, help wanted

#85 - [Feature] LAMB optimizer

Issue - State: open - Opened by xrsrke 9 months ago
Labels: enhancement, help wanted

#85 - [Feature] LAMB optimizer

Issue - State: open - Opened by xrsrke 9 months ago
Labels: enhancement, help wanted

#84 - [Feature] Parallel transformer block

Issue - State: open - Opened by xrsrke 9 months ago
Labels: enhancement, help wanted, High Priority

#84 - [Feature] Parallel transformer block

Issue - State: open - Opened by xrsrke 9 months ago
Labels: enhancement, help wanted, High Priority

#83 - Add Mamba PR

Pull Request - State: closed - Opened by 3outeille 9 months ago - 1 comment

#83 - Add Mamba PR

Pull Request - State: closed - Opened by 3outeille 9 months ago - 1 comment

#82 - [Bug] Not saving `lm_head` in checkpoint

Issue - State: closed - Opened by xrsrke 9 months ago
Labels: bug

#82 - [Bug] Not saving `lm_head` in checkpoint

Issue - State: closed - Opened by xrsrke 9 months ago
Labels: bug

#81 - [Fix] Assert the wrong tolerance of FA2's Layer Norm kernel

Pull Request - State: closed - Opened by xrsrke 9 months ago

#81 - [Fix] Assert the wrong tolerance of FA2's Layer Norm kernel

Pull Request - State: closed - Opened by xrsrke 9 months ago

#80 - [Feature] Add loading different datasets based on training stages

Pull Request - State: closed - Opened by xrsrke 9 months ago

#80 - [Feature] Add loading different datasets based on training stages

Pull Request - State: closed - Opened by xrsrke 9 months ago

#79 - Continued Pretraining on Llama 7b.

Issue - State: open - Opened by wiseyy 9 months ago - 8 comments

#79 - Continued Pretraining on Llama 7b.

Issue - State: open - Opened by wiseyy 9 months ago - 8 comments

#78 - Continued Pretraining on Llama7b.

Issue - State: closed - Opened by wiseyy 9 months ago - 1 comment

#78 - Continued Pretraining on Llama7b.

Issue - State: closed - Opened by wiseyy 9 months ago - 1 comment

#77 - [Feature] Refactor `ParallelContext.world_rank_matrix`

Issue - State: open - Opened by NouamaneTazi 9 months ago
Labels: enhancement, good first issue, help wanted

#77 - [Feature] Refactor `ParallelContext.world_rank_matrix`

Issue - State: open - Opened by NouamaneTazi 9 months ago
Labels: enhancement, good first issue, help wanted

#76 - Deprecate `recompute_granularity` in config

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#76 - Deprecate `recompute_granularity` in config

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#75 - Refactor dMoE

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#75 - Refactor dMoE

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#74 - [Feature] Fix support for sequence parallelism with MoEs

Issue - State: open - Opened by NouamaneTazi 9 months ago
Labels: enhancement, help wanted

#74 - [Feature] Fix support for sequence parallelism with MoEs

Issue - State: open - Opened by NouamaneTazi 9 months ago
Labels: enhancement, help wanted

#73 - Add MoEs support

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#73 - Add MoEs support

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#72 - Support Expert Parallelism

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#72 - Support Expert Parallelism

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#71 - Implement pipeline parallel size-agnostic optimizer state loading

Pull Request - State: closed - Opened by nopperl 9 months ago

#71 - Implement pipeline parallel size-agnostic optimizer state loading

Pull Request - State: closed - Opened by nopperl 9 months ago

#70 - [FP8 Training] End-to-end FP8 Training

Pull Request - State: open - Opened by xrsrke 9 months ago - 3 comments

#70 - [FP8 Training] End-to-end FP8 Training

Pull Request - State: open - Opened by xrsrke 9 months ago - 3 comments

#69 - Refactor `ParallelContext` and some process groups creation

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#69 - Refactor `ParallelContext` and some process groups creation

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#68 - Fix topology agnostic loading

Pull Request - State: closed - Opened by nopperl 9 months ago - 1 comment

#68 - Fix topology agnostic loading

Pull Request - State: closed - Opened by nopperl 9 months ago - 1 comment

#67 - fix configs

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#67 - fix configs

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#66 - quick fix train steps assertion

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago - 1 comment

#66 - quick fix train steps assertion

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago - 1 comment

#65 - quick fix train steps assertion

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#65 - quick fix train steps assertion

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#64 - Update bench script

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#64 - Update bench script

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#63 - [`Docs`] Fix typos

Pull Request - State: closed - Opened by tolgacangoz 9 months ago

#63 - [`Docs`] Fix typos

Pull Request - State: closed - Opened by tolgacangoz 9 months ago

#62 - Refactoring tying mechanism + small fixes

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#62 - Refactoring tying mechanism + small fixes

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#61 - [Feature request] Performance and accuracy benchmarks

Issue - State: open - Opened by brianyu-nexusflowai 9 months ago - 2 comments

#61 - [Feature request] Performance and accuracy benchmarks

Issue - State: open - Opened by brianyu-nexusflowai 9 months ago - 2 comments

#60 - Lighteval naming

Pull Request - State: closed - Opened by thomwolf 9 months ago

#60 - Lighteval naming

Pull Request - State: closed - Opened by thomwolf 9 months ago

#59 - Making lr schedule more flexible

Pull Request - State: closed - Opened by thomwolf 9 months ago

#59 - Making lr schedule more flexible

Pull Request - State: closed - Opened by thomwolf 9 months ago

#58 - Adding eval config

Pull Request - State: closed - Opened by thomwolf 9 months ago

#58 - Adding eval config

Pull Request - State: closed - Opened by thomwolf 9 months ago

#57 - [Question] Modification for Performing Fine-Tuning

Issue - State: open - Opened by allanj 9 months ago - 3 comments

#57 - [Question] Modification for Performing Fine-Tuning

Issue - State: open - Opened by allanj 9 months ago - 3 comments

#56 - [FP8 Training] A single forward and backward pass for a linear in FP8

Pull Request - State: closed - Opened by xrsrke 9 months ago

#56 - [FP8 Training] A single forward and backward pass for a linear in FP8

Pull Request - State: closed - Opened by xrsrke 9 months ago

#55 - random fixes

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#55 - random fixes

Pull Request - State: closed - Opened by NouamaneTazi 9 months ago

#54 - [Feature] DoReMi

Pull Request - State: closed - Opened by xrsrke 10 months ago - 1 comment

#54 - [Feature] DoReMi

Pull Request - State: closed - Opened by xrsrke 10 months ago - 1 comment

#53 - Make custom models a bit easier to use

Pull Request - State: closed - Opened by thomwolf 10 months ago

#53 - Make custom models a bit easier to use

Pull Request - State: closed - Opened by thomwolf 10 months ago

GitHub / huggingface/nanotron issues and pull requests