kozistr/pytorch_optimizer issues and pull requests

#273 - Support SaRA

Issue - State: open - Opened by sdbds 19 days ago
Labels: feature request

#272 - [Feature] Implement AdEMAMix optimizer

Pull Request - State: closed - Opened by kozistr 19 days ago - 1 comment
Labels: documentation, dependencies, feature, size/M

#271 - [Fix] Add missing kwargs

Pull Request - State: closed - Opened by kozistr about 2 months ago - 1 comment
Labels: bug, size/M

#270 - The `create_optimizer()` function fails when Lookahead arguments are provided

Issue - State: closed - Opened by Vectorrent about 2 months ago - 1 comment
Labels: bug

#269 - [Release] v3.1.1

Pull Request - State: closed - Opened by kozistr about 2 months ago - 1 comment
Labels: size/XS

#268 - [Update] Coverage

Pull Request - State: closed - Opened by kozistr about 2 months ago - 1 comment
Labels: size/XS

#267 - [Fix] device mismatch problems

Pull Request - State: closed - Opened by kozistr about 2 months ago
Labels: bug, size/S

#266 - Prodigy optimizer cannot be resumed from checkpoint

Issue - State: closed - Opened by Vectorrent about 2 months ago - 4 comments
Labels: bug

#265 - [Feature] Implement AdamG optimizer

Pull Request - State: closed - Opened by kozistr about 2 months ago - 1 comment
Labels: documentation, dependencies, feature request, size/M

#264 - AdamG: Towards Stability of Parameter-free Optimization

Issue - State: closed - Opened by Vectorrent about 2 months ago - 6 comments
Labels: feature request

#263 - [Feature] Implement TRAC optimizer

Pull Request - State: closed - Opened by kozistr about 2 months ago - 1 comment
Labels: feature request, size/XL

#262 - Add WeLore

Issue - State: closed - Opened by rotem154154 2 months ago - 2 comments
Labels: feature request

#261 - [Refactor] code

Pull Request - State: closed - Opened by kozistr 2 months ago - 1 comment
Labels: enhancement, dependencies, size/L

#260 - [Update] Improve the performance

Pull Request - State: closed - Opened by kozistr 2 months ago - 1 comment
Labels: enhancement, performance, dependencies, size/S

#259 - [Refactor] Stuffs

Pull Request - State: closed - Opened by kozistr 2 months ago - 1 comment
Labels: enhancement, performance, size/M

#258 - [Feature] Implement `AdaLOMO` optimizer and others

Pull Request - State: closed - Opened by kozistr 3 months ago - 1 comment
Labels: documentation, enhancement, refactoring, dependencies, feature, size/XL

#257 - [BUG Fix]Fix 3 bugs for Adam-Mini

Pull Request - State: closed - Opened by sdbds 3 months ago - 1 comment
Labels: bug, size/XS

#256 - Add TRAC

Issue - State: closed - Opened by sdbds 3 months ago
Labels: feature request

#255 - [Feature] Add visualizations

Pull Request - State: closed - Opened by kozistr 3 months ago - 1 comment
Labels: documentation, feature

#254 - [Update] v3.0.2

Pull Request - State: closed - Opened by kozistr 3 months ago - 1 comment
Labels: feature, size/XS

#253 - [Feature] Implement AdamMini optimizer

Pull Request - State: closed - Opened by kozistr 3 months ago - 1 comment
Labels: documentation, feature request, size/L

#252 - [Feature] Implement StableAdamW optimizer

Pull Request - State: closed - Opened by kozistr 3 months ago - 1 comment
Labels: documentation, feature request, size/M

#251 - [Feature] Implement Kate optimizer

Pull Request - State: closed - Opened by kozistr 3 months ago - 1 comment
Labels: documentation, feature request, size/L

#250 - Add StableAdamW

Issue - State: closed - Opened by tfriedel 3 months ago
Labels: feature request

#249 - Add Kate

Issue - State: closed - Opened by tfriedel 3 months ago - 6 comments
Labels: feature request

#248 - [Feature] Implement WSD lr scheduler

Pull Request - State: closed - Opened by kozistr 3 months ago - 1 comment
Labels: documentation, enhancement, refactoring, dependencies, feature request, size/L

#247 - Add WSD LR scheduler

Issue - State: closed - Opened by sdbds 3 months ago
Labels: feature request

#246 - Add Adam-Mini

Issue - State: closed - Opened by sdbds 3 months ago - 1 comment
Labels: feature request

#245 - [Feature] Implement GrokFast optimizer

Pull Request - State: closed - Opened by kozistr 3 months ago - 1 comment
Labels: documentation, dependencies, feature request, size/L

#244 - GrokFast: Accelerated Grokking by Amplifying Slow Gradients

Issue - State: closed - Opened by redknightlois 3 months ago
Labels: feature request

#243 - [Feature] Tweak AdaFactor optimizer

Pull Request - State: closed - Opened by kozistr 4 months ago - 1 comment
Labels: feature request, size/S

#242 - [Feature] Implement FAdam optimizer

Pull Request - State: closed - Opened by kozistr 4 months ago - 1 comment
Labels: documentation, feature request, size/M

#241 - FAdam

Issue - State: closed - Opened by tranquan687 4 months ago - 1 comment
Labels: feature request

#240 - fix: reg_noise typing

Pull Request - State: closed - Opened by michaldyczko 4 months ago - 1 comment
Labels: bug, size/XS

#239 - Wrong typing of `reg_noise`

Issue - State: closed - Opened by michaldyczko 4 months ago
Labels: bug

#238 - fix: change Lookahead's param_groups to a property

Pull Request - State: closed - Opened by michaldyczko 4 months ago - 1 comment
Labels: bug, size/XS

#237 - Lookahead`s param_groups attribute is not loaded from checkpoint.

Issue - State: closed - Opened by michaldyczko 4 months ago
Labels: bug

#236 - Modified AdaFactor by ViT paper

Issue - State: closed - Opened by aliencaocao 4 months ago - 1 comment
Labels: feature request

#235 - [Docs] `v3.0.0`

Pull Request - State: closed - Opened by kozistr 4 months ago - 1 comment
Labels: documentation, dependencies, size/S

#234 - build(deps): bump jinja2 from 3.1.3 to 3.1.4

Pull Request - State: closed - Opened by dependabot[bot] 5 months ago - 1 comment
Labels: dependencies, size/XS

#233 - [Feature] Lots of stuffs

Pull Request - State: closed - Opened by kozistr 5 months ago - 1 comment
Labels: documentation, enhancement, dependencies, feature request, size/XL

#232 - Ranger sign inversion

Issue - State: open - Opened by i404788 5 months ago - 1 comment
Labels: bug

#231 - Entropy-MCMC: Sampling from flat basins with ease

Issue - State: closed - Opened by redknightlois 6 months ago
Labels: feature request

#230 - ScheduleFree

Issue - State: closed - Opened by sdbds 6 months ago
Labels: feature request

#229 - [Feature] Implement Adalite optimizer

Pull Request - State: closed - Opened by kozistr 6 months ago - 1 comment
Labels: feature request, size/M

#228 - [Feature] Implement GaLore optimizer

Pull Request - State: closed - Opened by kozistr 6 months ago - 1 comment
Labels: dependencies, feature request, size/L

#227 - [Bugfix] Fix SRMM to allow operation beyond memory_length

Pull Request - State: closed - Opened by i404788 6 months ago - 1 comment
Labels: bug, size/XS

#226 - build(deps-dev): bump black from 24.2.0 to 24.3.0

Pull Request - State: closed - Opened by dependabot[bot] 6 months ago - 1 comment
Labels: dependencies, size/XS

#225 - Adalite

Issue - State: closed - Opened by sdbds 6 months ago - 1 comment
Labels: feature request

#224 - GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

Issue - State: closed - Opened by redknightlois 7 months ago - 1 comment
Labels: feature request

#223 - [Build] Use `uv` to install pip packages

Pull Request - State: closed - Opened by kozistr 7 months ago
Labels: refactoring, dependencies, size/S

#222 - [Feature] Implement REX lr scheduler

Pull Request - State: closed - Opened by kozistr 7 months ago - 4 comments
Labels: feature request, size/M

#221 - [Feature] Implement Aida optimizer

Pull Request - State: closed - Opened by kozistr 7 months ago - 1 comment
Labels: dependencies, feature request, size/L

#220 - Aida optimizer

Issue - State: closed - Opened by okbalefthanded 8 months ago - 3 comments
Labels: feature request

#219 - fix links about papers

Pull Request - State: closed - Opened by sdbds 8 months ago
Labels: documentation, size/XS

#218 - build(deps): bump jinja2 from 3.1.2 to 3.1.3

Pull Request - State: closed - Opened by dependabot[bot] 9 months ago - 1 comment
Labels: dependencies, size/XS

#217 - [Feature request]REX LR scheduler

Issue - State: closed - Opened by sdbds 9 months ago - 2 comments
Labels: feature request

#216 - [Feature] Implement WSAM optimizer

Pull Request - State: closed - Opened by kozistr 10 months ago - 1 comment
Labels: test, feature request, size/XL

#215 - [Docs] Add missing Ranger21 parameters

Pull Request - State: closed - Opened by kozistr 10 months ago - 1 comment
Labels: documentation, test, dependencies, size/M

#214 - Ranger21 has undocumented required arguments

Issue - State: closed - Opened by LuciferianInk 10 months ago - 3 comments
Labels: bug

#213 - FR: Sharpness-Aware Minimization Revisited: Weighted Sharpness as a Regularization Term (WSAM)

Issue - State: closed - Opened by LiutongZhou 11 months ago - 2 comments
Labels: feature request

#212 - Improvement to SAM: SAM as an Optimal Relaxation of Bayes

Issue - State: closed - Opened by redknightlois 11 months ago - 1 comment
Labels: feature request

#211 - [Feature] Support `bitsandbytes` optimizers

Pull Request - State: closed - Opened by kozistr 12 months ago - 1 comment
Labels: dependencies, feature, size/XXL

#210 - ipex failed for Adan from pytorch_optimizer

Issue - State: closed - Opened by ldv1 about 1 year ago - 1 comment
Labels: bug

#209 - [Docs] Update documentation

Pull Request - State: closed - Opened by kozistr about 1 year ago - 1 comment
Labels: documentation, size/S

#208 - Request to add 4-bit AdamW

Issue - State: open - Opened by LiutongZhou about 1 year ago - 3 comments
Labels: feature request

#207 - [Docs] Index page

Pull Request - State: closed - Opened by kozistr about 1 year ago - 1 comment
Labels: documentation, size/L

#206 - [Docs] Migrate to mkdocs

Pull Request - State: closed - Opened by kozistr about 1 year ago - 1 comment
Labels: documentation, size/XXL

#205 - [Fix] Documentation isn't built successfully

Pull Request - State: closed - Opened by kozistr about 1 year ago - 1 comment
Labels: bug, documentation, size/S

#204 - Empty Docs Sections

Issue - State: closed - Opened by InfluxOW about 1 year ago - 6 comments
Labels: bug

#203 - [Feature] Implement DAdaptLion optimizer

Pull Request - State: closed - Opened by kozistr about 1 year ago - 1 comment
Labels: documentation, feature, size/M

#202 - [Refactor] Lookahead optimizer

Pull Request - State: closed - Opened by kozistr about 1 year ago - 1 comment
Labels: dependencies, cleanup, size/S

#201 - [Fix] Make Lookahead a subclass of Optimizer

Pull Request - State: closed - Opened by georg-wolflein about 1 year ago - 3 comments
Labels: bug, size/XS

#200 - Lookahead is not a subclass of torch.optim.Optimizer

Issue - State: closed - Opened by georg-wolflein about 1 year ago - 4 comments
Labels: bug

#199 - [Refactor] Fix styles

Pull Request - State: closed - Opened by kozistr about 1 year ago - 1 comment
Labels: refactoring, size/XS

#198 - [Feature] Implement loss functions

Pull Request - State: closed - Opened by kozistr about 1 year ago - 1 comment
Labels: dependencies, feature, size/M

#197 - [Docs] Fix typo

Pull Request - State: closed - Opened by kozistr about 1 year ago
Labels: documentation, size/XS

#196 - [Feature] Implement CAME optimizer

Pull Request - State: closed - Opened by kozistr about 1 year ago - 1 comment
Labels: feature, size/XL

#195 - Adding the CAME optimizer

Issue - State: closed - Opened by LiutongZhou about 1 year ago - 2 comments
Labels: feature request

#194 - sophiah in https://github.com/booydar/LM-RMT

Issue - State: open - Opened by robotzheng about 1 year ago - 8 comments
Labels: bug

#193 - sophiah bug

Issue - State: closed - Opened by robotzheng about 1 year ago - 5 comments
Labels: bug

#192 - [Feature] Implement Tiger optimizer

Pull Request - State: closed - Opened by kozistr about 1 year ago - 1 comment
Labels: feature, size/M

#191 - Can a variant of Lion named Tiger be added to your package?

Issue - State: closed - Opened by liujuncn about 1 year ago - 2 comments
Labels: feature request

#190 - [Docs] SECURITY.md

Pull Request - State: closed - Opened by kozistr over 1 year ago
Labels: documentation, size/XS

#189 - [Feature] Implement loss functions

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, feature, size/XXL

#188 - [Feature] Implement LOMO optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: refactoring, dependencies, feature, size/L

#187 - LOMO: LOw-Memory Optimization

Issue - State: closed - Opened by sdbds over 1 year ago - 1 comment
Labels: feature request

#186 - [Feature] Implement PAdam optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/M

#185 - [Docs] Fix typo

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, size/XS

#184 - [Docs] README

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, size/XS

#183 - [Feature] Implement Prodigy optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: refactoring, feature, size/XL

#182 - Prodigy: An Expeditiously Adaptive Parameter-Free Learner

Issue - State: closed - Opened by sdbds over 1 year ago - 2 comments
Labels: feature request

#181 - [Fix] Chebyshev LR Scheduler

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: bug, documentation, size/M

#180 - [Refactor] Hessian methods

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, refactoring, size/M

#179 - [Fix] `perturb` isn't multiplied by `-step_size` in SWATS optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: bug, dependencies, size/L

#178 - [Docs] Fix typo

Pull Request - State: closed - Opened by kozistr over 1 year ago
Labels: documentation, size/XS

#177 - [Refactor] AdaHessian, SophiaH optimizers

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, refactoring, cleanup, size/L

#176 - Implement SophiaH & AdaHessian

Pull Request - State: closed - Opened by i404788 over 1 year ago - 8 comments
Labels: feature, size/L

#175 - [Feature] Implement Sophia optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#174 - [Feature] Implement Amos optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

GitHub / kozistr/pytorch_optimizer issues and pull requests