kozistr/pytorch_optimizer issues and pull requests

#185 - [Docs] Fix typo

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, size/XS

#184 - [Docs] README

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, size/XS

#183 - [Feature] Implement Prodigy optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: refactoring, feature, size/XL

#182 - Prodigy: An Expeditiously Adaptive Parameter-Free Learner

Issue - State: closed - Opened by sdbds over 1 year ago - 2 comments
Labels: feature request

#181 - [Fix] Chebyshev LR Scheduler

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: bug, documentation, size/M

#180 - [Refactor] Hessian methods

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, refactoring, size/M

#179 - [Fix] `perturb` isn't multiplied by `-step_size` in SWATS optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: bug, dependencies, size/L

#178 - [Docs] Fix typo

Pull Request - State: closed - Opened by kozistr over 1 year ago
Labels: documentation, size/XS

#177 - [Refactor] AdaHessian, SophiaH optimizers

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, refactoring, cleanup, size/L

#176 - Implement SophiaH & AdaHessian

Pull Request - State: closed - Opened by i404788 over 1 year ago - 8 comments
Labels: feature, size/L

#175 - [Feature] Implement Sophia optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#174 - [Feature] Implement Amos optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#173 - Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training

Issue - State: closed - Opened by redknightlois over 1 year ago - 2 comments
Labels: feature request

#172 - [Fix] Typo

Pull Request - State: closed - Opened by kozistr over 1 year ago
Labels: bug, size/XS

#171 - [Fix] tag name

Pull Request - State: closed - Opened by kozistr over 1 year ago
Labels: bug, size/XS

#170 - [Fix] weight decay in Ranger21 optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: bug, dependencies, size/S

#169 - Ranger21 causing loss to spike and model never converges

Issue - State: closed - Opened by aliencaocao over 1 year ago - 8 comments
Labels: bug

#168 - get_chebyshev_schedule not working

Issue - State: closed - Opened by aliencaocao over 1 year ago - 2 comments
Labels: bug

#167 - Question about using Ranger21 with Hugging Face Trainer

Issue - State: closed - Opened by aliencaocao over 1 year ago - 4 comments
Labels: question

#166 - [Docs] Fix table

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, size/XS

#165 - [Docs] Fix table

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, size/XS

#164 - Plans for pytorch_optimizer v3

Issue - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature

#163 - [Refactor] codes

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, test, refactoring, size/L

#162 - [CI] github action & tests

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, test, size/XXL

#161 - [Refactor] codes

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: refactoring, size/M

#160 - [Feature] Implement AdaDelta optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: bug, feature, size/L

#159 - [Fix] bias correction in D-Adaptation Adam v3 optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: bug, size/S

#158 - [Update] D-Adaptation v3

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#157 - [Feature] Implement AdaShift optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#156 - [Docs] Fix `urllib3` dependency issue

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: bug, documentation, size/L

#155 - [Feature] Implement AvaGrad optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, feature, size/XL

#154 - [Feature] Implement SRMM optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#153 - [Feature] Implement AdaSmooth optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#152 - Implement optimizers

Issue - State: closed - Opened by kozistr over 1 year ago
Labels: feature

#151 - [Feature] Implement Gravity optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#150 - [Refactor] improve code re-usability & remove member variables

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: refactoring, cleanup, feature, size/XXL

#149 - [Refactor] re-usability of the optimization features

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: refactoring, feature, size/XXL

#148 - [Feature] Implement AdaMax optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: enhancement, refactoring, feature, size/L

#147 - [Feature] Implement PID optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#146 - [Feature] Implement QHAdam, QHM optimizers

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#145 - [Feature] Implement AggMo optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#144 - [Feature] Implement AdaMod optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#143 - [Feature] Implement MSVAG optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#142 - [Feature] Implement Fromage optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#141 - [Feature] Implement SWATS optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#140 - [Feature] Implement Yogi optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#139 - [Feature] Implement SGDW, ASGD optimizers

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: bug, feature, size/L

#138 - Implement optimizers

Issue - State: closed - Opened by kozistr over 1 year ago
Labels: feature

#137 - [Feature] Implement Accelerated SGD optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/XL

#136 - [Feature] Implement A2Grad optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#135 - [Update] Lion optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: bug, feature, size/M

#134 - [Feature] Implement RotoGrad optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: bug, enhancement, feature, feature request, size/XL

#133 - [Feature] Implement AdaNorm optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, enhancement, refactoring, cleanup, feature, size/XXL

#132 - [Fix] variables are not located on the same device with the gradients

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: bug, size/S

#131 - Adafactor: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

Issue - State: closed - Opened by Bing-su over 1 year ago - 3 comments
Labels: bug

#130 - [Feature] Implement SM3 Optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: feature, size/L

#129 - [Update] yet another tweak for Scalable Shampoo optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: refactoring, performance, feature, size/L

#128 - [Refactor] Refactor/Optimize Scalable Shampoo optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: enhancement, refactoring, performance, size/L

#127 - [Docs] README

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, size/S

#126 - [Refactor] Ranger21 optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: refactoring, size/M

#125 - [Fix] Ranger21 not skipping update

Pull Request - State: closed - Opened by jdb78 over 1 year ago - 2 comments
Labels: bug, size/XS

#124 - RotoGrad: Gradient Homogenization in Multitask Learning

Issue - State: closed - Opened by redknightlois over 1 year ago - 1 comment
Labels: feature request

#123 - [Fix] `Lookahead` optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: bug, dependencies, size/L

#122 - `AttributeError: 'Lookahead' object has no attribute '_optimizer_step_pre_hooks'` in Pytorch 2.0

Issue - State: closed - Opened by kozistr over 1 year ago - 3 comments
Labels: bug

#121 - [Update] Add epsilon to stablize the optimizng

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: enhancement, refactoring, size/M

#120 - Lion NaN on optimization

Issue - State: closed - Opened by redknightlois over 1 year ago - 2 comments
Labels: enhancement

#119 - [Fix] `str` method supposed to be a function, not property

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: bug, size/L

#118 - Unable to print out Ranger21 Optimizer

Issue - State: closed - Opened by Interpause over 1 year ago - 3 comments
Labels: bug

#117 - [CI] test workflow

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: enhancement, dependencies, size/S

#116 - [Feature] Implement Ali-G optimizer

Pull Request - State: closed - Opened by kozistr over 1 year ago - 1 comment
Labels: documentation, feature, feature request, size/L

#115 - AliG: Training Neural Networks for and by Interpolation

Issue - State: closed - Opened by redknightlois over 1 year ago - 2 comments
Labels: feature request

#114 - foreach implementation for optimizers

Issue - State: closed - Opened by kozistr almost 2 years ago
Labels: feature

#113 - [Feature] Lion optimizer

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: documentation, feature request, size/L

#112 - Gradient Descent: The Ultimate Optimizer

Issue - State: open - Opened by redknightlois almost 2 years ago
Labels: feature request

#111 - VeLO: Training Versatile Learned Optimizers by Scaling Up

Issue - State: open - Opened by redknightlois almost 2 years ago - 1 comment
Labels: feature request

#110 - EvoLved Sign Momentum [Lion] optimizer

Issue - State: closed - Opened by redknightlois almost 2 years ago - 2 comments
Labels: feature request

#109 - [Feature] Implement NovoGrad optimizer

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: documentation, feature, size/L

#108 - [Feature] Implement Apollo optimizer

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: documentation, feature, size/L

#107 - [Feature] Implement AdaFactor optimizer

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: documentation, feature, size/L

#106 - [Deps] Support Pytorch 2.0

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: dependencies, size/M

#105 - support torch 2.0

Issue - State: closed - Opened by Bing-su almost 2 years ago - 2 comments
Labels: dependencies

#104 - [Fix] singular value in `compute_power_svd()`

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: bug, refactoring, size/M

#103 - [Update] Support SVD method to calculate `M^{-1/p}`

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: enhancement, feature, size/L

#102 - [Feature] Stable Weight Decay

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: feature, size/L

#101 - [Feature] Implements D-Adaptation optimizers

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: enhancement, performance, feature, size/XXL

#100 - Updated Shampoo uber slow performance

Issue - State: open - Opened by redknightlois almost 2 years ago - 10 comments
Labels: performance

#99 - [Feature] Implement more Shampoo features

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: feature, size/L

#98 - [Test] Coverage

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: refactoring, size/L

#97 - [Feature] Re-Implement Shampoo Optimizer w/ Grafting & Partitioner

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: enhancement, refactoring, dependencies, cleanup, feature, feature request, size/XXL

#96 - [Refactor] Cleanup the codes

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: bug, refactoring, size/L

#95 - [Feature] Implement & Optimize a few optimizer options

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: enhancement, feature, size/L

#94 - [Feature] Implement GSAM optimizer

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: documentation, feature, size/XL

#93 - Scalable Second Order Optimization for Deep Learning

Issue - State: closed - Opened by redknightlois almost 2 years ago - 3 comments
Labels: feature request

#92 - [Docs] API documentations

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: documentation, size/L

#91 - [Test] Increase the test coverage

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: documentation, enhancement, size/XL

#90 - [Refactor/Docs] Organize Class docstring & Add custom exceptions

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: bug, documentation, enhancement, refactoring, size/XXL

#89 - [Feature] Implement `Adai` optimizer

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: feature, size/L

#88 - [CI] Reduce `num_iterations` to speed up the testing

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: enhancement, size/L

#87 - [CI] Add `pytest-testmon` to reduce testing time

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: dependencies, size/M

#86 - [Build] Upgrade Python version to 3.11 for CI/CD pipeline

Pull Request - State: closed - Opened by kozistr almost 2 years ago - 1 comment
Labels: dependencies, size/L

GitHub / kozistr/pytorch_optimizer issues and pull requests