huggingface/peft issues and pull requests

#1906 - [WIP] ENH Add support for Qwen2

Pull Request - State: open - Opened by BenjaminBossan 2 days ago - 1 comment

#1905 - Add MoSLoRA

Pull Request - State: open - Opened by wutaiqiang 3 days ago - 1 comment

#1904 - Different results from multiple lora weight merge method

Issue - State: open - Opened by daegonYu 3 days ago - 3 comments

#1903 - How to use multiple GPUs

Issue - State: open - Opened by Lihwnlp 3 days ago - 6 comments

#1902 - Model encapsulation

Issue - State: open - Opened by tomekrut 4 days ago - 1 comment

#1901 - support Grouped-Query Attention

Pull Request - State: open - Opened by ttw1018 4 days ago - 5 comments

#1900 - FIX: More VeRA tests, fix tests, more checks

Pull Request - State: open - Opened by BenjaminBossan 4 days ago - 5 comments

#1899 - chore: markdown formatting

Pull Request - State: closed - Opened by stillmatic 5 days ago - 4 comments

#1898 - FIX TEST Even higher tolerance for AdaLoRA in test

Pull Request - State: closed - Opened by BenjaminBossan 5 days ago - 2 comments

#1897 - FIX TEST: Higher tolerance for AdaLoRA in test

Pull Request - State: closed - Opened by BenjaminBossan 5 days ago - 2 comments

#1896 - CI: Don't fail fast in test matrix

Pull Request - State: closed - Opened by BenjaminBossan 5 days ago - 1 comment

#1894 - PeftModel.from_pretrained not load modules_to_save in config file

Issue - State: closed - Opened by double-fire-0 8 days ago - 4 comments

#1893 - LORA finetuning gradients are scaled by a unknown constant factor

Issue - State: closed - Opened by goliaro 8 days ago - 2 comments

#1892 - FIX Recursion while accessing attribute before initialization

Pull Request - State: closed - Opened by ret-1 8 days ago - 6 comments

#1891 - test: bump absolute tolerance level in test

Pull Request - State: closed - Opened by kallewoof 9 days ago - 7 comments

#1890 - ValueError: Trying to set a tensor of shape torch.Size([43176, 8192]) in "weight" (which has shape torch.Size([0])), this look incorrect.

Issue - State: closed - Opened by KarasZhang 9 days ago - 15 comments

#1889 - local pytest run puts torch.allclose() out of range in test_4bit_lora_mixed_adapter_batches_lora

Issue - State: closed - Opened by kallewoof 9 days ago - 6 comments

#1888 - bypass print_trainable_parameter() if model is not peft model

Pull Request - State: closed - Opened by delock 10 days ago - 1 comment

#1887 - FIX Make special LoRA inits DeepSpeed compatible

Pull Request - State: closed - Opened by BenjaminBossan 11 days ago - 5 comments

#1886 - you cannot fine-tune quantized model with torch.compile

Issue - State: closed - Opened by fengyunflya 11 days ago - 2 comments

#1885 - Dora Fine-tuning added to examples

Pull Request - State: closed - Opened by shirinyamani 11 days ago - 5 comments

#1884 - FIX Init AdaLoRA to be identity transform

Pull Request - State: closed - Opened by BenjaminBossan 12 days ago - 1 comment

#1883 - After fine-tuning, the model inference is abnormal

Issue - State: open - Opened by mumu029 12 days ago - 1 comment

#1882 - Workflow: Update bug report template

Pull Request - State: closed - Opened by younesbelkada 15 days ago - 1 comment

#1881 - eval_loss missing when using peft model with STFTrainer

Issue - State: open - Opened by J4Q8 15 days ago - 1 comment

#1880 - Add GloRa Implementation

Pull Request - State: open - Opened by viliamvolosv 15 days ago - 3 comments

#1879 - Fix early import of torch extension in BOFT

Pull Request - State: closed - Opened by PhyscalX 15 days ago - 5 comments

#1878 - Add MoSLoRA

Issue - State: closed - Opened by wutaiqiang 15 days ago - 4 comments

#1877 - just import of peft leads to cuda context initialization

Issue - State: closed - Opened by PhyscalX 15 days ago - 1 comment

#1876 - Model with LoRA adapter errors when saving with `safe_serialization=True`

Issue - State: open - Opened by mliu-aqtc 15 days ago - 3 comments

#1875 - ENH: LoRA support for dynamically dispatching to custom layers

Pull Request - State: closed - Opened by BenjaminBossan 17 days ago - 1 comment

#1874 - FIX Error when using VeRA with float16 or bfloat16

Pull Request - State: closed - Opened by BenjaminBossan 17 days ago - 2 comments

#1873 - Tutorial Notebook: Using the PEFT library with a DNA Language Model.

Pull Request - State: open - Opened by rahulbshrestha 18 days ago - 6 comments

#1872 - VeRA doesn't work with lower-precision models

Issue - State: closed - Opened by 51616 18 days ago - 4 comments

#1871 - CI Downgrade numpy to <2.0 for Mac and Windows

Pull Request - State: closed - Opened by BenjaminBossan 18 days ago - 2 comments

#1870 - CI Use latest MacOS in test matrix

Pull Request - State: closed - Opened by BenjaminBossan 18 days ago - 2 comments

#1869 - Revert "FIX / Workflow: Fix Mac-OS CI issues"

Pull Request - State: closed - Opened by younesbelkada 18 days ago - 1 comment

#1868 - Workflow: Attempt to fix the current failures

Pull Request - State: closed - Opened by younesbelkada 19 days ago - 2 comments

#1867 - FEAT: LoRA: Possibility to dynamically register layers to dispatch

Issue - State: closed - Opened by BenjaminBossan 19 days ago

#1866 - CI testing BNB: remove single GPU tests

Pull Request - State: closed - Opened by BenjaminBossan 19 days ago - 1 comment

#1865 - Does PEFT support CodeBERT?

Issue - State: open - Opened by MabelQi 20 days ago - 5 comments

#1864 - support HRA

Pull Request - State: open - Opened by DaShenZi721 20 days ago - 6 comments

#1863 - Geometric Parametrization (GmP) [CLIP] - is it compatible with PEFT?

Issue - State: open - Opened by zer0int 20 days ago - 1 comment

#1862 - FIX multitask prompt tuning paper link

Pull Request - State: closed - Opened by cep-ter 20 days ago - 1 comment

#1861 - Update lora_based_methods.md

Pull Request - State: closed - Opened by jtatman 21 days ago - 1 comment

#1860 - Do we have to delete the PiSSA adapter after save_pissa_as_lora

Issue - State: open - Opened by hiyouga 21 days ago - 4 comments

#1859 - CI Testing: Remove import check

Pull Request - State: closed - Opened by BenjaminBossan 22 days ago - 1 comment

#1857 - ephemeral GPU offload support

Pull Request - State: closed - Opened by kallewoof 23 days ago - 16 comments

#1856 - DOC Move helpers section to dev developer guide

Pull Request - State: closed - Opened by BenjaminBossan 23 days ago - 1 comment

#1855 - DOC FIX Comment about init of LoRA Embedding

Pull Request - State: closed - Opened by BenjaminBossan 23 days ago - 1 comment

#1854 - Workflow / FIX: Fix red status on our CI

Pull Request - State: closed - Opened by younesbelkada 23 days ago - 2 comments

#1853 - Is merge_and_unload() an awq model and its peft qlora adapter available now?

Issue - State: closed - Opened by cywuuuu 23 days ago - 3 comments

#1852 - FIX: Adalora ranknum loaded on wrong device

Pull Request - State: closed - Opened by BenjaminBossan 24 days ago - 1 comment

#1851 - MNT Upgrade ruff version to ~0.4.8

Pull Request - State: closed - Opened by BenjaminBossan 24 days ago - 1 comment

#1850 - Fixed PeftMixedModel docstring example #1824

Pull Request - State: closed - Opened by namanvats 24 days ago - 5 comments

#1849 - Bug/style mixed model

Pull Request - State: closed - Opened by namanvats 24 days ago

#1848 - Fix PeftMixedModel docstring #1824 #1

Pull Request - State: closed - Opened by namanvats 24 days ago - 4 comments

#1847 - working on update old code to ne peft tech

Pull Request - State: closed - Opened by viliamvolosv 24 days ago

#1846 - `Conv1D` not supported using AdaLora

Issue - State: closed - Opened by bukun46 25 days ago - 5 comments

#1845 - CI Activate env to prevent bnb import error

Pull Request - State: closed - Opened by BenjaminBossan 25 days ago - 1 comment

#1844 - Fail to use zero_init to construct llama2 with deepspeed zero3 and qlora!

Issue - State: open - Opened by CHNRyan 25 days ago - 6 comments

#1843 - DOC Describe torch_device argument in from_pretrained docstring

Pull Request - State: closed - Opened by BenjaminBossan 26 days ago - 1 comment

#1842 - Workflow: Fix broken messages

Pull Request - State: closed - Opened by younesbelkada 26 days ago - 1 comment

#1841 - feat(ci): add trufflehog secrets detection

Pull Request - State: closed - Opened by McPatate 26 days ago - 4 comments

#1840 - device_map not work

Issue - State: closed - Opened by bxrjmfh 26 days ago - 2 comments

#1839 - question about bits_pattern feature

Issue - State: open - Opened by harrysyz99 26 days ago - 1 comment

#1838 - FourierFT Support

Pull Request - State: open - Opened by Phoveran 26 days ago - 24 comments

#1837 - Tutorial notebook for applying PEFT with DNA Language models

Issue - State: open - Opened by rahulbshrestha 27 days ago - 1 comment

#1836 - Different results when merging LORA weights into the base model vs. when not

Issue - State: closed - Opened by ankur6ue 27 days ago - 9 comments

#1835 - Glora implementation from https://github.com/Arnav0400/ViT-Slim/tree/master/GLoRA

Pull Request - State: closed - Opened by viliamvolosv 28 days ago - 15 comments

#1834 - A Question about new PEFT methods on FSDP

Issue - State: closed - Opened by Abhrant 29 days ago - 4 comments

#1833 - Fix a documentation typo

Pull Request - State: closed - Opened by sparsh2 30 days ago - 2 comments

#1832 - FIX Failing Llama tests due to new kv cache

Pull Request - State: closed - Opened by BenjaminBossan 30 days ago - 1 comment

#1831 - RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat2 in method wrapper_CUDA_mm)

Issue - State: closed - Opened by kevalshah90 about 1 month ago - 9 comments

#1830 - Workflow / Bnb: Add a mechanism to inform us if the import fails

Pull Request - State: closed - Opened by younesbelkada about 1 month ago - 11 comments

#1829 - How to change to PEFT model dynamically?

Issue - State: closed - Opened by whr819987540 about 1 month ago - 4 comments

#1828 - Adding support for an optional initialization strategy OLoRA

Pull Request - State: closed - Opened by tokenizer-decode about 1 month ago - 23 comments

#1827 - Some issues with attributes on custom models

Issue - State: closed - Opened by BenjaminBossan about 1 month ago - 1 comment

#1826 - [BugReport] init_lora_weights with pissa is not compatible with deepspeed stage3

Issue - State: open - Opened by wsp317 about 1 month ago - 1 comment

#1825 - I met a problem when I use torch=2.0.0, I can't use loftq config in loraConfig

Issue - State: closed - Opened by LiManyuan663 about 1 month ago - 13 comments

#1824 - Fix PeftMixedModel example

Issue - State: closed - Opened by ringohoffman about 1 month ago - 2 comments

#1823 - ENH / Workflow: Use repository variable

Pull Request - State: closed - Opened by younesbelkada about 1 month ago - 1 comment

#1822 - FIX CI: Install pytest-reportlog package

Pull Request - State: closed - Opened by BenjaminBossan about 1 month ago - 1 comment

#1821 - ENH / Workflow: Notify on slack about peft + transformers main test results

Pull Request - State: closed - Opened by younesbelkada about 1 month ago - 1 comment

#1820 - FIX CI: Remove potentially problematic git command

Pull Request - State: closed - Opened by BenjaminBossan about 1 month ago - 1 comment

#1819 - Update peft_integrations.md

Pull Request - State: closed - Opened by rodrigomeireles about 1 month ago - 1 comment

#1817 - Support for different layer shapes for VeRA

Pull Request - State: closed - Opened by dkopi about 1 month ago - 11 comments

#1816 - Support for different layer shapes for VeRA

Issue - State: closed - Opened by dkopi about 1 month ago

#1815 - QLoRA perfomance degradation after loading with PeftModel.from_pretrained

Issue - State: closed - Opened by vanerk03 about 1 month ago - 4 comments

#1814 - Lora Training Tab

Issue - State: open - Opened by ClericerQ about 1 month ago - 5 comments

#1813 - MoRA support

Pull Request - State: open - Opened by winglian about 1 month ago - 4 comments

#1812 - Correctly Uploading PEFT/QLoRA model to HuggingFace

Issue - State: open - Opened by taoofstefan about 1 month ago - 9 comments

#1811 - MNT Remove deprecated use of load_in_8bit

Pull Request - State: closed - Opened by BenjaminBossan about 1 month ago - 1 comment

#1810 - TESTING CI DON'T MERGE

Pull Request - State: closed - Opened by BenjaminBossan about 1 month ago - 1 comment

#1808 - CI Make torch compile tests run on GPU

Pull Request - State: closed - Opened by BenjaminBossan about 1 month ago - 2 comments

#1807 - Integrating Riemannian Preconditioner

Pull Request - State: open - Opened by fangzhaozhang about 1 month ago - 3 comments

#1806 - Refactor to make DoRA and QDoRA work with FSDP

Pull Request - State: closed - Opened by BenjaminBossan about 1 month ago - 1 comment

#1805 - Docs / LoRA: Add more information on `merge_and_unload` docs

Pull Request - State: closed - Opened by younesbelkada about 1 month ago - 1 comment

#1803 - Add Special Optimizer for LoRA training

Issue - State: open - Opened by fangzhaozhang about 1 month ago - 3 comments

#1802 - Issues when switching between multiple adapters LoRAs

Issue - State: open - Opened by JhonDan1999 about 1 month ago - 7 comments

GitHub / huggingface/peft issues and pull requests