huggingface/peft issues and pull requests

#2035 - Cannot use prefix tuning on quantized Codellama

Issue - State: open - Opened by MabelQi about 1 month ago - 5 comments
Labels: bug, help wanted

#2034 - Bug: set correct pre-commit-hooks version

Pull Request - State: closed - Opened by ltoniazzi about 1 month ago - 1 comment

#2033 - FIX: Don't target the classification head when using target_modules="all-linear"

Pull Request - State: closed - Opened by BenjaminBossan about 1 month ago - 1 comment

#2032 - Unobvious behavior of model after applying merge_and_unload

Issue - State: closed - Opened by MezentsevIlya about 1 month ago - 3 comments

#2031 - [tests] enable cuda-only tests in `test_common_gpu.py` to work on XPU

Pull Request - State: closed - Opened by faaany about 1 month ago - 2 comments

#2030 - If you are interested in multimodal large models, you can join our group

Issue - State: closed - Opened by feihuamantian about 1 month ago

#2029 - Update layer.py

Pull Request - State: closed - Opened by DogitoErgoSum about 1 month ago - 7 comments

#2028 - ENH Raise error when applying modules_to_save on tuner layer

Pull Request - State: closed - Opened by BenjaminBossan about 1 month ago - 1 comment

#2027 - Unable to load peft checkpoint

Issue - State: closed - Opened by BigDataMLexplorer about 1 month ago - 7 comments

#2026 - [tests] make cuda-only cases in `TestModelAndLayerStatus` device-agnostic

Pull Request - State: closed - Opened by faaany about 1 month ago - 3 comments

#2025 - Warn if using tied target module with `tie_word_embeddings`

Pull Request - State: closed - Opened by ltoniazzi about 1 month ago - 16 comments

#2024 - CI Remove regression tests from BNB CI

Pull Request - State: closed - Opened by BenjaminBossan about 1 month ago - 1 comment

#2023 - CI Comment BNB multi-backend multi-GPU test

Pull Request - State: closed - Opened by BenjaminBossan about 1 month ago - 2 comments

#2022 - ENH: Better error msg for replace_lora_weights_loftq when using a local model.

Pull Request - State: closed - Opened by BenjaminBossan about 1 month ago - 1 comment

#2021 - [tests] enable `test_mixed_adapter_batches_lora_opt_timing` on XPU

Pull Request - State: closed - Opened by faaany about 1 month ago - 14 comments

#2020 - replace_lora_weights_loftq() throw HFValidationError when base model is loaded using a local path

Issue - State: closed - Opened by Anstinus about 1 month ago - 2 comments

#2019 - [tests] enable regression tests on XPU

Pull Request - State: closed - Opened by faaany about 1 month ago - 4 comments

#2018 - With tied embeddings adapter merged to tied layers

Issue - State: closed - Opened by ltoniazzi about 1 month ago - 5 comments

#2017 - [tests] enable `test_vera_dtypes` on XPU

Pull Request - State: closed - Opened by faaany about 1 month ago - 3 comments

#2016 - Cannot apply both PEFT QLoRA and DeepSpeed ZeRO3

Issue - State: open - Opened by echo-yi about 1 month ago - 9 comments

#2015 - XLoRA: training issues, Gradients will be None

Issue - State: open - Opened by benjamin-marie about 1 month ago - 9 comments

#2014 - QLora with DeepSpeed support

Issue - State: closed - Opened by ysj1173886760 about 1 month ago - 5 comments

#2013 - Add MoSLoRA

Pull Request - State: closed - Opened by wutaiqiang about 2 months ago - 11 comments

#2012 - Incorporating contrastive prefixes with PrefixTuning

Issue - State: closed - Opened by AvantiB about 2 months ago - 2 comments

#2011 - FIX: Error with OLoRA init when using bnb

Pull Request - State: closed - Opened by BenjaminBossan about 2 months ago - 15 comments

#2010 - Fix usage of deprecated parameters/functions in X-LoRA

Pull Request - State: closed - Opened by EricLBuehler about 2 months ago - 6 comments

#2009 - Missing PEFT model

Issue - State: closed - Opened by LAJ-THU about 2 months ago - 3 comments

#2008 - CI Activate single core multi backend bnb tests

Pull Request - State: closed - Opened by BenjaminBossan about 2 months ago - 1 comment

#2007 - FIX CI Correctly report outcome of bnb import test

Pull Request - State: closed - Opened by BenjaminBossan about 2 months ago - 2 comments

#2006 - [Add] DoRA Embedding

Pull Request - State: closed - Opened by ariG23498 about 2 months ago - 24 comments

#2005 - TST Add LNTuningConfig and LoKrConfig to tests

Pull Request - State: closed - Opened by BenjaminBossan about 2 months ago - 1 comment

#2004 - ENH: Warn when a user provided model name in the config renamed

Pull Request - State: closed - Opened by BenjaminBossan about 2 months ago - 2 comments

#2003 - Update lora.md (typos)

Pull Request - State: closed - Opened by nir-sh-automat-it about 2 months ago - 1 comment

#2002 - Update docs for X-LoRA and some bugfixes

Pull Request - State: closed - Opened by EricLBuehler about 2 months ago - 8 comments

#2001 - Issue with PEFT model save_pretrained

Issue - State: closed - Opened by Quicksilver466 about 2 months ago - 5 comments

#2000 - [FEAT] Add support for optimum-quanto

Pull Request - State: open - Opened by BenjaminBossan about 2 months ago - 3 comments
Labels: wip

#1999 - Lora initialisation with olora and pissa not working with quantisation.

Issue - State: closed - Opened by Haakooto about 2 months ago - 9 comments
Labels: bug

#1998 - TST: Potentially Skip 8bit bnb regression test if compute capability is too low

Pull Request - State: closed - Opened by BenjaminBossan about 2 months ago - 1 comment

#1997 - Support optimum-quanto

Issue - State: open - Opened by BenjaminBossan about 2 months ago - 13 comments
Labels: contributions-welcome

#1996 - Update OFT to fix merge bugs

Pull Request - State: open - Opened by Zeju1997 about 2 months ago - 38 comments

#1995 - FIX Import error in BOFT half precision test

Pull Request - State: closed - Opened by BenjaminBossan about 2 months ago - 3 comments

#1994 - BOFT bug fix when saving

Pull Request - State: closed - Opened by Zeju1997 about 2 months ago - 9 comments

#1993 - MAINT: Default to loading weights only for torch.load

Pull Request - State: closed - Opened by BenjaminBossan about 2 months ago - 2 comments

#1992 - FIX: Adjust transformers version check for bloom

Pull Request - State: closed - Opened by BenjaminBossan about 2 months ago - 1 comment

#1991 - Linear4bit not supported error when running QDoRA_finetuning.ipynb cookbook

Issue - State: closed - Opened by hmehdi515 about 2 months ago - 1 comment

#1990 - fixing multiple LoRA in the same batch or vit

Pull Request - State: closed - Opened by saeid93 about 2 months ago - 16 comments

#1989 - [Suggestions] Updates suggested for `helper.rescale_adapter_scale`

Pull Request - State: closed - Opened by ariG23498 about 2 months ago - 11 comments

#1988 - TypeError: WhisperForConditionalGeneration.forward() got an unexpected keyword argument 'input_ids'

Issue - State: closed - Opened by YOUSEFNANIS 2 months ago - 4 comments

#1987 - fix: fix docker build gpus

Pull Request - State: closed - Opened by XciD 2 months ago - 1 comment

#1986 - DOC Small fixes for HQQ and section title

Pull Request - State: closed - Opened by BenjaminBossan 2 months ago - 1 comment

#1985 - chore: Update CI configuration for workflows

Pull Request - State: closed - Opened by XciD 2 months ago - 1 comment

#1984 - DoRA with fsdp model cannot work

Issue - State: closed - Opened by Wongboo 2 months ago - 15 comments

#1983 - Gumbel-sigmoid (poly)

Issue - State: closed - Opened by adws2 2 months ago - 3 comments

#1982 - LoftQ init without scaling

Issue - State: open - Opened by cobbswordsman 2 months ago - 3 comments

#1981 - AdaLora: Trigger warning when user uses 'r' inplace of 'init_r'

Pull Request - State: closed - Opened by bhargavyagnik 2 months ago - 6 comments

#1980 - RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

Issue - State: closed - Opened by jjzhu0579 2 months ago - 2 comments

#1979 - LoraConfig.init() got an unexpected keyword argument 'layer_replication' when using custom model handler

Issue - State: closed - Opened by topher 2 months ago - 2 comments

#1978 - Cast to fp32 if using bf16 weights on cpu during `merge_and_unload`

Pull Request - State: closed - Opened by snarayan21 2 months ago - 10 comments

#1977 - Merging adapters for BF16 model on CPU is very slow

Issue - State: closed - Opened by snarayan21 2 months ago - 1 comment

#1976 - FIX Loading adapter honors offline mode

Pull Request - State: closed - Opened by BenjaminBossan 2 months ago - 6 comments

#1975 - FIX active_adapters for transformers models

Pull Request - State: closed - Opened by BenjaminBossan 2 months ago - 1 comment

#1974 - AttributeError: 'NoneType' object has no attribute 'shape'

Issue - State: closed - Opened by jjzhu0579 2 months ago - 16 comments

#1973 - fix: lora+: include lr in optimizer kwargs

Pull Request - State: closed - Opened by kallewoof 2 months ago - 3 comments

#1972 - PEFT QLoRA with DeepSpeed ZeRO3 on multiple GPUs not working as in documentation

Issue - State: closed - Opened by echo-yi 2 months ago - 6 comments

#1971 - hyper-paramter `r` in adalora_confg and lora_config

Issue - State: closed - Opened by Vincent950129 2 months ago - 1 comment
Labels: good first issue, contributions-welcome

#1970 - Add docs and examples for X-LoRA

Pull Request - State: closed - Opened by EricLBuehler 2 months ago - 8 comments

#1969 - FIX: New bloom changes breaking prompt learning

Pull Request - State: closed - Opened by BenjaminBossan 2 months ago - 3 comments

#1967 - MobileViT does not work with Inference with different LoRA adapters in the same batch

Issue - State: open - Opened by saeid93 2 months ago - 7 comments

#1966 - Documentation and optimization of X-LoRA

Issue - State: closed - Opened by EricLBuehler 2 months ago - 4 comments

#1965 - MAINT: Update ruff version to ~0.6.1

Pull Request - State: closed - Opened by BenjaminBossan 2 months ago - 2 comments

#1964 - Changing lora_alpha during inference

Issue - State: closed - Opened by SoumiDas 2 months ago - 5 comments

#1963 - Training with Multiple LoRAs

Issue - State: closed - Opened by Jyonn 2 months ago - 5 comments

#1962 - Deprecation: Transformers will no longer support `past_key_values` to be tuples

Issue - State: open - Opened by BenjaminBossan 2 months ago - 13 comments
Labels: help wanted, contributions-welcome

#1961 - ENH: Allow empty initialization of adapter weight

Pull Request - State: closed - Opened by BenjaminBossan 2 months ago - 28 comments
Labels: contributions-welcome

#1960 - Inference with different LoRA adapters in the same batch does not use the correct module_to_save classifier

Issue - State: open - Opened by saeid93 2 months ago - 8 comments
Labels: contributions-welcome

#1959 - Incorrect target modules does not error

Issue - State: closed - Opened by irenedea 2 months ago - 2 comments

#1957 - Peng add pclora

Pull Request - State: open - Opened by pzdkn 2 months ago - 1 comment

#1955 - DON'T MERGE: Dummy PR to trigger CI w/ PyTorch 2.4

Pull Request - State: closed - Opened by BenjaminBossan 2 months ago - 2 comments

#1954 - [VeRA] doesn't support different dim when using shared buffer

Issue - State: closed - Opened by AaronZLT 2 months ago - 4 comments

#1953 - 拉了一个多模态大模型技术交流群，大家可以加入进来进行技术交流

Issue - State: closed - Opened by feihuamantian 2 months ago - 1 comment

#1952 - CI Fix Windows permission error on merge test

Pull Request - State: closed - Opened by BenjaminBossan 2 months ago - 1 comment

#1951 - [Add] scaling LoRA adapter weights with a context manager

Pull Request - State: closed - Opened by ariG23498 2 months ago - 13 comments

#1950 - Bump version to 0.12.1.dev0

Pull Request - State: closed - Opened by BenjaminBossan 2 months ago - 2 comments

#1949 - Optimal hyperparameter for fine tuning LLM

Issue - State: closed - Opened by BigDataMLexplorer 2 months ago

#1948 - Optimal hyperparameter for fine tuning LLM

Issue - State: closed - Opened by BigDataMLexplorer 2 months ago - 8 comments

#1947 - CI Only run a single Windows workflow

Pull Request - State: closed - Opened by BenjaminBossan 2 months ago - 1 comment

#1946 - Release v0.12.0

Pull Request - State: closed - Opened by BenjaminBossan 2 months ago - 1 comment

#1944 - Decrease memory usage of `merge_and_unload`

Pull Request - State: closed - Opened by snarayan21 2 months ago - 8 comments

#1943 - AttributeError: 'LlamaForCausalLM' object has no attribute 'add_weighted_adapter'

Issue - State: closed - Opened by mingkangz 2 months ago - 4 comments

#1942 - Check if past_key_values is provided when using prefix_tuning in peft_model

Pull Request - State: closed - Opened by Nidhogg-lyz 2 months ago - 8 comments

#1941 - ENH Update VeRA preconfigured models

Pull Request - State: closed - Opened by BenjaminBossan 2 months ago - 3 comments

#1940 - Improving generalization of LoRA with wise-ft

Issue - State: closed - Opened by SimJeg 2 months ago - 22 comments
Labels: PRs welcome to address this, contributions-welcome, good-second-pr

#1939 - Memory usage spike when using `merge_and_unload`

Issue - State: closed - Opened by snarayan21 2 months ago - 7 comments

#1938 - Check if `past_key_values` is provided when using prefix_tuning in peft_model

Issue - State: open - Opened by Nidhogg-lyz 2 months ago

#1937 - Error on finetuning mistral-nemo-2407 model with peft LoRA

Issue - State: open - Opened by YeonwooSung 2 months ago - 1 comment

#1935 - [Call for contributions] help us improve LoKr, LoHa, and other LyCORIS

Issue - State: open - Opened by sayakpaul 3 months ago - 30 comments
Labels: contributions-welcome, good-second-pr

#1934 - load a model with unmerged lora adapters

Issue - State: closed - Opened by Ledzy 3 months ago - 4 comments

#1933 - PiSSA, OLoRA: Delete initial adapter after conversion instead of the active adapter

Pull Request - State: closed - Opened by BenjaminBossan 3 months ago - 7 comments

#1932 - `load_adapter` seems to require the base model to be identical

Issue - State: closed - Opened by zyzhang1130 3 months ago - 12 comments
Labels: contributions-welcome

#1931 - PEFT implementations of adapters are outdated and languishing

Issue - State: closed - Opened by bghira 3 months ago - 10 comments

GitHub / huggingface/peft issues and pull requests