bitsandbytes-foundation/bitsandbytes issues and pull requests

#1367 - Enable packaging for ROCm 6.2

Pull Request - State: open - Opened by pnunna93 about 17 hours ago

#1366 - cpu benchmark

Pull Request - State: open - Opened by jiqing-feng 1 day ago - 1 comment

#1365 - Change 8bit optimizer blocksize 2048->256; additional bf16 support

Pull Request - State: closed - Opened by matthewdouglas 3 days ago - 2 comments
Labels: enhancement

#1363 - RuntimeError: CUDA error: CUBLAS_STATUS_NOT_SUPPORTED when calling cublasGemmEx

Issue - State: open - Opened by LukeLIN-web 5 days ago - 1 comment

#1362 - Bump the minor-patch group with 3 updates

Pull Request - State: closed - Opened by dependabot[bot] 5 days ago
Labels: dependencies

#1361 - Update matplotlib requirement from ~=3.9.1 to ~=3.9.2 in the major group

Pull Request - State: closed - Opened by dependabot[bot] 5 days ago
Labels: dependencies

#1360 - Add AdEMAMix optimizer

Pull Request - State: closed - Opened by matthewdouglas 5 days ago - 1 comment
Labels: enhancement

#1359 - Merge LoRA into 405B

Issue - State: open - Opened by junzhang-zj 5 days ago - 4 comments

#1358 - check grad before using ipex

Pull Request - State: closed - Opened by jiqing-feng 7 days ago - 1 comment

#1357 - Nf4 grad

Pull Request - State: closed - Opened by jiqing-feng 7 days ago

#1356 - Lion Optimizer With Triton Kernel

Issue - State: closed - Opened by lapp0 8 days ago - 1 comment

#1355 - cuda is available but import bnb error

Issue - State: open - Opened by ZeroneBo 8 days ago - 2 comments

#1354 - Model not able to quantize

Issue - State: open - Opened by alielfilali01 9 days ago

#1353 - add CPU benchmark

Pull Request - State: closed - Opened by jiqing-feng 9 days ago - 1 comment

#1352 - docs: add internal reference to multi-backend guide

Pull Request - State: closed - Opened by Titus-von-Koeller 10 days ago - 1 comment

#1351 - docs: add internal reference to multi-backend guide

Pull Request - State: closed - Opened by Titus-von-Koeller 10 days ago - 1 comment

#1350 - Bug when using optimizer LAMB 32bits

Issue - State: open - Opened by FrsECM 16 days ago
Labels: bug

#1349 - fix nf4 memory issue by init op_context in forward

Pull Request - State: closed - Opened by jiqing-feng 16 days ago - 3 comments

#1348 - Nf4 reload

Pull Request - State: closed - Opened by jiqing-feng 16 days ago - 1 comment

#1347 - Torch autograd support for dequantize methods

Issue - State: open - Opened by yaldashbz 17 days ago

#1346 - Bump the minor-patch group with 2 updates

Pull Request - State: closed - Opened by dependabot[bot] 19 days ago - 2 comments
Labels: dependencies

#1345 - Update matplotlib requirement from ~=3.9.1 to ~=3.9.2 in the major group

Pull Request - State: closed - Opened by dependabot[bot] 19 days ago - 2 comments
Labels: dependencies

#1344 - Add `move_to_device` kwarg to the optimizer's `load_state_dict`

Pull Request - State: closed - Opened by koute 21 days ago - 1 comment

#1343 - Cannot load decoder.lm_head.weight when loading 4 bit quantized model using VisionEncoderDecoder.from_pretrained

Issue - State: open - Opened by AditiJain14 22 days ago - 1 comment

#1342 - quantize_4bit/dequantize_4bit gives wrong output on in-contiguous tensor

Issue - State: open - Opened by chenqianfzh 22 days ago

#1341 - Update for VS2022 17.11 compatibility with CUDA < 12.4

Pull Request - State: closed - Opened by matthewdouglas 23 days ago
Labels: Windows, build, cicd

#1336 - rm warn for multi backend

Pull Request - State: closed - Opened by jiqing-feng 25 days ago

#1335 - Bump the minor-patch group with 2 updates

Pull Request - State: closed - Opened by dependabot[bot] 26 days ago - 1 comment
Labels: dependencies

#1334 - Update matplotlib requirement from ~=3.9.1 to ~=3.9.2 in the major group

Pull Request - State: closed - Opened by dependabot[bot] 26 days ago - 1 comment
Labels: dependencies

#1333 - Update diagnostic functions for ROCm

Pull Request - State: closed - Opened by pnunna93 27 days ago - 5 comments

#1332 - Linear8bitLt can not be moved back to cpu

Issue - State: open - Opened by Nerogar 28 days ago
Labels: bug

#1331 - Pretrained Causal LM cannot be loaded in 4bit/8bit

Issue - State: open - Opened by adrienchaton 29 days ago - 6 comments
Labels: likely not a BNB issue

#1330 - Enable certain CUDA kernels to accept specified cuda stream

Pull Request - State: closed - Opened by jeejeelee about 1 month ago - 8 comments

#1329 - Any plan to support block size 32?

Issue - State: open - Opened by lllyasviel about 1 month ago - 4 comments
Labels: enhancement

#1328 - Cuda source cleanup , refactor and fixes

Pull Request - State: closed - Opened by abhilash1910 about 1 month ago - 4 comments

#1327 - RuntimeError: Failed to import transformers.integrations.bitsandbytes because of the following error (look up to see its traceback):

Issue - State: open - Opened by pradeep10kumar about 1 month ago - 1 comment

#1326 - MPS progress

Pull Request - State: open - Opened by tcdent about 1 month ago - 2 comments

#1325 - fix 4bit dtype

Pull Request - State: closed - Opened by jiqing-feng about 1 month ago - 1 comment

#1324 - Error occurred when executing DownloadAndLoadFlorence2Model:

Issue - State: closed - Opened by JunpeakChen about 1 month ago - 1 comment

#1323 - Error while trying to install the multi-backend-refactor branch for rocm in WSL2

Issue - State: closed - Opened by Kademo15 about 1 month ago - 1 comment

#1322 - RuntimeError: CUDA Setup failed despite GPU being available. Please run the following command to get more information:

Issue - State: open - Opened by pradeep10kumar about 1 month ago - 1 comment

#1321 - 'nf4' compute datatype?

Issue - State: open - Opened by dorsa-zeinali about 1 month ago - 1 comment
Labels: question

#1320 - where are the outliers stored in LLM.int8 quantization for inference suing transformers library on AMD GPU?

Issue - State: open - Opened by vbayanag about 1 month ago - 1 comment

#1319 - About fusion of kdequantize kernel and simple bf16/fp16 matmul

Issue - State: open - Opened by Ther-nullptr about 1 month ago - 1 comment

#1318 - Bugfix: Load correct nocublaslt library variant when BNB_CUDA_VERSION override is set

Pull Request - State: closed - Opened by matthewdouglas about 1 month ago - 1 comment
Labels: bug, CUDA Setup

#1317 - Communicate blocksize constraints to kernels that take blocksize as a runtime argument

Issue - State: open - Opened by mm04926412 about 1 month ago - 6 comments
Labels: enhancement, Low Risk

#1316 - Initial support for ppc64le

Pull Request - State: closed - Opened by mgiessing about 1 month ago - 3 comments
Labels: cross-platform

#1315 - Unable to override PyTorch CUDA Version

Issue - State: open - Opened by tinglvv about 1 month ago - 4 comments
Labels: CUDA Setup, waiting for info

#1314 - AdamE optimizer with decoupled L1 and L2 regularization

Pull Request - State: open - Opened by vincenzo-scotti about 1 month ago - 3 comments
Labels: enhancement

#1313 - libcudart.so Not Found

Issue - State: closed - Opened by arunsandy1309 about 1 month ago - 2 comments
Labels: Windows, waiting for info

#1312 - libbitsandbytes_cpu.so，libbitsandbytes_cuda124_nocublaslt124.so

Issue - State: closed - Opened by magicwang1111 about 1 month ago - 4 comments
Labels: waiting for info, build

#1311 - 4bit quantized model.dequantize() fails on CPU

Issue - State: open - Opened by npbool about 2 months ago

#1310 - Crash running FSDP on BF16-prequantized models

Issue - State: closed - Opened by dmitrii-palisaderesearch about 2 months ago - 4 comments
Labels: likely not a BNB issue, huggingface-related

#1309 - Runtime Error, cannot import name 'get_keys_to_not_convert' from 'transformers.integrations'

Issue - State: closed - Opened by zeruiz99 about 2 months ago - 1 comment

#1308 - dequantize_4bit() gives wrong output when working in cuda graph mode

Issue - State: closed - Opened by chenqianfzh about 2 months ago - 3 comments
Labels: bug, contributions-welcome

#1307 - RuntimeError: CUDA Setup failed despite GPU being available. Please run the following command to get more information: python -m bitsandbytes Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

Issue - State: open - Opened by senzawapoi about 2 months ago

#1306 - Regarding bnb import error

Issue - State: open - Opened by Mubashirshariq about 2 months ago - 2 comments
Labels: waiting for info

#1305 - Update wheel requirement from ~=0.43.0 to ~=0.44.0 in the minor-patch group

Pull Request - State: closed - Opened by dependabot[bot] about 2 months ago - 1 comment
Labels: dependencies

#1304 - bitsbytes 8bit quantized LLama 3.1 gets stuck sometimes when producing output

Issue - State: open - Opened by Techbhatia about 2 months ago

#1299 - Enable bitsandbytes packaging for ROCm

Pull Request - State: closed - Opened by pnunna93 about 2 months ago - 3 comments
Labels: AMD integration

#1294 - script to estimate qlora mem usage

Pull Request - State: closed - Opened by Titus-von-Koeller about 2 months ago - 2 comments

#1293 - FLUTE Integration for Fast Inference

Issue - State: open - Opened by HanGuo97 about 2 months ago - 11 comments
Labels: enhancement

#1292 - Embedding4bit and Embedding8bit implementation

Pull Request - State: closed - Opened by galqiwi about 2 months ago - 6 comments
Labels: enhancement

#1281 - NameError: name 'str2optimizer32bit' is not defined

Issue - State: open - Opened by qingqinggu 2 months ago - 3 comments

#1271 - ROCm Backend Status Tracker

Issue - State: open - Opened by pnunna93 2 months ago - 3 comments
Labels: AMD integration

#1270 - chore: delete useless buffered activation

Pull Request - State: closed - Opened by Ther-nullptr 3 months ago - 3 comments

#1257 - Add xpu support for 4bit (nf4/fp4) and 8bit quantization

Pull Request - State: open - Opened by zhuhong61 3 months ago - 2 comments
Labels: Intel Integration

#1235 - out kwarg in matmul_4bit() is not working

Issue - State: open - Opened by chenqianfzh 4 months ago - 3 comments
Labels: bug, contributions-welcome

#1232 - "Only Tensors of floating point and complex dtype can require gradients", on FSDP, Accelerate, quatization

Issue - State: closed - Opened by artkpv 4 months ago - 5 comments

#1223 - Add empty stubs for Ascend NPU

Pull Request - State: closed - Opened by statelesshz 4 months ago - 1 comment

#1220 - (WIP) Multi backend refactor -> main (full diff of all already merged PRs)

Pull Request - State: open - Opened by Titus-von-Koeller 4 months ago - 1 comment

#1210 - AttributeError: 'NoneType' object has no attribute 'cquantize_blockwise_fp16_nf4'

Issue - State: open - Opened by wissamee 4 months ago - 9 comments

#1199 - bitsandbytes import error in colab

Issue - State: open - Opened by ss8101 5 months ago - 2 comments

#1190 - CUDA Setup failed despite CUDA being Available :: NameError: name 'str2optimizer32bit' is not defined

Issue - State: open - Opened by NavinKumarMNK 5 months ago - 5 comments

#1166 - AttributeError: module 'bitsandbytes' has no attribute 'nn'

Issue - State: open - Opened by Harsh-raj 6 months ago - 5 comments

#1137 - Initial kernel changes to support GaLore

Pull Request - State: open - Opened by matthewdouglas 6 months ago - 3 comments
Labels: enhancement

#1103 - Remove separate NO_CUBLASLT build.

Pull Request - State: open - Opened by matthewdouglas 7 months ago - 6 comments
Labels: build

#1100 - ImportError: Using `bitsandbytes` 8-bit quantization requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes: `pip install -i https://pypi.org/simple/ bitsandbytes`

Issue - State: open - Opened by hrmgxuni 7 months ago - 16 comments

#1097 - 8 bit quantization and finetuning with lora is not working - receiving runtime error

Issue - State: open - Opened by solomonmanuelraj 7 months ago - 7 comments
Labels: waiting for info

#1093 - RuntimeError: Failed to import transformers.integrations.bitsandbytes because of the following error (look up to see its traceback):

Issue - State: closed - Opened by SumaiyaSultan2002 7 months ago - 11 comments

GitHub / bitsandbytes-foundation/bitsandbytes issues and pull requests