microsoft/VPTQ issues and pull requests

#124 - Update model_base.py

Pull Request - State: closed - Opened by YangWang92 6 days ago

#123 - fix setup version number

Pull Request - State: closed - Opened by wejoncy 6 days ago

#122 - fix version__

Pull Request - State: closed - Opened by wejoncy 6 days ago

#121 - Bump to 0.0.4

Pull Request - State: closed - Opened by wejoncy 7 days ago

#120 - fix config format for transformers

Pull Request - State: closed - Opened by wejoncy 7 days ago

#119 - When I use the parameter npercent=1 to quantize the model, I have the following problem：

Issue - State: closed - Opened by half-lang 9 days ago - 2 comments
Labels: question

#118 - Enhance the implementation of the CUDA inference kernel.

Issue - State: open - Opened by haruhi55 12 days ago
Labels: features

#117 - Update README.md

Pull Request - State: closed - Opened by YangWang92 13 days ago

#116 - VLM Support

Issue - State: closed - Opened by YangWang92 18 days ago - 1 comment
Labels: new models

#115 - Huggingface Transformer Support

Issue - State: open - Opened by YangWang92 18 days ago - 1 comment
Labels: inference

#114 - CPU support

Issue - State: open - Opened by YangWang92 18 days ago
Labels: inference

#113 - Custom Model support

Issue - State: open - Opened by huangtingwei9988 18 days ago - 3 comments
Labels: question, new models

#112 - Add CUDA_HOME instructions to README

Pull Request - State: closed - Opened by caronzh03 19 days ago

#111 - Docker image for development

Issue - State: closed - Opened by caronzh03 20 days ago - 6 comments
Labels: question

#110 - update algorithm

Pull Request - State: closed - Opened by YangWang92 24 days ago

#109 - Update README.md

Pull Request - State: closed - Opened by YangWang92 24 days ago

#108 - Update README.md

Pull Request - State: closed - Opened by YangWang92 24 days ago

#107 - Update README.md

Pull Request - State: closed - Opened by YangWang92 24 days ago

#106 - update pyproject

Pull Request - State: closed - Opened by YangWang92 24 days ago

#105 - update algorithm

Pull Request - State: closed - Opened by YangWang92 24 days ago

#104 - add package info

Pull Request - State: closed - Opened by YangWang92 24 days ago

#103 - Update vqlinear.py

Pull Request - State: closed - Opened by laomao0 25 days ago

#102 - Add evaluation codes

Issue - State: open - Opened by YangWang92 25 days ago
Labels: enhancement

#101 - index unpack problem

Issue - State: closed - Opened by laomao0 25 days ago - 2 comments

#100 - update version

Pull Request - State: closed - Opened by YangWang92 27 days ago

#99 - update version

Pull Request - State: closed - Opened by YangWang92 27 days ago

#98 - fix compiling error

Pull Request - State: closed - Opened by YangWang92 28 days ago

#97 - Set version

Pull Request - State: closed - Opened by YangWang92 28 days ago

#96 - fix format

Pull Request - State: closed - Opened by YangWang92 28 days ago

#95 - fix format

Pull Request - State: closed - Opened by YangWang92 28 days ago

#94 - Update README.md

Pull Request - State: closed - Opened by YangWang92 28 days ago

#93 - init algorithm

Pull Request - State: closed - Opened by YangWang92 28 days ago

#92 - update main

Pull Request - State: closed - Opened by YangWang92 28 days ago

#91 - Init quantization algorithm

Pull Request - State: closed - Opened by YangWang92 28 days ago

#90 - Use absolute imports

Pull Request - State: closed - Opened by bndos 28 days ago - 3 comments

#89 - CUDA Kernel Not Found: Fallback to Torch Implementation During vptq Execution in Local Setup

Issue - State: closed - Opened by bndos 28 days ago

#88 - Add FP8/INT8 support

Issue - State: open - Opened by YangWang92 30 days ago
Labels: alogrithm

#87 - about permutation and quant channel

Issue - State: closed - Opened by laomao0 about 1 month ago - 3 comments
Labels: question, alogrithm

#86 - Slow Inference on A2 with VPTQ Compared to Ollama

Issue - State: closed - Opened by WpythonW about 1 month ago - 5 comments
Labels: question, inference

#85 - add acknowledgement and disclaimer

Pull Request - State: closed - Opened by YangWang92 about 1 month ago

#84 - add math example

Pull Request - State: closed - Opened by YangWang92 about 1 month ago

#83 - Update README.md

Pull Request - State: closed - Opened by OpenSourceRonin about 1 month ago

#82 - Update README.md

Pull Request - State: closed - Opened by OpenSourceRonin about 1 month ago

#81 - Update README.md

Pull Request - State: closed - Opened by OpenSourceRonin about 1 month ago

#80 - Update README.md

Pull Request - State: closed - Opened by OpenSourceRonin about 1 month ago

#79 - Improve hessian/invhessian collection

Issue - State: open - Opened by YangWang92 about 1 month ago - 3 comments
Labels: alogrithm

#78 - New Quantized Model Request

Issue - State: closed - Opened by JoesSattes about 1 month ago - 3 comments
Labels: question, new models

#77 - Add Ollama/llama.cpp/ggml support

Issue - State: open - Opened by YangWang92 about 1 month ago
Labels: inference

#76 - Add VLM/Multimodality support

Issue - State: open - Opened by YangWang92 about 1 month ago
Labels: new models, alogrithm

#75 - Add vLLM support

Issue - State: open - Opened by YangWang92 about 1 month ago - 2 comments
Labels: inference

#74 - update device map

Pull Request - State: closed - Opened by YangWang92 about 1 month ago

#73 - Multiple GPU Support

Issue - State: closed - Opened by twoxfh about 1 month ago - 3 comments
Labels: bug, question

#72 - Update setup.py

Pull Request - State: closed - Opened by YangWang92 about 1 month ago

#71 - RuntimeError: un-supported index_bits:10

Issue - State: closed - Opened by Excuses123 about 1 month ago - 3 comments

#70 - Update README.md

Pull Request - State: closed - Opened by OpenSourceRonin about 1 month ago

#69 - refine online demo

Pull Request - State: closed - Opened by YangWang92 about 1 month ago

#68 - Update setup.py

Pull Request - State: closed - Opened by YangWang92 about 1 month ago

#67 - Is it possible to run VPTQ in CPU only?

Issue - State: closed - Opened by yueqianh about 1 month ago - 5 comments
Labels: question, inference

#66 - improve web app demo

Pull Request - State: closed - Opened by YangWang92 about 1 month ago

#65 - rocm fix

Pull Request - State: closed - Opened by wejoncy about 1 month ago

#64 - Is this a typo in the paper？

Issue - State: closed - Opened by FdyCN about 1 month ago - 2 comments

#63 - support rocm

Pull Request - State: closed - Opened by wejoncy about 1 month ago

#62 - may be some issuses about llama-2-7b model.

Issue - State: closed - Opened by laomao0 about 2 months ago - 5 comments

#61 - Revert "fix offload bug in accelerator"

Pull Request - State: closed - Opened by wejoncy about 2 months ago

#60 - fix offload bug in accelerator

Pull Request - State: closed - Opened by wejoncy about 2 months ago

#59 - 'centroids must be a CUDA tensor' error when running Qwen2.5-72B-Instruct 2-bit in RTX4090

Issue - State: closed - Opened by yueqianh about 2 months ago - 2 comments
Labels: bug, question

#58 - Delete models directory

Pull Request - State: closed - Opened by YangWang92 about 2 months ago

#57 - Possibility of quantising multimodal models like Qwen-VL and llama 3.2 Vision?

Issue - State: closed - Opened by yueqianh about 2 months ago - 4 comments
Labels: question, alogrithm

#56 - How can I use VPTQ to quantize my own models?

Issue - State: closed - Opened by IEI-mjx about 2 months ago - 15 comments
Labels: question, alogrithm

#55 - update installation

Pull Request - State: closed - Opened by wejoncy about 2 months ago

#54 - CUDA kernel not found, please check CUDA and VPTQ installation

Issue - State: closed - Opened by laomao0 about 2 months ago - 7 comments
Labels: question

#53 - Does not work in Oobabooga

Issue - State: open - Opened by Kaszebe about 2 months ago - 1 comment
Labels: question, inference

#52 - cc1plus: out of memory

Issue - State: closed - Opened by DietmarGrabowski about 2 months ago - 5 comments
Labels: question

#51 - support bf16

Pull Request - State: closed - Opened by wejoncy about 2 months ago

#50 - add gpu monitor at web app

Pull Request - State: closed - Opened by TITC about 2 months ago - 3 comments

#49 - add catlog and index for readme

Pull Request - State: closed - Opened by wejoncy about 2 months ago

#48 - add notebook

Pull Request - State: closed - Opened by YangWang92 about 2 months ago

#47 - docs: update README.md

Pull Request - State: closed - Opened by eltociear about 2 months ago - 1 comment

#46 - Update README.md

Pull Request - State: closed - Opened by OpenSourceRonin about 2 months ago

#45 - add sm_89

Pull Request - State: closed - Opened by wejoncy about 2 months ago

#44 - RuntimeError: un-supported dtype: bfloat16

Issue - State: closed - Opened by dillfrescott about 2 months ago - 16 comments
Labels: question, investigate

#43 - Update README.md

Pull Request - State: closed - Opened by YangWang92 about 2 months ago

#42 - stdout captures and injects userwarnings into TextStreamer

Issue - State: open - Opened by JoeHelbing about 2 months ago - 3 comments
Labels: question

#41 - (Eventually) submit to exllama?

Issue - State: closed - Opened by Downtown-Case about 2 months ago - 3 comments
Labels: question, inference

#40 - Update README.md

Pull Request - State: closed - Opened by OpenSourceRonin about 2 months ago

#39 - bump to 0.0.2

Pull Request - State: closed - Opened by wejoncy about 2 months ago

#38 - support cuda-arch_list

Pull Request - State: closed - Opened by wejoncy about 2 months ago

#37 - update readme and tech report

Pull Request - State: closed - Opened by YangWang92 about 2 months ago

#36 - add prompt args and check cuda kernel

Pull Request - State: closed - Opened by YangWang92 about 2 months ago

#35 - Update README.md

Pull Request - State: closed - Opened by OpenSourceRonin about 2 months ago

#34 - Update README.md

Pull Request - State: closed - Opened by OpenSourceRonin about 2 months ago

#33 - Update README.md

Pull Request - State: closed - Opened by YangWang92 about 2 months ago

#32 - Update README.md

Pull Request - State: closed - Opened by YangWang92 about 2 months ago

#31 - Update README.md

Pull Request - State: closed - Opened by OpenSourceRonin about 2 months ago

#30 - Update README.md

Pull Request - State: closed - Opened by YangWang92 about 2 months ago

#29 - Release date for quantization code?

Issue - State: closed - Opened by cakeng about 2 months ago - 13 comments
Labels: question, alogrithm

#28 - Update README.md

Pull Request - State: closed - Opened by OpenSourceRonin about 2 months ago

#27 - Patch 1

Pull Request - State: closed - Opened by OpenSourceRonin about 2 months ago

#26 - Llama-2-7b models

Issue - State: closed - Opened by laomao0 about 2 months ago - 5 comments
Labels: question, new models

#25 - Update README.md

Pull Request - State: closed - Opened by YangWang92 about 2 months ago

GitHub / microsoft/VPTQ issues and pull requests