PygmalionAI/aphrodite-engine issues and pull requests

#799 - ci: bump version to 0.6.3

Pull Request - State: closed - Opened by AlpinDale 10 days ago

#798 - feat: add TP support for bitsandbytes

Pull Request - State: open - Opened by AlpinDale 10 days ago

#797 - fix: kobold lite embedded UI on windows

Pull Request - State: closed - Opened by AlpinDale 10 days ago

#796 - build(deps): bump rollup from 4.21.0 to 4.24.3 in /docs

Pull Request - State: open - Opened by dependabot[bot] 11 days ago
Labels: dependencies

#795 - feat: add HQQ quantization support

Pull Request - State: closed - Opened by AlpinDale 11 days ago

#794 - fix: windows wheel url

Pull Request - State: closed - Opened by AlpinDale 11 days ago

#793 - [Usage]: Distributed Inference Without Docker.

Issue - State: open - Opened by Abdulhanan535 13 days ago - 3 comments

#792 - [New Method]: VPTQ, Vector Post-Training Quantization

Issue - State: open - Opened by YangWang92 14 days ago - 2 comments

#791 - [Installation]: Unable to make openvino / CPU install from source work: "Failed to import from aphrodite._C with No module named 'aphrodite._C'"

Issue - State: open - Opened by bolaft 15 days ago

#790 - feat: windows support

Pull Request - State: closed - Opened by AlpinDale 15 days ago - 8 comments

#789 - [Bug]: unable to load 14B Qwen2.5 GGUF with newest version (0.6.2.post1)

Issue - State: open - Opened by NeoChen1024 20 days ago - 1 comment
Labels: bug

#788 - [Bug]: strange repetition issue

Issue - State: open - Opened by ehartford 20 days ago - 5 comments
Labels: bug

#787 - frontend: minor logging improvements

Pull Request - State: closed - Opened by AlpinDale 22 days ago

#786 - [Bug]: Several errors when deploying GGUF models

Issue - State: open - Opened by musoles 22 days ago
Labels: bug

#785 - Stream models rather than load them completely into RAM.

Pull Request - State: closed - Opened by 50h100a 23 days ago - 2 comments

#784 - [Installation]: FYI: they fixed the stupid conda pytorch-cuda=12.4 / cuda 12.4.1 strict dependency issue

Issue - State: open - Opened by BlairSadewitz 24 days ago

#783 - [Bug]: Impossible dependency requirement with GGUF

Issue - State: open - Opened by musoles 24 days ago
Labels: bug

#782 - [Bug]: Metrics incorrect when having zero throughput

Issue - State: open - Opened by mrseeker 25 days ago
Labels: bug

#781 - [Bug]: Llama3 VocabParallelEmbedding error when loading

Issue - State: open - Opened by gelim 25 days ago
Labels: bug

#780 - [Installation]: I tried compile GFX1100 on WSL2 but it does not seems work

Issue - State: open - Opened by sorasoras 27 days ago - 13 comments

#779 - ci: bump version to 0.6.2.post1

Pull Request - State: closed - Opened by AlpinDale 27 days ago

#778 - fix: demote skip_special_tokens assertion to logger error

Pull Request - State: closed - Opened by AlpinDale 27 days ago

#777 - docker: apply AMD patch in the dockerfile

Pull Request - State: closed - Opened by AlpinDale 27 days ago

#776 - feat: ministral support

Pull Request - State: closed - Opened by AlpinDale 27 days ago

#775 - Make amd usable

Pull Request - State: closed - Opened by Naomiusearch 29 days ago - 1 comment

#774 - [Installation]: AMD MI60 (gfx906) installation errors with ROCm 6.1 and 6.2

Issue - State: open - Opened by Said-Akbar about 1 month ago - 10 comments

#773 - [Bug]: KeyError during loading of Mixtral 8x22B in FP8

Issue - State: open - Opened by IowaSovereign about 1 month ago
Labels: bug

#772 - Add OLMoE

Pull Request - State: closed - Opened by fizzAI about 1 month ago - 5 comments

#771 - [Bug]: KAIGenerationInputSchema has no attribute 'get'

Issue - State: closed - Opened by Luke100000 about 1 month ago - 2 comments
Labels: bug

#770 - Modified throughput benchmark to allow --max-num-seqs

Pull Request - State: closed - Opened by Pyroserenus about 1 month ago

#769 - [IMPORTANT] updating test units

Pull Request - State: open - Opened by AlpinDale about 1 month ago

#768 - feat: add shrek sampler (entropy)

Pull Request - State: open - Opened by AlpinDale about 1 month ago - 2 comments

#767 - [Feature]: tensor parallelism support for bnb quantization (via IBM's fork)

Issue - State: open - Opened by BlairSadewitz about 2 months ago - 3 comments

#766 - Simplify construction of sampling_metadata

Pull Request - State: closed - Opened by 50h100a about 2 months ago

#765 - [Bug]: Tekken tokenizer fails to load

Issue - State: closed - Opened by iamsuperdupercool about 2 months ago - 8 comments
Labels: bug

#764 - Fix for a crash from token bans

Pull Request - State: closed - Opened by Pyroserenus about 2 months ago - 2 comments

#763 - fix: kobold api for horde

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#762 - [Bug]: Banned EOS_TOKEN still stopping generation

Issue - State: open - Opened by mrseeker about 2 months ago - 3 comments
Labels: bug

#761 - [Feature]: xtc sampling support for kai api

Issue - State: open - Opened by BlairSadewitz about 2 months ago - 4 comments

#760 - chore: update klite.embd

Pull Request - State: closed - Opened by eltociear about 2 months ago - 1 comment

#759 - [Feature]: Support for tpu v3-8

Issue - State: open - Opened by Abdulhanan535 about 2 months ago

#758 - ci: bump version to 0.6.2

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#757 - docs: update readme and quant docs

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#756 - fix: add pandas to requirements

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#755 - feat: quant_llm support

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#754 - feat: bring back dynatemp

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#753 - [Bug]: Error from vllm when trying to load a quant model from docker

Issue - State: closed - Opened by puppetm4st3r about 2 months ago - 4 comments
Labels: bug

#752 - [Bug]: latest tag for docker pull did not work

Issue - State: closed - Opened by puppetm4st3r about 2 months ago - 2 comments
Labels: bug

#751 - chore: re-enable custom token bans

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#750 - ci: fix docs deployment

Pull Request - State: closed - Opened by ahme-dev about 2 months ago

#749 - ci: fix dep install using pnpm

Pull Request - State: closed - Opened by ahme-dev about 2 months ago

#748 - chore: refactor llama3 rope

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#747 - fix: metrics endpoint with RPC server

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#746 - chore: various TPU fixes and optimizations

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#745 - chore: refactor `MultiModalConfig` initialization and profiling

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#744 - fix: minor bug fixes & clean-ups

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#743 - feat: add Exaone model support

Pull Request - State: closed - Opened by shing100 about 2 months ago

#742 - [Bug]: Stops are not working

Issue - State: closed - Opened by miku448 about 2 months ago - 2 comments
Labels: bug

#741 - Xtchmm

Pull Request - State: closed - Opened by 50h100a about 2 months ago

#740 - feat: add XTC Sampling

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#739 - fix: use nvml to get consistent device names

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#738 - fix: clear engine ref in RPC server

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#737 - fix: `custom_ar` check

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#736 - fix: types in AQLM and GGUF for dynamo support

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#735 - [Bug]: LORA is massively slower after triton punica kernel upgrade

Issue - State: open - Opened by Nero10578 about 2 months ago - 2 comments
Labels: bug

#734 - [Installation]: Failed to initialize NumPy: No module named 'numpy'

Issue - State: open - Opened by Nero10578 about 2 months ago - 2 comments

#733 - feat: dynamo support for ScalarType

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#732 - chore: register lora functions as torch ops

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#731 - chore: move update_flash_attn_metadata to attn backend

Pull Request - State: closed - Opened by AlpinDale about 2 months ago

#730 - feat: add experts_int8 support

Pull Request - State: closed - Opened by AlpinDale about 2 months ago