Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / dvmazur/mixtral-offloading issues and pull requests

#39 - Hard to benchmark the operation in the repo

Issue - State: open - Opened by mynotwo 24 days ago - 1 comment

#38 - Mixtral Instruct tokenizer from Colab notebook doesn't work.

Issue - State: open - Opened by jmuntaner-smd 3 months ago - 2 comments

#37 - Change of query weight matrices shapes

Issue - State: closed - Opened by avani17101 4 months ago

#36 - Support DeepSeek V2 model

Issue - State: open - Opened by Minami-su 4 months ago

#35 - Having issue loading my HQQ quantized model

Issue - State: open - Opened by BeichenHuang 5 months ago

#32 - runtimeerror when nbit = 4 and group_size =64

Issue - State: open - Opened by Eutenacity 5 months ago

#31 - Trition Issues in Running the Code Locally

Issue - State: closed - Opened by amangupt01 5 months ago - 1 comment

#30 - Can this be used for Jambo inference

Issue - State: open - Opened by freQuensy23-coder 6 months ago - 1 comment

#29 - FastAPI Integration and Performance Benchmarking

Pull Request - State: open - Opened by Jnmz 6 months ago

#29 - FastAPI Integration and Performance Benchmarking

Pull Request - State: open - Opened by Jnmz 6 months ago

#27 - Update build_model.py

Pull Request - State: open - Opened by fire717 6 months ago

#25 - Update Requirements.txt

Issue - State: open - Opened by Soumadip-Saha 7 months ago

#24 - Run on second GPU (torch.device("cuda:1"))

Issue - State: open - Opened by imabot2 8 months ago - 1 comment

#22 - Run without quantization

Issue - State: open - Opened by freQuensy23-coder 8 months ago - 9 comments

#21 - hqq_aten package not installed.

Issue - State: closed - Opened by LeMoussel 8 months ago - 1 comment

#20 - Update typo in README.md

Pull Request - State: open - Opened by kaushalpowar 8 months ago

#18 - CUDA OOM errors in wsl2

Issue - State: open - Opened by MrNova111 8 months ago

#17 - Is it possible to finetune this on a custom dataset?

Issue - State: open - Opened by asmith26 9 months ago - 8 comments

#16 - Can it run with LlamaIndex?

Issue - State: open - Opened by LeMoussel 9 months ago

#15 - Can it run on multi-GPU?

Issue - State: open - Opened by drdh 9 months ago - 10 comments

#14 - Doesn't work

Issue - State: closed - Opened by SanskarX10 9 months ago - 11 comments

#13 - How to use the offloading in my MoE model?

Issue - State: closed - Opened by WangRongsheng 9 months ago - 4 comments

#12 - CLI interface added

Pull Request - State: open - Opened by NJannasch 9 months ago - 3 comments

#11 - Mixtral OffLoading/GGUF/ExLlamaV2, which approach to use?

Issue - State: open - Opened by LeMoussel 9 months ago - 1 comment

#9 - Utilized pop for meta keys cleanup

Pull Request - State: closed - Opened by vivekmaru36 9 months ago - 1 comment

#8 - Update README.md

Pull Request - State: closed - Opened by eltociear 9 months ago - 1 comment

#7 - Session crashed on colab

Issue - State: closed - Opened by bitsnaps 9 months ago - 4 comments

#6 - Revert "Some refactoring"

Pull Request - State: closed - Opened by lavawolfiee 9 months ago

#5 - Some refactoring

Pull Request - State: closed - Opened by lavawolfiee 9 months ago

#4 - exl2

Issue - State: open - Opened by eramax 9 months ago - 2 comments

#3 - Refactor

Pull Request - State: closed - Opened by dvmazur 9 months ago

#2 - adding requirements.txt

Pull Request - State: open - Opened by h9-tect 9 months ago - 1 comment

#1 - Fix colab

Pull Request - State: closed - Opened by dvmazur 9 months ago