Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / SJTU-IPADS/PowerInfer issues and pull requests

#229 - Error: the provided PTX was compiled with an unsupported toolchain

Issue - State: open - Opened by jiangzizi 21 days ago
Labels: bug-unconfirmed

#228 - about the use of OPT model

Issue - State: open - Opened by bobzhang208 about 1 month ago
Labels: question

#227 - add new model in power-infer2

Issue - State: open - Opened by Francis235 about 1 month ago - 1 comment
Labels: question

#226 - Qualcomm chips support

Issue - State: open - Opened by Francis235 about 1 month ago
Labels: question

#225 - Question about the perplexity

Issue - State: open - Opened by eljrte about 1 month ago
Labels: question

#224 - 关于注意力块权重如何分配？

Issue - State: open - Opened by Yues007 about 2 months ago
Labels: question

#223 - 请问我该如何获得opt模型相关的weight文件？

Issue - State: open - Opened by a1bc2def6g 2 months ago
Labels: question

#222 - What does "co-activation" mean in Section 4.3 of the PowerInfer-2 paper?

Issue - State: closed - Opened by exhyy 2 months ago
Labels: question

#221 - 关于README视频Demo的问题

Issue - State: closed - Opened by lyeXzot 2 months ago
Labels: question

#220 - 统计predictor的overhead

Issue - State: open - Opened by guanchenl 2 months ago
Labels: question

#219 - Help! Want a toy example to run matmul with q40 weight by cuda kernel

Issue - State: open - Opened by Eutenacity 2 months ago
Labels: question

#218 - CUDA toolkit version?

Issue - State: open - Opened by shujiehan 3 months ago - 1 comment
Labels: question

#216 - Am i doing something wrong?

Issue - State: open - Opened by RealMrCactus 3 months ago - 1 comment
Labels: question

#213 - Some question about Fig4.

Issue - State: open - Opened by rhmaaa 4 months ago - 5 comments
Labels: question

#196 - 支持的量化类型

Issue - State: closed - Opened by deleteeeee 5 months ago - 1 comment
Labels: question

#189 - ReluFalcon 40B 在llama.cpp上无效输出

Issue - State: closed - Opened by Zctoylm0927 6 months ago - 4 comments
Labels: question

#154 - The quesion about Neuron-aware Operator

Issue - State: closed - Opened by YuMJie 9 months ago - 3 comments
Labels: question

#108 - CUDA error 13 at /home/PowerInfer/ggml-cuda.cu:9619: invalid device symbol

Issue - State: open - Opened by zilunzhang 11 months ago - 2 comments
Labels: bug-unconfirmed

#102 - 01-ai的Yi模型系列可以适配吗，我看模型结构是跟llama一样的

Issue - State: closed - Opened by felixstander 11 months ago - 1 comment
Labels: question

#99 - Fix generation error under INT4 quantization and batched prompting

Pull Request - State: closed - Opened by hodlen 11 months ago

#98 - Further optimisation of hybrid inference

Issue - State: open - Opened by hodlen 11 months ago
Labels: tracker

#97 - Optimize CUDA sparse operator with Tensor Core

Issue - State: open - Opened by hodlen 11 months ago
Labels: enhancement

#96 - Kernel fusion to reduce communication overhead

Issue - State: open - Opened by hodlen 11 months ago
Labels: enhancement

#95 - Reclaim memory from offloaded model weights

Issue - State: open - Opened by hodlen 11 months ago - 1 comment
Labels: enhancement

#94 - How to convert llama family model to powerinfer.gguf?

Issue - State: closed - Opened by Mokuroh0924 11 months ago - 1 comment
Labels: question

#93 - Meta: Wider model support for PowerInfer

Issue - State: open - Opened by hodlen 11 months ago - 10 comments
Labels: tracker

#92 - Meta: Implementing hybrid inference across key desktop platforms

Issue - State: open - Opened by hodlen 11 months ago
Labels: tracker

#90 - Update issue templates of PowerInfer

Pull Request - State: closed - Opened by hodlen 11 months ago

#89 - Add our Kanban to README.md

Pull Request - State: closed - Opened by hodlen 11 months ago

#88 - macOS/Metal inference support

Issue - State: open - Opened by hodlen 11 months ago
Labels: tracker

#87 - WSL + CUDA issues

Issue - State: open - Opened by hodlen 11 months ago
Labels: tracker

#86 - Windows CPU/GPU support

Issue - State: closed - Opened by hodlen 11 months ago - 2 comments
Labels: tracker

#85 - Fix offloading / VRAM budget bugs

Issue - State: open - Opened by hodlen 11 months ago - 2 comments
Labels: tracker

#84 - 请问original weight, predictor weights是怎么生成的？

Issue - State: open - Opened by sunnyregion 11 months ago - 2 comments
Labels: question

#83 - Can we make it run on other models?

Issue - State: open - Opened by YLSnowy 11 months ago - 6 comments
Labels: question

#82 - Converting GGUF Models and Support for Smaller Models

Issue - State: open - Opened by nndnnv 11 months ago - 1 comment
Labels: enhancement

#81 - didn't use gpu

Issue - State: closed - Opened by yuxx0218 11 months ago - 4 comments

#80 - cmake -S . -B build -DLLAMA_CUBLAS=ON

Issue - State: open - Opened by hungptit123 11 months ago - 1 comment
Labels: bug-unconfirmed

#79 - 我不懂编程

Issue - State: closed - Opened by dyt06 11 months ago - 2 comments
Labels: help wanted

#78 - pip install -r requirements 提示 ./gguf-py not installable

Issue - State: closed - Opened by jqliu42 11 months ago - 3 comments
Labels: help wanted

#77 - When I enable the gpu split,the inference result is unacceptable

Issue - State: closed - Opened by Gengchunsheng 11 months ago - 6 comments
Labels: bug

#76 - Convert HF models with sparse threshold specified

Pull Request - State: closed - Opened by Szy0127 11 months ago - 1 comment

#74 - Seems not support long prompt well.

Issue - State: open - Opened by swankong 11 months ago - 3 comments
Labels: question

#73 - Add Windows CPU/GPU CMake support

Pull Request - State: closed - Opened by bobozi-cmd 11 months ago - 7 comments

#72 - Update README.md

Pull Request - State: closed - Opened by YixinSong-e 11 months ago

#71 - Add news

Pull Request - State: closed - Opened by YixinSong-e 11 months ago

#70 - 请问下针对消费级卡的服务器的适配。

Issue - State: open - Opened by hua-bang 11 months ago - 2 comments
Labels: question

#69 - 请问下针对消费级卡的服务器的适配。

Issue - State: closed - Opened by hua-bang 11 months ago
Labels: enhancement

#68 - Add demo link to README.md

Pull Request - State: closed - Opened by hodlen 11 months ago

#67 - 请问你们是否有兴趣支持deepseek？

Issue - State: closed - Opened by homosapien-lcy 11 months ago - 3 comments

#66 - is it possible in future run mixtal8x7b

Issue - State: open - Opened by zotona 11 months ago - 3 comments
Labels: enhancement

#65 - [HELP WANTED] 支持 InternLM 吗？

Issue - State: closed - Opened by vansin 11 months ago - 1 comment
Labels: enhancement

#64 - How to integrate with LangChain?

Issue - State: open - Opened by tigerinus 11 months ago - 1 comment
Labels: enhancement

#63 - CUDA error 1 in ggml-cuda.cu:8332: invalid argument, and then segmentation fault

Issue - State: open - Opened by 3dluvr 11 months ago - 3 comments
Labels: bug

#62 - Fix VRAM capacity assertion bug

Pull Request - State: closed - Opened by hodlen 11 months ago

#59 - GitHub

Issue - State: closed - Opened by maxrubelvai 11 months ago - 2 comments
Labels: invalid

#58 - windows visual studio编译失败

Issue - State: open - Opened by ChenXiaoTemp 11 months ago - 3 comments
Labels: bug

#57 - 精度的对比

Issue - State: closed - Opened by FL77N 11 months ago - 1 comment
Labels: question

#56 - llama2中文 hf格式.bin 如何转换成PowerInfer格式?

Issue - State: closed - Opened by Chenhuaqi6 11 months ago - 3 comments
Labels: question

#55 - No module named powerinfer, can ot split gpu

Issue - State: closed - Opened by Gengchunsheng 11 months ago - 6 comments
Labels: bug

#54 - 请问想要部署自己的模型

Issue - State: closed - Opened by tanklandry 11 months ago - 1 comment
Labels: question

#53 - server cannot run

Issue - State: closed - Opened by Gengchunsheng 11 months ago - 3 comments
Labels: bug

#52 - In-depth Analysis of Memory Management for Enhanced Performance on Consumer-grade GPUs

Issue - State: open - Opened by yihong1120 11 months ago - 1 comment
Labels: enhancement

#51 - Chat model

Issue - State: closed - Opened by yzc111 11 months ago - 2 comments

#50 - testing vs ollama mistral gives same speed results on llama2 7b

Issue - State: open - Opened by jtoy 11 months ago - 9 comments

#48 - Small code change - IFs to mapping

Pull Request - State: closed - Opened by 3x0dv5 11 months ago - 1 comment

#47 - Bitcoin

Issue - State: closed - Opened by Thato2009 11 months ago

#46 - no CUDA-capable device is detected

Issue - State: open - Opened by jasonmhead 11 months ago - 4 comments

#45 - add this line in readme

Pull Request - State: closed - Opened by samehpalas 11 months ago - 1 comment

#44 - Add more FAQs

Pull Request - State: closed - Opened by YixinSong-e 11 months ago

#43 - Correct misleading description about offloading in README

Pull Request - State: closed - Opened by hodlen 11 months ago

#42 - llama.cpp:3107: vram_allocated_bytes < vram_capacity

Issue - State: open - Opened by theodorDiaconu 11 months ago - 12 comments
Labels: bug

#41 - Add more details on README evaluation

Pull Request - State: closed - Opened by hodlen 11 months ago

#40 - Jetson Orin+ RTXA6000

Issue - State: open - Opened by Gengchunsheng 11 months ago - 2 comments
Labels: help wanted

#39 - Combined with LLM in a flash

Issue - State: closed - Opened by qwopqwop200 11 months ago - 4 comments
Labels: enhancement

#38 - vram-budget doesn't work well.

Issue - State: open - Opened by YixinSong-e 11 months ago
Labels: bug

#37 - 会提供Docker镜像吗

Issue - State: closed - Opened by lychee-2724540853 11 months ago
Labels: enhancement

#36 - [HELP WANTED] 支持qwen吗？

Issue - State: open - Opened by xxm1668 11 months ago - 8 comments
Labels: help wanted

#35 - Length

Issue - State: closed - Opened by cyzhh 11 months ago - 2 comments
Labels: enhancement

#33 - 想请问一下有没有在A100上运行PowerInfer的效果情况

Issue - State: open - Opened by jayfeather9 11 months ago - 1 comment
Labels: enhancement

#32 - 从meta-llama/Llama-2-13b-hf到SparseLLM/ReluLLaMA-13B

Issue - State: closed - Opened by Vincent131499 11 months ago - 3 comments
Labels: enhancement

#31 - Why performance dropped a lot?

Issue - State: closed - Opened by lucasjinreal 11 months ago - 6 comments

#30 - [HELP WANTED] aquila,aquila2是类llama模型，希望能支持

Issue - State: open - Opened by lizhiling12345 11 months ago - 1 comment
Labels: help wanted

#29 - Update citation

Pull Request - State: closed - Opened by YixinSong-e 11 months ago

#28 - 请问PowerInfer团队有计划支持Bo Peng团队开发的RWKV-LM吗？

Issue - State: open - Opened by yuunnn-w 11 months ago - 2 comments
Labels: enhancement

#27 - Evaluation

Issue - State: closed - Opened by nd7141 11 months ago - 3 comments

#26 - BUG: LLaMA-7B will not fully offload to GPU

Issue - State: open - Opened by YixinSong-e 11 months ago
Labels: bug

#25 - what is the recommended wy to run with this python code?

Issue - State: open - Opened by jtoy 11 months ago - 1 comment
Labels: help wanted

#24 - Update README.md

Pull Request - State: closed - Opened by eltociear 11 months ago

#23 - nvcc fails due to illegal options

Issue - State: closed - Opened by gkoundry 11 months ago - 3 comments
Labels: bug