Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / tlntin/qwen-tensorrt-llm issues and pull requests

#103 - web_demo无法显示模型响应

Issue - State: closed - Opened by elegant-bot 8 months ago - 2 comments

#103 - web_demo无法显示模型响应

Issue - State: closed - Opened by elegant-bot 8 months ago - 2 comments

#102 - Adjust the semantics of max_output_len to be consistent with TRT-LLM

Pull Request - State: closed - Opened by BasicCoder 8 months ago - 1 comment

#102 - Adjust the semantics of max_output_len to be consistent with TRT-LLM

Pull Request - State: closed - Opened by BasicCoder 8 months ago - 1 comment

#101 - triton 部署， 生成乱码

Issue - State: closed - Opened by maozixi1 8 months ago - 18 comments

#101 - triton 部署， 生成乱码

Issue - State: closed - Opened by maozixi1 8 months ago - 18 comments

#100 - qwen_14b_chat build error

Issue - State: closed - Opened by dangerous-xu 8 months ago - 3 comments

#100 - qwen_14b_chat build error

Issue - State: closed - Opened by dangerous-xu 8 months ago - 3 comments

#99 - Fix bug

Pull Request - State: closed - Opened by zhaohb 8 months ago

#99 - Fix bug

Pull Request - State: closed - Opened by zhaohb 8 months ago

#98 - Qwen 2 build.py multi gpu with 2 different GPU's issue

Issue - State: open - Opened by teis-e 8 months ago - 17 comments

#98 - Qwen 2 build.py multi gpu with 2 different GPU's issue

Issue - State: open - Opened by teis-e 8 months ago - 17 comments

#97 - update verstion to 0.8.0

Pull Request - State: closed - Opened by Tlntin 8 months ago

#97 - update verstion to 0.8.0

Pull Request - State: closed - Opened by Tlntin 8 months ago

#96 - 编译tritonserver 镜像 失败

Issue - State: closed - Opened by maozixi1 8 months ago - 4 comments

#96 - 编译tritonserver 镜像 失败

Issue - State: closed - Opened by maozixi1 8 months ago - 4 comments

#95 - Triton 和 Langchain部署问题

Issue - State: closed - Opened by plt12138 8 months ago - 6 comments

#95 - Triton 和 Langchain部署问题

Issue - State: closed - Opened by plt12138 8 months ago - 6 comments

#94 - how to build Qwen-72B-Chat-Int4 with tp=2

Issue - State: closed - Opened by liyunhan 8 months ago - 27 comments

#94 - how to build Qwen-72B-Chat-Int4 with tp=2

Issue - State: closed - Opened by liyunhan 8 months ago - 27 comments

#93 - 运行run.py报错，Segmentation fault (core dumped)

Issue - State: closed - Opened by ArlanCooper 8 months ago - 8 comments

#93 - 运行run.py报错，Segmentation fault (core dumped)

Issue - State: closed - Opened by ArlanCooper 8 months ago - 8 comments

#92 - ModuleNotFoundError: No module named 'transformers.models.qwen2'

Issue - State: closed - Opened by ArlanCooper 8 months ago - 2 comments

#92 - ModuleNotFoundError: No module named 'transformers.models.qwen2'

Issue - State: closed - Opened by ArlanCooper 8 months ago - 2 comments

#91 - triton同步异步接口询问

Issue - State: closed - Opened by dongteng 8 months ago - 15 comments

#91 - triton同步异步接口询问

Issue - State: closed - Opened by dongteng 8 months ago - 15 comments

#88 - 请问如何支持正常的batch infer ？

Issue - State: closed - Opened by zhangyu68 8 months ago - 2 comments

#88 - 请问如何支持正常的batch infer ？

Issue - State: closed - Opened by zhangyu68 8 months ago - 2 comments

#87 - 请问为什么smoothquant量化后显存占用不降低呢

Issue - State: closed - Opened by tp-nan 8 months ago - 6 comments

#87 - 请问为什么smoothquant量化后显存占用不降低呢

Issue - State: closed - Opened by tp-nan 8 months ago - 6 comments

#82 - Qwen-72B-Chat-Int4 killed

Issue - State: closed - Opened by Hukongtao 8 months ago - 2 comments

#82 - Qwen-72B-Chat-Int4 killed

Issue - State: closed - Opened by Hukongtao 8 months ago - 2 comments

#81 - 测试hf吞吐OOM以及triton并发、流式输出问题

Issue - State: closed - Opened by dongteng 8 months ago - 23 comments
Labels: bug

#81 - 测试hf吞吐OOM以及triton并发、流式输出问题

Issue - State: closed - Opened by dongteng 8 months ago - 23 comments
Labels: bug

#80 - Qwen2 编译错误

Issue - State: closed - Opened by mogoxx 8 months ago - 5 comments

#80 - Qwen2 编译错误

Issue - State: closed - Opened by mogoxx 8 months ago - 5 comments

#78 - Qwen1.5 GPTQ编译错误

Issue - State: closed - Opened by compass-star 9 months ago - 1 comment

#78 - Qwen1.5 GPTQ编译错误

Issue - State: closed - Opened by compass-star 9 months ago - 1 comment

#77 - Qwen1.5 GPTQ-Int4 编译失败

Issue - State: closed - Opened by ljhssga 9 months ago - 15 comments

#77 - Qwen1.5 GPTQ-Int4 编译失败

Issue - State: closed - Opened by ljhssga 9 months ago - 15 comments

#76 - Qwen1.5 GPTQ用不了

Issue - State: closed - Opened by Pevernow 9 months ago - 2 comments

#76 - Qwen1.5 GPTQ用不了

Issue - State: closed - Opened by Pevernow 9 months ago - 2 comments

#75 - swift微调的qwen-vl支持吗

Issue - State: closed - Opened by xs818818 9 months ago - 1 comment

#75 - swift微调的qwen-vl支持吗

Issue - State: closed - Opened by xs818818 9 months ago - 1 comment

#74 - 函数调用会报错

Issue - State: closed - Opened by xzmagic 9 months ago

#74 - 函数调用会报错

Issue - State: closed - Opened by xzmagic 9 months ago

#72 - 大佬有没有对比和VLLM的推理效果？

Issue - State: open - Opened by white-wolf-tech 10 months ago - 2 comments

#72 - 大佬有没有对比和VLLM的推理效果？

Issue - State: open - Opened by white-wolf-tech 10 months ago - 2 comments

#70 - web demo error

Issue - State: closed - Opened by HappyKerry 10 months ago - 1 comment

#70 - web demo error

Issue - State: closed - Opened by HappyKerry 10 months ago - 1 comment

#68 - Qwen-14B-Chat-Int4运行后预测结果不对

Issue - State: closed - Opened by takemars 10 months ago - 4 comments
Labels: bug

#68 - Qwen-14B-Chat-Int4运行后预测结果不对

Issue - State: closed - Opened by takemars 10 months ago - 4 comments
Labels: bug

#66 - inflight_batching

Issue - State: closed - Opened by lyc728 10 months ago - 24 comments

#66 - inflight_batching

Issue - State: closed - Opened by lyc728 10 months ago - 24 comments

#65 - TypeError: missing a required argument: 'host_sink_token_length'

Issue - State: closed - Opened by Hukongtao 10 months ago - 2 comments

#65 - TypeError: missing a required argument: 'host_sink_token_length'

Issue - State: closed - Opened by Hukongtao 10 months ago - 2 comments

#64 - qwen-14b int4-awq 量化失败

Issue - State: closed - Opened by zhisunyy 10 months ago - 7 comments

#64 - qwen-14b int4-awq 量化失败

Issue - State: closed - Opened by zhisunyy 10 months ago - 7 comments

#63 - triron部署成功后，每个卡上多出来几个进程

Issue - State: closed - Opened by white-wolf-tech 10 months ago - 12 comments

#63 - triron部署成功后，每个卡上多出来几个进程

Issue - State: closed - Opened by white-wolf-tech 10 months ago - 12 comments

#62 - 使用triton + inflight_batching 后吞吐反而降了

Issue - State: closed - Opened by zhisunyy 10 months ago - 2 comments

#62 - 使用triton + inflight_batching 后吞吐反而降了

Issue - State: closed - Opened by zhisunyy 10 months ago - 2 comments

#61 - 推理加速效果怎么样？

Issue - State: closed - Opened by yanguowei316 11 months ago - 1 comment

#61 - 推理加速效果怎么样？

Issue - State: closed - Opened by yanguowei316 11 months ago - 1 comment

#60 - Triton部署TensorRT-LLM报错

Issue - State: closed - Opened by zhisunyy 11 months ago - 9 comments

#60 - Triton部署TensorRT-LLM报错

Issue - State: closed - Opened by zhisunyy 11 months ago - 9 comments

#59 - 请问是否有尝试过在mpirun -n 大于1的情况下提供http服务？

Issue - State: closed - Opened by xikaluo 11 months ago - 8 comments
Labels: enhancement

#59 - 请问是否有尝试过在mpirun -n 大于1的情况下提供http服务？

Issue - State: closed - Opened by xikaluo 11 months ago - 8 comments
Labels: enhancement

#58 - Qwen-14B INT4-AWQ 用tp=2时量化失败

Issue - State: closed - Opened by comeby 11 months ago - 1 comment

#58 - Qwen-14B INT4-AWQ 用tp=2时量化失败

Issue - State: closed - Opened by comeby 11 months ago - 1 comment

#56 - summarize.py运行解答

Issue - State: closed - Opened by lyc728 11 months ago - 1 comment

#56 - summarize.py运行解答

Issue - State: closed - Opened by lyc728 11 months ago - 1 comment

#55 - Qwen-14B-chat 多batch 报错

Issue - State: closed - Opened by zhisunyy 11 months ago - 3 comments

#55 - Qwen-14B-chat 多batch 报错

Issue - State: closed - Opened by zhisunyy 11 months ago - 3 comments

#54 - 使用autodl编译tensorrt-llm有问题

Issue - State: closed - Opened by oreo-lp 11 months ago - 6 comments

#52 - 想使用baichuan2部署api的话该修改什么地方适配百川模型呢？

Issue - State: closed - Opened by secain 11 months ago - 3 comments

#51 - Triton的显存占用是TensorRT—llm的两倍

Issue - State: open - Opened by lyc728 11 months ago - 20 comments