Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / NVIDIA/TensorRT-LLM issues and pull requests

#222 - Build engine for a different architecture than the current device

Issue - State: closed - Opened by yunfeng-scale about 1 year ago - 9 comments
Labels: good first issue, question, triaged

#174 - Support for DeBerta

Issue - State: open - Opened by kamalkraj about 1 year ago - 2 comments
Labels: triaged, feature request, new model

#157 - Support for Zephyr 7B model

Issue - State: open - Opened by rishabh279 about 1 year ago - 7 comments
Labels: triaged, feature request, new model

#124 - [FEA] Support Roberta model

Issue - State: open - Opened by esnvidia about 1 year ago - 2 comments
Labels: triaged, feature request, new model

#100 - Unable to compile Tensorrt_llm image,error msg is : error adding symbols: file in wrong format

Issue - State: closed - Opened by WangxuP about 1 year ago - 1 comment
Labels: triaged

#99 - parallel on 4 RTX3090 for llama 7b fails

Issue - State: closed - Opened by forrestjgq about 1 year ago - 5 comments
Labels: triaged

#98 - Weight cannot be divisible by tensor_parallel_size

Issue - State: closed - Opened by haojiwei about 1 year ago - 8 comments
Labels: triaged

#97 - Compilation issue about weightOnlyBatchedGemvBs4Int8b.cu

Issue - State: closed - Opened by isky-cd about 1 year ago - 2 comments
Labels: triaged

#95 - Error build llama-2-7b-chat-hf：first input has type Half but second input has type Float

Issue - State: closed - Opened by a1164714 about 1 year ago - 7 comments
Labels: triaged

#94 - error: libtensorrt_llm_batch_manager_static.a: file format not recting as linker script

Issue - State: closed - Opened by WangxuP about 1 year ago - 2 comments
Labels: triaged

#93 - does the chatglm2-6B example support codegeex2-6b's building and running?

Issue - State: closed - Opened by thendwk about 1 year ago - 9 comments
Labels: bug, triaged

#92 - gmake: Makefile: No such file or directory

Issue - State: closed - Opened by Kelsey2018 about 1 year ago - 6 comments
Labels: triaged

#91 - Error when quantize llama-2-7b-chat-hf with format `int4_awq`

Issue - State: closed - Opened by gesanqiu about 1 year ago - 8 comments
Labels: triaged

#90 - stop words can not work. how to set it

Issue - State: closed - Opened by yoyopdc about 1 year ago - 25 comments
Labels: triaged

#89 - Failed to launch Triton for Llama

Issue - State: closed - Opened by sleepwalker2017 about 1 year ago - 18 comments
Labels: triaged

#88 - set max_input_len, max_output_len=4096, But The actual input cannot reach this level

Issue - State: closed - Opened by callmezhangchenchenokay about 1 year ago - 2 comments
Labels: bug, triaged

#86 - [Feature Request] Support InternLM Model

Issue - State: closed - Opened by vansin about 1 year ago - 3 comments

#85 - Failed to Run batch size larger than 8 with LLaMA 13B

Issue - State: closed - Opened by CarsonSo about 1 year ago - 12 comments
Labels: duplicate, triaged

#84 - Building chatglm2-6b on single node multi gpus failed.

Issue - State: closed - Opened by shaunxiong about 1 year ago - 1 comment
Labels: triaged

#83 - cuda driver version

Issue - State: closed - Opened by Kelsey2018 about 1 year ago - 6 comments
Labels: bug, triaged

#82 - errors arised when build docker image

Issue - State: closed - Opened by shanekong about 1 year ago - 8 comments

#81 - Chinese, Korean, and other Asian languages not working

Issue - State: closed - Opened by michaelroyzen about 1 year ago - 6 comments
Labels: triaged

#80 - How to output intermediate result of model?

Issue - State: closed - Opened by yukavio about 1 year ago - 2 comments
Labels: good first issue, triaged

#79 - Complex beam search

Issue - State: closed - Opened by haramjo about 1 year ago - 6 comments
Labels: triaged

#78 - Donglu branch

Pull Request - State: closed - Opened by dongluw about 1 year ago

#77 - segmentation fault : llama 2 with num_beams > 1

Issue - State: closed - Opened by JosephChenHub about 1 year ago - 5 comments
Labels: triaged

#76 - Add Python bindings to `GptManager`

Pull Request - State: open - Opened by linden-li about 1 year ago

#75 - Performance decay when using paged attention

Issue - State: closed - Opened by sleepwalker2017 about 1 year ago - 9 comments
Labels: bug, triaged

#74 - Will the batch_manager be open-sourced in the future?

Issue - State: closed - Opened by chenzhengda about 1 year ago - 1 comment

#73 - How to benchmark offline throughput?

Issue - State: open - Opened by zhaoyang-star about 1 year ago - 6 comments
Labels: question, triaged

#72 - Build Failure: libtensorrt_llm_batch_manager_static.a:1: syntax error

Issue - State: closed - Opened by CarsonSo about 1 year ago - 5 comments
Labels: triaged

#71 - How to build the docker image

Issue - State: closed - Opened by LeoCeasar about 1 year ago - 6 comments
Labels: triaged

#70 - How to build Tensorrt-LLM without docker?

Issue - State: closed - Opened by beginlner about 1 year ago - 1 comment
Labels: question, triaged

#68 - Llama 2 with LoRA

Issue - State: closed - Opened by JosephChenHub about 1 year ago - 11 comments
Labels: triaged, feature request

#66 - gptManagerBenchmark std::bad_alloc error

Issue - State: closed - Opened by clockfly about 1 year ago - 19 comments
Labels: triaged

#64 - build failure: libnvparsers not found in tensorRT 9.1.0.4

Issue - State: closed - Opened by forrestjgq about 1 year ago - 8 comments
Labels: triaged

#63 - Improved Readability of Readme.md File

Pull Request - State: closed - Opened by Sanyam-2026 about 1 year ago - 1 comment
Labels: documentation

#62 - Nvidia Jetson device Support

Issue - State: open - Opened by shahizat about 1 year ago - 31 comments
Labels: triaged, feature request

#61 - Question:Why cannot the topk exceed 1024?

Issue - State: closed - Opened by SuperCB about 1 year ago - 1 comment
Labels: question, triaged

#60 - Update windows related documentation to main branch

Pull Request - State: closed - Opened by juney-nvidia about 1 year ago

#59 - Update windows related documentation

Pull Request - State: closed - Opened by juney-nvidia about 1 year ago

#58 - Perceived

Pull Request - State: closed - Opened by RichardScottOZ about 1 year ago - 1 comment
Labels: documentation

#57 - Seeking Clarification on TensorRT-LLM Workflow for Extending Model Support

Issue - State: closed - Opened by robosina about 1 year ago - 1 comment

#56 - Fix typo in batchScheduler.h

Pull Request - State: closed - Opened by eltociear about 1 year ago - 1 comment
Labels: documentation

#54 - starcoder engine building failed

Issue - State: closed - Opened by Missmiaom about 1 year ago - 2 comments

#53 - wrong output in GPT2 example

Issue - State: closed - Opened by r3sist-uniq about 1 year ago - 6 comments
Labels: triaged

#52 - Docker image

Issue - State: closed - Opened by SehajDxstiny about 1 year ago - 9 comments
Labels: triaged

#51 - Handling kv-cache in multi-modal GPT

Issue - State: closed - Opened by Selimonder about 1 year ago - 7 comments
Labels: triaged

#50 - x86_64-conda_cos6-linux-gnu-cc: command not found

Issue - State: open - Opened by JiushengChen about 1 year ago - 2 comments
Labels: triaged

#49 - Mistral 7B support

Issue - State: closed - Opened by casper-hansen about 1 year ago - 7 comments
Labels: triaged, feature request

#48 - Error on docker file build

Issue - State: closed - Opened by robosina about 1 year ago - 2 comments

#47 - Any support for RWKV plz?

Issue - State: open - Opened by Pevernow about 1 year ago - 17 comments
Labels: triaged, feature request, new model

#46 - fix Forward Compatibility mode is UNAVAILABLE error

Pull Request - State: closed - Opened by BasicCoder about 1 year ago - 2 comments

#45 - Failed to build with CUDA 11.8 due to change in cudaGraphExecUpdate parameter

Issue - State: closed - Opened by haojiwei about 1 year ago - 7 comments
Labels: triaged

#44 - Failed to run benchmark llama-7b.

Issue - State: closed - Opened by matichon-vultureprime about 1 year ago - 2 comments

#43 - Methods to evaluate Throughput (tokens/s)

Issue - State: closed - Opened by tiandiao123 about 1 year ago - 1 comment

#42 - Failed to run batch inference

Issue - State: closed - Opened by sleepwalker2017 about 1 year ago - 13 comments
Labels: triaged

#41 - TensorRT-LLM Releases

Issue - State: closed - Opened by jdemouth-nvidia about 1 year ago
Labels: documentation

#40 - Update docs/source/batch_manager.md

Pull Request - State: closed - Opened by kaiyux about 1 year ago

#39 - run tritonserver failed for chatglm2

Issue - State: closed - Opened by Lzhang-hub about 1 year ago - 24 comments
Labels: triaged

#38 - Cherry-pick https://github.com/NVIDIA/TensorRT-LLM/pull/21 to main branch

Pull Request - State: closed - Opened by kaiyux about 1 year ago

#37 - Baichuan V2 13B，INT4 weight only， 输出可能有问题

Issue - State: closed - Opened by callmezhangchenchenokay about 1 year ago - 20 comments
Labels: bug, triaged

#36 - Building in TensorRT docker container

Issue - State: closed - Opened by makaveli10 about 1 year ago - 9 comments
Labels: triaged

#35 - TensorRT 9 not available

Issue - State: closed - Opened by david-PHR about 1 year ago - 5 comments
Labels: triaged

#34 - Attribute issue while executing build.py file

Issue - State: closed - Opened by atharvnagrikar about 1 year ago - 18 comments
Labels: triaged

#33 - Will it be integrated into tritonserver-inference?

Issue - State: closed - Opened by coderchem about 1 year ago - 4 comments
Labels: triaged

#32 - Build failures

Issue - State: closed - Opened by divchenko about 1 year ago - 17 comments
Labels: bug, triaged

#31 - Does the project team plan to support MiniGPT4?

Issue - State: closed - Opened by xiexiaoshinick about 1 year ago - 1 comment

#30 - Fix link jump in windows readme.md

Pull Request - State: closed - Opened by yuanlehome about 1 year ago - 3 comments

#29 - Tactic running out of memory during Code Llama 34B build

Issue - State: closed - Opened by michaelroyzen about 1 year ago - 18 comments
Labels: triaged

#28 - a error when build tensorrt_llm

Issue - State: closed - Opened by nanmi about 1 year ago - 3 comments
Labels: triaged

#27 - can i use this for GPT-2?

Issue - State: closed - Opened by SehajDxstiny about 1 year ago - 1 comment
Labels: triaged

#26 - Will this repo support A10 in the future?

Issue - State: closed - Opened by Fangzhou-Ai about 1 year ago - 20 comments
Labels: bug, triaged

#25 - Support Medusa Sampling

Issue - State: closed - Opened by forpanyang about 1 year ago - 10 comments
Labels: Community want to contribute

#24 - Speed compare with vllm?

Issue - State: closed - Opened by lucasjinreal about 1 year ago - 6 comments
Labels: triaged

#22 - Can we install TensorRT-LLM without docker?

Issue - State: closed - Opened by ifromeast about 1 year ago - 24 comments
Labels: triaged

#21 - Fix two deadlinks in README.md

Pull Request - State: closed - Opened by wangkuiyi about 1 year ago

#20 - Main doc fix

Pull Request - State: closed - Opened by juney-nvidia about 1 year ago

#19 - Fix small doc issue

Pull Request - State: closed - Opened by juney-nvidia about 1 year ago

#18 - Include a working Windows wheel link..

Issue - State: closed - Opened by oscarbg about 1 year ago - 5 comments
Labels: triaged

#17 - Where is download for Tensorrt 9.1.0.4 for Windows?

Issue - State: closed - Opened by oscarbg about 1 year ago - 1 comment

#16 - Huggingface Transformers version should be bumped

Issue - State: closed - Opened by michaelroyzen about 1 year ago - 2 comments
Labels: bug, triaged

#15 - Fix the link to the documentation

Pull Request - State: closed - Opened by jdemouth about 1 year ago

#14 - revise the homepage (release)

Pull Request - State: closed - Opened by Shixiaowei02 about 1 year ago

#13 - revise the homepage (main)

Pull Request - State: closed - Opened by Shixiaowei02 about 1 year ago - 1 comment

#12 - add git-lfs dependency for binaries (main branch)

Pull Request - State: closed - Opened by Shixiaowei02 about 1 year ago

#11 - add git-lfs dependency for binaries (release)

Pull Request - State: closed - Opened by Shixiaowei02 about 1 year ago

#10 - update aarch64 batch manager libraries to release/0.5.0

Pull Request - State: closed - Opened by Shixiaowei02 about 1 year ago

#9 - update aarch64 batch manager libraries to main

Pull Request - State: closed - Opened by Shixiaowei02 about 1 year ago

#8 - Fix memory leak in falcon weight loader

Pull Request - State: closed - Opened by kaiyux about 1 year ago

#7 - update aarch64 libraries to release/0.5.0 branch

Pull Request - State: closed - Opened by Shixiaowei02 about 1 year ago

#6 - update aarch64 libraries to main branch

Pull Request - State: closed - Opened by Shixiaowei02 about 1 year ago