replicate/cog-triton issues and pull requests

#52 - tensorrt-llm 0.12.0.dev2024073000, triton 2.46.0

Pull Request - State: open - Opened by yorickvP 4 months ago

#51 - Build failure with 0.12pre: cannot find libnvinfer_builder_resource_win.so.10.2.0

Issue - State: closed - Opened by joehoover 4 months ago - 1 comment

#50 - tensorrt-llm: 0.9 -> 0.10, triton: 2.42.0 -> 2.44.0

Pull Request - State: open - Opened by yorickvP 5 months ago - 1 comment

#49 - tweak ci

Pull Request - State: closed - Opened by technillogue 5 months ago

#48 - Stop sequences fail for some sequences

Issue - State: open - Opened by joehoover 5 months ago - 1 comment
Labels: bug

#47 - don't raise an error if min/max tokens are both set to the same value

Pull Request - State: closed - Opened by technillogue 5 months ago

#46 - better prompt errors

Pull Request - State: closed - Opened by technillogue 6 months ago

#45 - a basic check at startup

Pull Request - State: closed - Opened by technillogue 6 months ago - 1 comment

#44 - reuse downloaded weights

Pull Request - State: closed - Opened by technillogue 6 months ago

#43 - better errors

Pull Request - State: closed - Opened by technillogue 6 months ago

#42 - handle triton 0.10.0 not returning the entire sequence

Pull Request - State: closed - Opened by technillogue 6 months ago

#41 - emit token count metrics and upgrade cog

Pull Request - State: closed - Opened by technillogue 7 months ago

#41 - emit token count metrics and upgrade cog

Pull Request - State: closed - Opened by technillogue 7 months ago

#40 - add `messages` input for chat formatting

Issue - State: open - Opened by technillogue 7 months ago

#40 - add `messages` input for chat formatting

Issue - State: open - Opened by technillogue 7 months ago

#39 - Update nix CI so that runner-86 is pushed to it's

Pull Request - State: closed - Opened by joehoover 7 months ago

#39 - Update nix CI so that runner-86 is pushed to it's

Pull Request - State: closed - Opened by joehoover 7 months ago

#38 - merge cog-trt-llm into this repo

Pull Request - State: closed - Opened by yorickvP 7 months ago - 1 comment

#37 - fix max tokens (and optimize imports)

Pull Request - State: closed - Opened by technillogue 7 months ago

#37 - fix max tokens (and optimize imports)

Pull Request - State: closed - Opened by technillogue 7 months ago

#36 - Yorickvp/tokenizers 0 19

Pull Request - State: closed - Opened by technillogue 7 months ago

#36 - Yorickvp/tokenizers 0 19

Pull Request - State: closed - Opened by technillogue 7 months ago

#35 - Backport some changes from trtllm-0.9 branch

Pull Request - State: closed - Opened by yorickvP 7 months ago

#35 - Backport some changes from trtllm-0.9 branch

Pull Request - State: closed - Opened by yorickvP 7 months ago

#34 - ensure that pad and end id are loaded as ints

Pull Request - State: closed - Opened by joehoover 8 months ago

#34 - ensure that pad and end id are loaded as ints

Pull Request - State: closed - Opened by joehoover 8 months ago

#33 - Update tensorrt-llm to v0.9.0

Pull Request - State: closed - Opened by yorickvP 8 months ago

#32 - Build in github actions

Pull Request - State: closed - Opened by yorickvP 8 months ago

#31 - Merge nvidia-*-cu12 python with nix's cudaPackages

Pull Request - State: closed - Opened by yorickvP 8 months ago

#31 - Merge nvidia-*-cu12 python with nix's cudaPackages

Pull Request - State: closed - Opened by yorickvP 8 months ago

#30 - ensure that max/min new tokens doesn't exceed max seq len

Pull Request - State: closed - Opened by joehoover 8 months ago

#30 - ensure that max/min new tokens doesn't exceed max seq len

Pull Request - State: closed - Opened by joehoover 8 months ago

#29 - Smaller Images

Pull Request - State: closed - Opened by yorickvP 8 months ago

#29 - Smaller Images

Pull Request - State: closed - Opened by yorickvP 8 months ago

#28 - Joe/yorickvp/ci/joe/lang 218 investigate tps performance degradation

Pull Request - State: closed - Opened by joehoover 8 months ago - 1 comment

#28 - Joe/yorickvp/ci/joe/lang 218 investigate tps performance degradation

Pull Request - State: closed - Opened by joehoover 8 months ago - 1 comment

#27 - Joe/yorickvp/ci/joe/lang 220 identify cog triton tps bottleneck

Pull Request - State: closed - Opened by joehoover 8 months ago

#27 - Joe/yorickvp/ci/joe/lang 220 identify cog triton tps bottleneck

Pull Request - State: closed - Opened by joehoover 8 months ago

#26 - superficial change to allow merge

Pull Request - State: closed - Opened by joehoover 8 months ago

#26 - superficial change to allow merge

Pull Request - State: closed - Opened by joehoover 8 months ago

#25 - fix dockerfile for pip installing trt-llm -- fix for mpi path

Pull Request - State: closed - Opened by joehoover 8 months ago

#25 - fix dockerfile for pip installing trt-llm -- fix for mpi path

Pull Request - State: closed - Opened by joehoover 8 months ago

#24 - Joe/lang 214 add mock cog triton concurrency test to cog triton directory

Pull Request - State: closed - Opened by joehoover 8 months ago

#23 - Unify cog-triton Dockerfile LANG-213

Pull Request - State: closed - Opened by joehoover 8 months ago

#22 - restart triton during setup if it crashes or doesn't start within 3 minutes

Pull Request - State: closed - Opened by technillogue 9 months ago

#22 - restart triton during setup if it crashes or doesn't start within 3 minutes

Pull Request - State: closed - Opened by technillogue 9 months ago

#21 - catch event keyerror

Pull Request - State: closed - Opened by technillogue 9 months ago