triton-inference-server/pytriton issues and pull requests

#81 - [Question] Can I bind additional models after triton has started serving?

Issue - State: closed - Opened by caffeinism about 2 months ago - 2 comments

#80 - PA Migration: Doc Updates

Pull Request - State: closed - Opened by fpetrini15 about 2 months ago

#79 - [Question] Tensor parallelism for tensorrt_llm

Issue - State: open - Opened by JoeLiu996 3 months ago - 1 comment
Labels: non-stale

#78 - disallow use of numpy 2 for now

Pull Request - State: closed - Opened by catwell 3 months ago - 5 comments
Labels: bug

#77 - Model is not initialized to GPU.

Issue - State: open - Opened by jaehyeong-bespin 3 months ago - 4 comments
Labels: non-stale

#76 - [Error following Quick Start] - tritonclient.utils.InferenceServerException: [400] client received an empty response from the server.

Issue - State: closed - Opened by sophot 3 months ago - 1 comment

#75 - multi-gpu inference with pytriton got worse TPS

Issue - State: open - Opened by lionsheep24 3 months ago - 4 comments
Labels: non-stale

#74 - [Question] About the subprocess for multi-instance

Issue - State: closed - Opened by leafjungle 5 months ago - 4 comments
Labels: Stale

#73 - [Question] will the server fork several subprocess when infer_func is a list?

Issue - State: closed - Opened by leafjungle 5 months ago - 1 comment

#72 - [HELP] what is the problem for this demo code?

Issue - State: closed - Opened by leafjungle 5 months ago - 1 comment

#71 - PyTriton produces DEBUG output by default?

Issue - State: closed - Opened by JanFSchulte 5 months ago - 1 comment

#70 - [Question] What is the relationship between "model_repository" and "infer_func"?

Issue - State: closed - Opened by leafjungle 5 months ago

#69 - Model instances question

Issue - State: closed - Opened by tinsss 5 months ago - 3 comments
Labels: Stale

#68 - Is there a way to run pytriton on glibc2.32?

Issue - State: closed - Opened by DZ9 6 months ago - 9 comments
Labels: Stale

#67 - double container in kubernetes

Issue - State: closed - Opened by leafjungle 6 months ago - 1 comment

#66 - [Bug] Fail to deploy serving model on the Azure Machine Learning Platform. Exited with failure (confusing error information and exit code)

Issue - State: closed - Opened by keli-wen 6 months ago - 3 comments
Labels: bug, Stale

#65 - Example of TensorRT-LLM Whisper backend for PyTriton

Issue - State: open - Opened by aleksandr-smechov 6 months ago - 5 comments
Labels: enhancement, non-stale

#64 - fix: Remove duplicated paragraph

Pull Request - State: closed - Opened by getty708 8 months ago - 3 comments
Labels: documentation

#63 - Python InferenceServerClient issue when call close() from del

Issue - State: closed - Opened by lionsheep0724 8 months ago - 7 comments
Labels: bug, Stale

#62 - Put `pytriton.client` in the separate package/wheel.

Issue - State: open - Opened by flyingleafe 8 months ago - 3 comments
Labels: enhancement, non-stale

#61 - pytriton use onnx is slower than onnx runtime for tiny bert model

Issue - State: open - Opened by yan123456jie 8 months ago - 1 comment
Labels: bug, non-stale

#60 - how to define a new api andinput like flask

Issue - State: closed - Opened by Pobby321 8 months ago - 3 comments
Labels: Stale

#59 - nav.optimize() bug

Issue - State: closed - Opened by Pobby321 8 months ago - 7 comments
Labels: Stale

#58 - Questions about new feature at 0.5.0 : decoupled model

Issue - State: closed - Opened by lionsheep0724 9 months ago - 4 comments
Labels: Stale

#57 - onnx and tensorrt model supported?

Issue - State: closed - Opened by oreo-lp 9 months ago - 3 comments
Labels: Stale

#56 - ModuleNotFoundError: No module named '_ctypes' error when run pytriton server with 0.5.0

Issue - State: closed - Opened by lionsheep0724 9 months ago - 5 comments
Labels: bug, Stale

#55 - The content of this document is wrong

Issue - State: closed - Opened by HJH0924 9 months ago - 2 comments
Labels: documentation, Stale

#54 - The content of this document is incorrect

Issue - State: closed - Opened by HJH0924 9 months ago - 2 comments
Labels: documentation, Stale

#53 - What is the proxy backend in pytriton?

Issue - State: closed - Opened by HJH0924 9 months ago - 4 comments
Labels: Stale

#52 - pytriton is slower than triton

Issue - State: closed - Opened by yan123456jie 9 months ago - 6 comments

#51 - AttributeError: '_thread.RLock' object has no attribute '_recursion_count'

Issue - State: closed - Opened by dogky123 9 months ago - 4 comments
Labels: bug, Stale

#50 - How to infer with sequence ?

Issue - State: open - Opened by monsterlyg 9 months ago - 3 comments
Labels: enhancement, question, non-stale

#49 - fix boot when allow_http=False

Pull Request - State: closed - Opened by catwell 9 months ago - 3 comments

#48 - [problem]How to allowed multiple models running on same GPU at same time?

Issue - State: closed - Opened by Firefly-Dance 10 months ago - 5 comments
Labels: Stale

#47 - while inference by running server.py and client.py why client is taking gpu memory.

Issue - State: closed - Opened by Justsubh01 10 months ago

#46 - Enabling Redis cache throws: Unable to find shared library libtritonserver.so

Issue - State: closed - Opened by zbloss 10 months ago - 5 comments
Labels: bug, Stale

#45 - Error deploying model on Vertex AI

Issue - State: closed - Opened by sricke 10 months ago - 16 comments
Labels: bug, non-stale

#44 - Support Mac installation

Issue - State: open - Opened by zbloss 10 months ago - 16 comments
Labels: enhancement, non-stale

#43 - Streaming and batching

Issue - State: closed - Opened by giuseppe915 11 months ago - 6 comments

#42 - How to pass priority level during inference?

Issue - State: open - Opened by jackielam918 11 months ago - 3 comments
Labels: enhancement, non-stale

#41 - TensorRT-LLM suport?

Issue - State: closed - Opened by LouisCastricato 11 months ago - 4 comments
Labels: non-stale

#40 - Pytriton don't nativly support pytorch or tensorflow dtype

Issue - State: closed - Opened by dahai331 11 months ago - 3 comments
Labels: question

#39 - tritonclient.grpc doesn't support timeout for other commands than infer.

Issue - State: closed - Opened by dogky123 11 months ago - 4 comments
Labels: enhancement, non-stale

#38 - Client network and/or connection timeout is smaller than requested timeout_s. This may cause unexpected behavior.

Issue - State: closed - Opened by lfxx 11 months ago - 2 comments
Labels: enhancement

#37 - Stub process 'REGIS_0' is not healthy

Issue - State: closed - Opened by lfxx 11 months ago - 2 comments

#36 - Support for ubuntu20.04

Issue - State: closed - Opened by lfxx 12 months ago - 3 comments
Labels: enhancement

#35 - OUTPUT triton: list or tuple or any kind of Iterables

Issue - State: closed - Opened by dogky123 about 1 year ago - 4 comments
Labels: Stale

#34 - Binary output truncated...

Issue - State: closed - Opened by rilango about 1 year ago - 4 comments
Labels: Stale

#33 - Update megatron example so that it would support latest changes in NeMo

Pull Request - State: closed - Opened by PeganovAnton about 1 year ago - 2 comments
Labels: Stale

#32 - Best practices with ModelClient

Issue - State: closed - Opened by markbarna about 1 year ago - 4 comments
Labels: question

#31 - Example of (or support for) Inference Callable of Triton ensemble definition

Issue - State: closed - Opened by michaelhagel about 1 year ago - 11 comments
Labels: enhancement, Stale

#30 - How to check if the server-side service is still online through the client-side?

Issue - State: closed - Opened by lfxx about 1 year ago - 2 comments

#29 - Has this repo been abandoned?

Issue - State: closed - Opened by lfxx about 1 year ago

#28 - Are there any examples of using python multiprocessing to run multiple copies of model on same GPU

Issue - State: closed - Opened by wilson97 about 1 year ago - 12 comments
Labels: Stale

#27 - Does pytriton support hot loading models?

Issue - State: closed - Opened by Fjallraven-hc about 1 year ago - 4 comments
Labels: enhancement, question

#26 - Support for aarch64

Issue - State: closed - Opened by j-delvaux about 1 year ago - 5 comments
Labels: enhancement, question, non-stale

#25 - Sagemaker example

Issue - State: closed - Opened by enricorotundo about 1 year ago - 3 comments

#24 - some questions

Issue - State: closed - Opened by lfxx about 1 year ago - 4 comments

#23 - why python3.7 is not supported?

Issue - State: closed - Opened by lfxx about 1 year ago - 2 comments

#22 - Issues inferencing HTTP with Bart model

Issue - State: closed - Opened by jridevapp about 1 year ago - 2 comments
Labels: question

#21 - How to unload models?

Issue - State: closed - Opened by lfxx about 1 year ago - 7 comments
Labels: Stale

#20 - Fix error in running example inference for add_sub_notebook

Pull Request - State: closed - Opened by yeahdongcn about 1 year ago - 2 comments
Labels: documentation

#19 - unable to install

Issue - State: closed - Opened by riyaj8888 about 1 year ago - 4 comments
Labels: Stale

#18 - Updated documentation links

Pull Request - State: closed - Opened by mahimairaja about 1 year ago - 5 comments
Labels: documentation

#17 - How to implement machine learning model, receive request as json file from client and process them

Issue - State: closed - Opened by thHust191 over 1 year ago - 3 comments
Labels: Stale

#16 - is there any performance compare table between pytriton depoly with original python depoly example on sd, gpt, tts, detection etc?

Issue - State: closed - Opened by foocker over 1 year ago - 3 comments
Labels: Stale

GitHub / triton-inference-server/pytriton issues and pull requests