ELS-RD/transformer-deploy issues and pull requests

#117 - embedding task fails with batch_size > 1 given variable length inputs

Issue - State: closed - Opened by bbartlett-nv over 2 years ago - 4 comments
Labels: bug

#116 - Fix/update Triton + ORT

Pull Request - State: closed - Opened by pommedeterresautee over 2 years ago - 1 comment
Labels: bug, documentation

#116 - Fix/update Triton + ORT

Pull Request - State: closed - Opened by pommedeterresautee over 2 years ago - 1 comment
Labels: bug, documentation

#115 - Cannot build TensorT engine for classification models

Issue - State: open - Opened by CecileGiang over 2 years ago - 2 comments

#115 - Cannot build TensorT engine for classification models

Issue - State: open - Opened by CecileGiang over 2 years ago - 2 comments

#114 - feat: add torchdynamo notebook

Pull Request - State: closed - Opened by pommedeterresautee over 2 years ago
Labels: bug, enhancement, performance, TensorRT, Pytorch

#113 - split model between host memory and GPU

Issue - State: open - Opened by pommedeterresautee over 2 years ago
Labels: enhancement, ONNX Runtime

#113 - split model between host memory and GPU

Issue - State: open - Opened by pommedeterresautee over 2 years ago
Labels: enhancement, ONNX Runtime

#112 - simple benchmark with torchdynamo

Issue - State: closed - Opened by pommedeterresautee over 2 years ago
Labels: enhancement

#112 - simple benchmark with torchdynamo

Issue - State: closed - Opened by pommedeterresautee over 2 years ago
Labels: enhancement

#111 - HF pipeline based inference

Issue - State: open - Opened by kamalkraj over 2 years ago - 9 comments
Labels: enhancement

#111 - HF pipeline based inference

Issue - State: open - Opened by kamalkraj over 2 years ago - 9 comments
Labels: enhancement

#110 - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Unexpected input data type. Actual: (tensor(int64)) , expected: (tensor(int32))

Issue - State: open - Opened by Matthieu-Tinycoaching over 2 years ago - 3 comments
Labels: question

#97 - Error Nodes in a graph must be topologically sorted, however input 'encoder_hidden_states' of node: name: MatMul_173_input_cast0 OpType: Cast is not output of any previous nodes

Issue - State: open - Opened by dheerajiiitv over 2 years ago - 1 comment

#95 - Switch T5 notebook to 3B model

Issue - State: closed - Opened by pommedeterresautee over 2 years ago
Labels: enhancement

#95 - Switch T5 notebook to 3B model

Issue - State: closed - Opened by pommedeterresautee over 2 years ago
Labels: enhancement

#94 - do we support converd electra model to ONNX or Tensorrt

Issue - State: closed - Opened by zhaohb over 2 years ago - 3 comments
Labels: question

#94 - do we support converd electra model to ONNX or Tensorrt

Issue - State: closed - Opened by zhaohb over 2 years ago - 3 comments
Labels: question

#92 - add text generation for t5 model with tensorrt runtime

Pull Request - State: closed - Opened by ayoub-louati over 2 years ago

#92 - add text generation for t5 model with tensorrt runtime

Pull Request - State: closed - Opened by ayoub-louati over 2 years ago

#91 - fix: triton configuration refactoring

Pull Request - State: open - Opened by gaetansnl over 2 years ago
Labels: enhancement

#89 - Mixed precision conversion getting Assertion Error

Issue - State: open - Opened by caffeinetoomuch over 2 years ago - 15 comments

#89 - Mixed precision conversion getting Assertion Error

Issue - State: open - Opened by caffeinetoomuch over 2 years ago - 15 comments

#84 - Add question-answering model

Pull Request - State: closed - Opened by kamalkraj over 2 years ago - 15 comments
Labels: enhancement, Triton

#81 - Output difference between ONNX and Pytorch in T5 notebook

Issue - State: closed - Opened by VikasOjha666 over 2 years ago - 4 comments

#81 - Output difference between ONNX and Pytorch in T5 notebook

Issue - State: closed - Opened by VikasOjha666 over 2 years ago - 4 comments

#74 - got error in optimize onnx when ran gpt2 file from demo/generative-model

Issue - State: closed - Opened by rohitmishra94 over 2 years ago - 8 comments
Labels: bug

#74 - got error in optimize onnx when ran gpt2 file from demo/generative-model

Issue - State: closed - Opened by rohitmishra94 over 2 years ago - 8 comments
Labels: bug

#60 - Out of memeory error for batch size more than 1 for T5 models.

Issue - State: open - Opened by Ki6an over 2 years ago - 15 comments

#60 - Out of memeory error for batch size more than 1 for T5 models.

Issue - State: open - Opened by Ki6an over 2 years ago - 15 comments

#57 - Transforming https://huggingface.co/csarron/roberta-base-squad-v1

Issue - State: closed - Opened by elyast over 2 years ago - 1 comment

#57 - Transforming https://huggingface.co/csarron/roberta-base-squad-v1

Issue - State: closed - Opened by elyast over 2 years ago - 1 comment

#56 - quantized model inference slow with Triton server than inference directly in python code(notebook)

Issue - State: closed - Opened by Jaeker0512 almost 3 years ago - 2 comments

#54 - How to convert_model -m "./mycustomodel"

Issue - State: closed - Opened by christianjoun almost 3 years ago - 3 comments

#54 - How to convert_model -m "./mycustomodel"

Issue - State: closed - Opened by christianjoun almost 3 years ago - 3 comments

#50 - How does the local model generate configuration files

Issue - State: open - Opened by xiaolaimeme almost 3 years ago - 3 comments

#50 - How does the local model generate configuration files

Issue - State: open - Opened by xiaolaimeme almost 3 years ago - 3 comments

#49 - GPT2 has slow inference

Issue - State: closed - Opened by kobzaond almost 3 years ago - 6 comments

#49 - GPT2 has slow inference

Issue - State: closed - Opened by kobzaond almost 3 years ago - 6 comments

#47 - error in triton config files

Issue - State: closed - Opened by chintanckg almost 3 years ago - 2 comments

#47 - error in triton config files

Issue - State: closed - Opened by chintanckg almost 3 years ago - 2 comments

#40 - GPT-J support

Issue - State: open - Opened by JonathanLehner almost 3 years ago - 5 comments

#37 - do you have any plan to support wav2vec?

Issue - State: closed - Opened by firdota almost 3 years ago - 6 comments

#37 - do you have any plan to support wav2vec?

Issue - State: closed - Opened by firdota almost 3 years ago - 6 comments

#30 - Modified templates/triton.py. Added logs & additional arguments.

Pull Request - State: closed - Opened by vishalsrao almost 3 years ago - 5 comments

#6 - add more benchmarks

Issue - State: closed - Opened by pommedeterresautee about 3 years ago - 5 comments

#6 - add more benchmarks

Issue - State: closed - Opened by pommedeterresautee about 3 years ago - 5 comments

GitHub / ELS-RD/transformer-deploy issues and pull requests