pytorch/TensorRT issues and pull requests

#2193 - Update usage of PyTorch's custom op API

Pull Request - State: open - Opened by zou3519 about 1 year ago - 9 comments
Labels: component: lowering, component: api [Python], cla signed, component: dynamo

#2192 - Pull operations that can be evaluated out of converters

Issue - State: closed - Opened by narendasan about 1 year ago - 1 comment
Labels: component: conversion, No Activity, component: dynamo

#2191 - feat: Add ExportedProgram as an IR

Pull Request - State: closed - Opened by peri044 about 1 year ago
Labels: component: api [Python], cla signed

#2190 - feat: Add support for device compilation setting

Pull Request - State: open - Opened by gs-olive about 1 year ago
Labels: component: tests, component: conversion, component: api [Python], cla signed, component: dynamo

#2189 - Py38 compatibility

Pull Request - State: closed - Opened by narendasan about 1 year ago
Labels: component: lowering, component: conversion, component: converters, component: build system, component: api [Python], component: runtime, cla signed, component: dynamo, component: torch_compile

#2188 - fix: Address runtimes with 0D inputs

Pull Request - State: open - Opened by gs-olive about 1 year ago
Labels: component: tests, component: core, component: api [Python], component: runtime, cla signed

#2187 - 🐛 [Bug] Compiling Models with Int-to-Tensor Casted Inputs

Issue - State: open - Opened by gs-olive about 1 year ago
Labels: bug, component: runtime

#2186 - 🐛 [Bug] `aten.gelu` segfault

Issue - State: closed - Opened by gs-olive about 1 year ago
Labels: bug, component: converters

#2185 - 🐛 [Bug] `aten.convolution` fails when 1D stride is tuple

Issue - State: closed - Opened by gs-olive about 1 year ago - 3 comments
Labels: bug, component: converters

#2184 - 🐛 [Bug] ValueError: Unknown ir was requested

Issue - State: closed - Opened by LSC527 about 1 year ago - 1 comment
Labels: bug

#2182 - fix: Update `aten.embedding` to reflect schema

Pull Request - State: closed - Opened by gs-olive about 1 year ago
Labels: component: conversion, component: converters, component: api [Python], cla signed, Story: ATen Op Support, component: dynamo

#2181 - ↔ [Converter] TensorRT native operations in Dynamo

Issue - State: open - Opened by narendasan about 1 year ago
Labels: feature request, component: converters, Story

#2180 - 🐛 [Bug] BERT model failure after adding broadcasting rules

Issue - State: closed - Opened by apbose about 1 year ago
Labels: bug

#2178 - ❓ [Question] RuntimeError: Unsupported TypeMeta in ATen: nullptr (uninitialized)?

Issue - State: closed - Opened by jestiny0 about 1 year ago - 10 comments
Labels: bug, No Activity, component: partitioning

#2177 - Make a CI job to report opset coverage

Issue - State: closed - Opened by narendasan about 1 year ago - 1 comment
Labels: No Activity

#2175 - feat: Improve Dynamo partitioning System Performance on Large Models

Pull Request - State: closed - Opened by gs-olive about 1 year ago - 1 comment
Labels: component: tests, component: lowering, component: conversion, component: build system, component: api [Python], cla signed, component: dynamo, Story: Export/Compile Unification, Story: Dynamo Compile Improvements, component: torch_compile

#2174 - Support amax dynamo converter

Pull Request - State: closed - Opened by zewenli98 about 1 year ago - 1 comment
Labels: component: tests, component: conversion, component: converters, component: api [Python], cla signed, component: dynamo

#2173 - feat: Add Selective ATen decompositions

Pull Request - State: open - Opened by gs-olive about 1 year ago - 3 comments
Labels: component: lowering, component: api [Python], cla signed, component: dynamo, Story: Export/Compile Unification, component: torch_compile

#2171 - 🐛PEP517 compatible changes require some path changes to setup.py

Issue - State: closed - Opened by apbose about 1 year ago - 1 comment
Labels: bug, No Activity

#2169 - 🐛 [Bug] SegFormer TensorRT exported model size is higher than expected

Issue - State: closed - Opened by laclouis5 about 1 year ago - 2 comments
Labels: bug, No Activity

#2168 - 🐛 [Bug] Cannot export with PTQ using a cached calibrator: "TypeError: cannot pickle DataLoaderCalibrator object"

Issue - State: open - Opened by laclouis5 about 1 year ago - 7 comments
Labels: bug

#2167 - ❓ [Question] Is a INT8 calibrator specific to a given model or just specific to a dataset?

Issue - State: closed - Opened by laclouis5 about 1 year ago - 2 comments
Labels: question

#2164 - Dynamo converter support for torch.ops.aten.erf.default op

Pull Request - State: closed - Opened by bowang007 about 1 year ago - 1 comment
Labels: component: tests, component: conversion, component: converters, component: api [Python], cla signed, component: dynamo

#2163 - Fix Legacy CI Failures

Issue - State: closed - Opened by gs-olive about 1 year ago - 1 comment
Labels: No Activity, infrastructure

#2162 - feat: Improve layer naming

Pull Request - State: closed - Opened by peri044 about 1 year ago
Labels: component: conversion, component: api [Python], cla signed, component: dynamo

#2161 - feat: Add `_to_copy`, `operator.get` and `clone` ATen converters

Pull Request - State: closed - Opened by gs-olive about 1 year ago
Labels: component: tests, component: conversion, component: converters, component: api [Python], cla signed, component: dynamo, Story: Export/Compile Unification

#2159 - ❓ [Question] Could torch-tensorrt support mixed-precision inference?

Issue - State: closed - Opened by sanbuphy about 1 year ago - 2 comments
Labels: question

#2157 - Dynamo converter support

Issue - State: closed - Opened by Christina-Young-NVIDIA about 1 year ago - 1 comment
Labels: No Activity

#2156 - ✨[Feature] Dynamo quantization tests

Issue - State: closed - Opened by apbose about 1 year ago - 1 comment
Labels: feature request, No Activity

#2155 - Write RFC for engine caching (compile)

Issue - State: closed - Opened by Christina-Young-NVIDIA about 1 year ago - 1 comment
Labels: No Activity

#2154 - Write RFC for serialization (export)

Issue - State: closed - Opened by Christina-Young-NVIDIA about 1 year ago - 1 comment
Labels: No Activity

#2153 - TorchTRT to support ODISE

Issue - State: closed - Opened by Christina-Young-NVIDIA about 1 year ago - 2 comments
Labels: No Activity

#2152 - ✨[Feature] aten::embedding max_norm argument

Issue - State: closed - Opened by apbose about 1 year ago - 1 comment
Labels: feature request, No Activity

#2151 - Dynamo rsub lowering pass

Pull Request - State: closed - Opened by apbose about 1 year ago - 1 comment
Labels: component: tests, component: lowering, component: api [Python], cla signed, component: dynamo

#2150 - Dynamo sqrt implementation

Pull Request - State: open - Opened by apbose about 1 year ago
Labels: component: tests, component: conversion, component: converters, component: api [Python], cla signed, component: fx, component: dynamo

#2149 - Dynamo Var_mean dynamo pass

Pull Request - State: closed - Opened by apbose about 1 year ago
Labels: component: tests, component: lowering, component: api [Python], cla signed, component: dynamo, component: torch_compile

#2148 - 🐛 [Bug] Torch-TensorRT PTQ fails with LRASPP MobileNetV3 Large

Issue - State: closed - Opened by laclouis5 about 1 year ago - 5 comments
Labels: bug, No Activity, component: quantization

#2147 - add initial support for torch.ops.aten.neg.default converter

Pull Request - State: closed - Opened by bowang007 about 1 year ago
Labels: component: tests, component: conversion, component: converters, component: api [Python], cla signed, component: dynamo

#2146 - abandoned create_plugin() function

Pull Request - State: closed - Opened by zewenli98 about 1 year ago - 2 comments
Labels: component: conversion, component: core, component: converters, cla signed

#2145 - Port these converters to dynamo_tensorrt_converter

Issue - State: closed - Opened by narendasan about 1 year ago - 1 comment
Labels: No Activity, component: dynamo, Story: Dynamo Compile Improvements

#2144 - Exploring Stable Diffusion in Torch Compile Path

Issue - State: closed - Opened by gs-olive about 1 year ago - 4 comments
Labels: No Activity, component: dynamo, Story: Dynamo Compile Improvements

#2143 - Create RFC for Dynamo Dynamic Shapes using SymInt + FakeTensor

Issue - State: closed - Opened by Christina-Young-NVIDIA about 1 year ago - 1 comment

#2138 - feat: Add support for `require_full_compilation` in Dynamo

Pull Request - State: closed - Opened by gs-olive about 1 year ago
Labels: component: tests, component: lowering, component: api [Python], cla signed, component: dynamo, Story: Export/Compile Unification, component: torch_compile

#2136 - Add ExportedProgram as a supported IR

Issue - State: closed - Opened by narendasan about 1 year ago - 1 comment
Labels: feature request, No Activity, component: dynamo

#2135 - Support for dynamic range (batch) via dynamo.compile (torch.export)

Issue - State: closed - Opened by narendasan about 1 year ago - 1 comment
Labels: component: api [Python], No Activity, component: dynamo

#2134 - Support for quantization methods via compile (torch.export)

Issue - State: closed - Opened by narendasan about 1 year ago - 1 comment
Labels: feature request, No Activity, component: quantization, component: dynamo

#2133 - Test DLFW NGC with PEP517 compatible changes

Issue - State: closed - Opened by apbose about 1 year ago - 2 comments

#2131 - ✨[Feature] Add `min_block_size` exemption for fully-supported models

Issue - State: closed - Opened by gs-olive about 1 year ago
Labels: feature request, component: dynamo, Story: Export/Compile Unification

#2128 - feat: Add preliminary support for freezing tensors in Dynamo

Pull Request - State: closed - Opened by gs-olive about 1 year ago
Labels: component: tests, component: lowering, component: conversion, component: converters, component: api [Python], cla signed, component: fx, component: dynamo, Story: Export/Compile Unification, component: torch_compile

#2127 - ↔ [Converter] Add Dynamo converters for all Elementwise ATen Operators

Issue - State: closed - Opened by gs-olive about 1 year ago - 4 comments
Labels: No Activity, Story: ATen Op Support, component: dynamo, Story: Export/Compile Unification

#2126 - Improve Python tooling

Pull Request - State: closed - Opened by narendasan about 1 year ago
Labels: documentation, component: tests, component: lowering, component: conversion, component: core, component: converters, component: build system, component: api [Python], component: api [C++], component: evaluators, component: runtime, component: partitioning, cla signed, component: fx, fx, component: dynamo, component: torch_compile

#2125 - 🐛 [Bug] Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

Issue - State: closed - Opened by sanbuphy about 1 year ago - 2 comments
Labels: bug, No Activity, component: partitioning

#2124 - Prototype Constant Freezing in Dynamo

Issue - State: closed - Opened by gs-olive about 1 year ago - 1 comment
Labels: component: dynamo, Story: Export/Compile Unification

#2123 - torch_tensorrt.dynamo needs an implementation of convert_method_to_engine

Issue - State: closed - Opened by narendasan about 1 year ago - 1 comment
Labels: No Activity, component: dynamo

#2122 - ❓ Why the speed (time) in PTQ and QAT are different?

Issue - State: closed - Opened by panahikhas about 1 year ago - 2 comments
Labels: question, No Activity, component: quantization

#2121 - ✨[Feature] Move `_C` code out of `_util.py`

Issue - State: closed - Opened by gs-olive about 1 year ago - 2 comments
Labels: feature request, No Activity

#2120 - Add BatchNorm Converter Fix / Replacement for new Partitioner

Issue - State: closed - Opened by gs-olive about 1 year ago - 2 comments
Labels: No Activity, component: dynamo, Story: Export/Compile Unification

#2119 - 📖 [Story] Modularlizing TS core in Python

Issue - State: closed - Opened by narendasan about 1 year ago - 1 comment
Labels: No Activity, Story

#2113 - 🐛 [Bug] Expected input tensors to have type Half, found type float

Issue - State: open - Opened by thesword53 about 1 year ago - 11 comments
Labels: bug

#2108 - ❓ [Question] How can I learn to convert an intermediate format IR to the TensorRT target?

Issue - State: closed - Opened by sanbuphy about 1 year ago - 2 comments
Labels: question

#2103 - examples: Add example usage script for Function/Module-level acceleration

Pull Request - State: open - Opened by gs-olive about 1 year ago
Labels: documentation, cla signed, component: dynamo, Story: Dynamo Compile Improvements

#2102 - 🐛 [Bug] torch_tensorrt.ts.convert_method_to_trt_engine -> Unsupported operator

Issue - State: closed - Opened by proevgenii about 1 year ago - 6 comments
Labels: bug, No Activity

#2102 - 🐛 [Bug] torch_tensorrt.ts.convert_method_to_trt_engine -> Unsupported operator

Issue - State: open - Opened by proevgenii about 1 year ago - 6 comments
Labels: bug, No Activity

#2101 - 🐛 [Bug] Can't do quantization aware training for ViT

Issue - State: closed - Opened by proevgenii about 1 year ago - 3 comments
Labels: bug, No Activity, component: quantization

#2101 - 🐛 [Bug] Can't do quantization aware training for ViT

Issue - State: open - Opened by proevgenii about 1 year ago - 3 comments
Labels: bug, No Activity, component: quantization

#2098 - 🐛 [Bug] Detectron2 with TensorRT is slower than vanilla Detectron2

Issue - State: closed - Opened by fumin about 1 year ago - 8 comments
Labels: bug, No Activity

#2096 - ↔ [Converter] ATen Softmax Intercept Function

Issue - State: closed - Opened by gs-olive about 1 year ago - 2 comments
Labels: feature request, No Activity, Story: ATen Op Support

#2095 - ↔ [Converter] Add support for amax in Torch-TensorRT

Issue - State: closed - Opened by narendasan about 1 year ago - 1 comment
Labels: feature request, component: converters, Story: ATen Op Support

#2094 - Export BERT failure investigation

Issue - State: closed - Opened by peri044 about 1 year ago - 1 comment
Labels: No Activity

#2093 - TorchBench Integration Part 3

Issue - State: open - Opened by gs-olive about 1 year ago - 5 comments
Labels: component: dynamo, Story: Export/Compile Unification

#2092 - 🐛 [Bug] Encountered bug when using Torch-TensorRT MONAI

Issue - State: closed - Opened by peri044 about 1 year ago - 2 comments
Labels: bug, No Activity

#2089 - 🐛 [Bug] compiling detectron2 segmentation fault

Issue - State: closed - Opened by fumin about 1 year ago - 4 comments
Labels: bug, No Activity, bug: triaged [verified], component: partitioning

#2082 - ❓ [Question] How to decrease the latency of the inference?

Issue - State: closed - Opened by willdone1337 about 1 year ago - 2 comments
Labels: question, No Activity, component: runtime, performance

#2081 - ✨[Feature] Compilation Dry-Run in Dynamo

Issue - State: closed - Opened by gs-olive about 1 year ago - 3 comments
Labels: feature request, component: dynamo, Story: Export/Compile Unification, Story: Dynamo Compile Improvements

#2079 - 🐛 [Bug] Regression : Torch-TensorRT now fail to convert due to unsupported negative pad for torch.nn.ConstantPad2d

Issue - State: closed - Opened by fabricecarles about 1 year ago - 5 comments
Labels: bug, component: converters, No Activity

#2078 - Write RFC for QAT PyTorch toolkit

Issue - State: closed - Opened by Christina-Young-NVIDIA about 1 year ago - 1 comment
Labels: No Activity

#2077 - Add Dynamo TS Compile Backend

Issue - State: closed - Opened by gs-olive about 1 year ago - 1 comment
Labels: No Activity, component: dynamo, Story: Dynamo Compile Improvements

#2075 - 🐛 [Bug] TRT Error when compiling ViT with Dynamic Shape

Issue - State: closed - Opened by gs-olive about 1 year ago - 3 comments
Labels: bug, No Activity

#2074 - 🐛 [Bug] Cannot compile ViT to TensorRT

Issue - State: closed - Opened by SohamTamba over 1 year ago - 7 comments
Labels: bug, No Activity

#2073 - infra: testing out GHA CI

Pull Request - State: closed - Opened by narendasan over 1 year ago - 2 comments
Labels: component: tests, component: core, component: build system, component: api [Python], component: runtime, cla signed, component: fx

#2069 - ✨[Feature] Add detailed layer name to the NVTX markers in torch-TRT

Issue - State: closed - Opened by alexmsettle over 1 year ago - 1 comment
Labels: feature request

#2066 - ✨[Feature] Add TorchScript-path `torch.compile` backend

Issue - State: closed - Opened by gs-olive over 1 year ago - 2 comments
Labels: feature request, No Activity, component: dynamo, Story: Dynamo Compile Improvements

#2062 - ❓ [Question] "When the performance of an int8 model improves compared to an fp32 model after QAT"

Issue - State: closed - Opened by JongSeok553 over 1 year ago - 4 comments
Labels: question, No Activity, component: quantization

#2061 - 🐛 [Bug] AssertionError: to_numpy can only be called on None or a torch.Tensor, got: <tensorrt_bindings.tensorrt.ITensor

Issue - State: closed - Opened by peri044 over 1 year ago - 3 comments
Labels: bug, No Activity

#2060 - 🐛 [Bug] RuntimeError: linear convolution has bias of type <class 'tensorrt_bindings.tensorrt.ITensor'>, Expect Optional[Tensor]

Issue - State: closed - Opened by peri044 over 1 year ago - 3 comments
Labels: bug, No Activity

#2058 - ↔ [Converter] Add support for Granular Op-Support Checks in Dynamo

Issue - State: closed - Opened by gs-olive over 1 year ago - 1 comment
Labels: feature request, component: converters, Story: ATen Op Support

#2057 - Upstream Dynamo Backend to Torch

Issue - State: closed - Opened by gs-olive over 1 year ago - 1 comment
Labels: component: dynamo, Story: Dynamo Compile Improvements

#2051 - 🐛 [Bug] prim::SetAttr[name="attn_mask"](%_11, %7869)

Issue - State: open - Opened by seoungbae-park over 1 year ago - 2 comments
Labels: bug, No Activity

#2050 - Write RFC for Engine Caching

Issue - State: open - Opened by Christina-Young-NVIDIA over 1 year ago - 2 comments

#2049 - torch.compile backend fallback to inductor

Issue - State: open - Opened by Christina-Young-NVIDIA over 1 year ago - 1 comment
Labels: feature request

#2048 - Additional lower precision support through Transformer Engine

Issue - State: closed - Opened by Christina-Young-NVIDIA over 1 year ago - 2 comments
Labels: No Activity

#2044 - ❓ [Question] How can I install the latest version of python API? Torch and Tensorrt's CUDA dependencies conflict with each other.

Issue - State: closed - Opened by 1585231086 over 1 year ago - 8 comments
Labels: question, No Activity

GitHub / pytorch/TensorRT issues and pull requests