onnx/turnkeyml issues and pull requests

#51 - Proposal: Release process for Version 1.0

Issue - State: closed - Opened by jeremyfowers 12 months ago - 1 comment
Labels: testing

#50 - Eliminate the `benchmark_model()` API

Issue - State: closed - Opened by jeremyfowers 12 months ago - 1 comment
Labels: cleanup, p0, breaking

#49 - Proposal: Rename the `benchmark` command to `evaluate`

Issue - State: closed - Opened by jeremyfowers 12 months ago - 2 comments
Labels: cleanup, ui/ux, p0, breaking

#48 - Refactor build_model() out of benchmark_model()

Pull Request - State: closed - Opened by jeremyfowers 12 months ago - 1 comment

#47 - Fix a transformer model I missed

Pull Request - State: closed - Opened by jeremyfowers 12 months ago

#46 - Add back SLURM CI tests

Pull Request - State: closed - Opened by danielholanda 12 months ago

#45 - Upload a new example onnx file and fix the test and tutorial

Pull Request - State: closed - Opened by jeremyfowers 12 months ago

#44 - No special LFS rules

Pull Request - State: closed - Opened by jeremyfowers 12 months ago

#43 - Repair the transformers corpus

Pull Request - State: closed - Opened by jeremyfowers 12 months ago

#42 - Fix/remove issue references

Pull Request - State: closed - Opened by jeremyfowers 12 months ago

#41 - ONNX model size on disk is not captured for models >2GB in size

Issue - State: open - Opened by jeremyfowers 12 months ago
Labels: bug, p2

#40 - Help users who forgot to install the models requirements

Issue - State: open - Opened by jeremyfowers 12 months ago
Labels: enhancement, good first issue, ui/ux, p2

#39 - `benchmarking_status` stat is ambiguous

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: bug, p0, breaking

#38 - Update issue references in the code

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: bug, cleanup

#37 - Add plugin API tutorials

Issue - State: closed - Opened by jeremyfowers 12 months ago - 1 comment
Labels: documentation, enhancement, p1

#37 - Add plugin API tutorials

Issue - State: closed - Opened by jeremyfowers 12 months ago - 1 comment
Labels: documentation, enhancement, p1

#36 - Remove the set_success stage

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: cleanup, ui/ux, p1, breaking

#35 - stages_completed is out of order in the report

Issue - State: closed - Opened by jeremyfowers 12 months ago - 1 comment
Labels: bug, cleanup, p2

#34 - Proposal: Expose stages in the CLI

Issue - State: closed - Opened by jeremyfowers 12 months ago - 1 comment
Labels: enhancement, ui/ux, p2

#34 - Proposal: Expose stages in the CLI

Issue - State: closed - Opened by jeremyfowers 12 months ago - 1 comment
Labels: enhancement, ui/ux, p2

#33 - Handling what happens when `--process-isolation` crashes

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: bug, p0

#32 - Analysis does weird and incorrect things with Stable Diffusion

Issue - State: closed - Opened by jeremyfowers 12 months ago - 1 comment
Labels: bug, p1

#32 - Analysis does weird and incorrect things with Stable Diffusion

Issue - State: closed - Opened by jeremyfowers 12 months ago - 1 comment
Labels: bug, p1

#31 - Proposal: Set batch size as a first-class argument in the CLI

Issue - State: open - Opened by jeremyfowers 12 months ago
Labels: enhancement, p2

#30 - TurnkeyML shows an unhelpful error message when docker daemon is not running

Issue - State: open - Opened by jeremyfowers 12 months ago
Labels: bug, p2

#29 - Proposal: Enable setting all optional Turnkey CLI arguments using a config file

Issue - State: open - Opened by jeremyfowers 12 months ago
Labels: enhancement, ui/ux, p2

#28 - Proposal: Add --max-throughput to CLI and expose it to runtimes

Issue - State: open - Opened by jeremyfowers 12 months ago
Labels: enhancement, p2

#27 - Proposal: Enable sequences to have custom arguments exposed in the CLI (`--seq-arg`)

Issue - State: closed - Opened by jeremyfowers 12 months ago - 2 comments
Labels: enhancement, p2

#27 - Proposal: Enable sequences to have custom arguments exposed in the CLI (`--seq-arg`)

Issue - State: closed - Opened by jeremyfowers 12 months ago - 2 comments
Labels: enhancement, p2

#26 - Clarify naming for builds and evaluations

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: cleanup, p1, breaking

#25 - turnkey reporting hashes don't match analysis hashes

Issue - State: closed - Opened by jeremyfowers 12 months ago - 1 comment
Labels: bug, p2

#24 - Feature Request - report device utilization per model/benchmark

Issue - State: closed - Opened by jeremyfowers 12 months ago - 1 comment
Labels: enhancement, p1

#23 - Request: Add a column to the CSV file showing the last stage that successfully completed

Issue - State: closed - Opened by jeremyfowers 12 months ago - 1 comment
Labels: enhancement, p1

#22 - Analysis of model scripts finds two models

Issue - State: open - Opened by jeremyfowers 12 months ago
Labels: bug, p1

#21 - Remove quantization_helpers from TKML

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: cleanup, p1, breaking

#20 - Add `build` and `discover` commands

Issue - State: closed - Opened by jeremyfowers 12 months ago - 2 comments
Labels: enhancement, ui/ux, p1

#19 - Proposal: Benchmark a build directory

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: bug, enhancement, cleanup, p0

#18 - Design proposal: Simplify the `spawn` args

Issue - State: closed - Opened by jeremyfowers 12 months ago - 2 comments
Labels: enhancement, ui/ux, p1

#18 - Design proposal: Simplify the `spawn` args

Issue - State: closed - Opened by jeremyfowers 12 months ago - 2 comments
Labels: enhancement, ui/ux, p1

#17 - Models that require CUDA don't work by default

Issue - State: open - Opened by jeremyfowers 12 months ago
Labels: bug, model, p2

#16 - midas_v3_large.py in torch_hub export failed

Issue - State: open - Opened by jeremyfowers 12 months ago
Labels: bug, model, p2

#15 - `--script-args` argument is not recorded in stats

Issue - State: closed - Opened by jeremyfowers 12 months ago - 1 comment
Labels: bug, p0

#14 - Multi-cache reporting bug

Issue - State: closed - Opened by jeremyfowers 12 months ago - 1 comment
Labels: bug, p2

#13 - Move rwkv to skip

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: bug, model

#12 - Fix the transformers models

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: bug, model

#11 - Graph convolution models don't handle --pretrained

Issue - State: closed - Opened by jeremyfowers 12 months ago - 2 comments
Labels: bug, model, p1

#10 - Build directories in the cache are ignored

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: bug, p1

#9 - New feature: Model acceptance test

Issue - State: open - Opened by jeremyfowers 12 months ago
Labels: enhancement, testing, p1

#8 - Eliminate the cache/labels directory

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: cleanup, p1, breaking

#7 - Every model in the transformers corpus uses the task::Generative_AI label

Issue - State: open - Opened by jeremyfowers 12 months ago
Labels: bug, model, p0

#6 - Uncomment the Slurm test

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: bug, testing, p0

#5 - Uncomment the sample.onnx test

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: bug, testing, p0

#4 - ONNX Benchmarking tutorial references the old UX

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: bug, documentation, p0

#3 - Fix links in the readme

Pull Request - State: closed - Opened by jeremyfowers 12 months ago

#2 - Polish the readme

Issue - State: closed - Opened by jeremyfowers 12 months ago
Labels: documentation

#1 - Merge fork from cleanup

Pull Request - State: closed - Opened by ramkrishna2910 12 months ago

GitHub / onnx/turnkeyml issues and pull requests