Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / iree-org/iree-comparative-benchmark issues and pull requests

#84 - [Doc] Document e2e process to onboard PT benchmarks

Issue - State: closed - Opened by pzread over 1 year ago - 1 comment

#83 - [Goal] Document process to onboard models and benchmarks

Issue - State: open - Opened by pzread over 1 year ago
Labels: goal

#82 - Add PJRT IREE Microkernel Benchmarks

Pull Request - State: closed - Opened by mariecwhite over 1 year ago

#81 - [NFC] Decouple target device from benchmark case

Pull Request - State: closed - Opened by pzread over 1 year ago

#80 - Add IREE PJRT benchmarks

Pull Request - State: closed - Opened by mariecwhite over 1 year ago

#79 - Add XLA CPU-Next benchmarks

Pull Request - State: closed - Opened by mariecwhite over 1 year ago

#78 - Change JAX compile time from seconds to milliseconds

Pull Request - State: closed - Opened by mariecwhite over 1 year ago

#77 - [DB Import] Add presubmit check

Pull Request - State: closed - Opened by beckerhe over 1 year ago - 5 comments

#76 - Reformat benchmark IDs

Pull Request - State: closed - Opened by pzread over 1 year ago

#75 - [DB Import] Add README file

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#74 - [DB Import] Also check SQL queries in verify command

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#73 - Upload benchmark artifacts to the comparative benchmark GCS

Pull Request - State: closed - Opened by pzread over 1 year ago - 1 comment

#72 - [DB Import] Add verify subcommand to CLI tool

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#71 - [DB Import] Replace bq mk by SQL in deploy command

Pull Request - State: closed - Opened by beckerhe over 1 year ago - 1 comment

#70 - Update HLO benchmarks

Pull Request - State: closed - Opened by mariecwhite over 1 year ago

#69 - Enable CPU benchmarks

Pull Request - State: closed - Opened by pzread over 1 year ago

#68 - [DB Import] Factor out config loading and add embedding support

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#67 - [DB Import] Add missing requirements

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#66 - [DB Import] Use anonymous credentials in bigquery emulator

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#65 - Generate PyTorch artifacts

Pull Request - State: closed - Opened by mariecwhite over 1 year ago

#64 - Fix JAX T5ForConditionalGeneration

Pull Request - State: closed - Opened by mariecwhite over 1 year ago

#63 - Migrate PyTorch ResNet50 benchmarks

Pull Request - State: closed - Opened by pzread over 1 year ago - 1 comment

#61 - [PT-Inductor:GPU] Out of memory on BERT_LARGE FP32 with large batch sizes

Issue - State: open - Opened by pzread over 1 year ago - 1 comment
Labels: bug

#60 - Refactor Model Interface to not use a tuple for its `forward` function

Issue - State: closed - Opened by mariecwhite over 1 year ago - 1 comment

#59 - Update JAX model artifacts

Pull Request - State: closed - Opened by mariecwhite over 1 year ago

#58 - Migrate PyTorch benchmark tools and BERT benchmarks

Pull Request - State: closed - Opened by pzread over 1 year ago - 1 comment

#57 - [DB Import] Add first SQL tests and bigquery emulator plumbing

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#56 - [DB Import] Rename cloud_functions config key to pipelines

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#55 - [DB Import] Unify quoting in InMemoryDatabase and LocalStorage

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#54 - Support data types in model template

Issue - State: open - Opened by pzread over 1 year ago

#53 - Migrate the rest of JAX T5 large benchmarks

Pull Request - State: closed - Opened by pzread over 1 year ago

#52 - Add TF ResNet and artifact generation

Pull Request - State: closed - Opened by mariecwhite over 1 year ago

#51 - Add deployment tool to DB Import

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#50 - Add TF models

Pull Request - State: closed - Opened by mariecwhite over 1 year ago

#49 - Add cloud function entry point

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#48 - [Goal] Show benchmark results in dashboard

Issue - State: open - Opened by pzread over 1 year ago
Labels: goal

#47 - [Migration] Cleanup benchmark ID and name format

Issue - State: closed - Opened by pzread over 1 year ago

#46 - [Goal] Run all iree-samples comparative benchmarks

Issue - State: closed - Opened by pzread over 1 year ago

#45 - [WIP] RFC: Stable Benchmark Identifiers

Pull Request - State: open - Opened by pzread over 1 year ago

#44 - Support multiple raw inputs

Issue - State: open - Opened by mariecwhite over 1 year ago

#43 - Migrate JAX ResNet50 benchmarks

Pull Request - State: closed - Opened by pzread over 1 year ago

#42 - Migrate JAX Bert benchmarks

Pull Request - State: closed - Opened by pzread over 1 year ago

#41 - Finalize benchmark id format

Issue - State: open - Opened by pzread over 1 year ago - 1 comment

#40 - [Migration] Enable CPU benchmarks in workflow

Issue - State: closed - Opened by pzread over 1 year ago - 1 comment

#39 - [Migration] Update dashboard to process results from openxla-benchmark

Issue - State: open - Opened by pzread over 1 year ago - 5 comments

#38 - [Migration] Migrate PyTorch/Inductor benchmark tool

Issue - State: closed - Opened by pzread over 1 year ago - 1 comment

#37 - [Migration] Migrate TF/XLA benchmark tool

Issue - State: closed - Opened by pzread over 1 year ago - 2 comments

#36 - [Migration] Migrate PyTorch models

Issue - State: closed - Opened by pzread over 1 year ago - 1 comment

#35 - [Migration] Migrate all Tensorflow models

Issue - State: closed - Opened by pzread over 1 year ago - 1 comment

#34 - [Migration] Migrate all JAX models

Issue - State: closed - Opened by pzread over 1 year ago

#33 - Generate JAX artifacts

Pull Request - State: closed - Opened by mariecwhite over 1 year ago - 1 comment

#32 - Add download helper tool

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#31 - Fix imports according to style guide

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#30 - Migrate XLA HLO benchmarks

Pull Request - State: closed - Opened by pzread over 1 year ago

#29 - Verify the benchamrk outputs

Pull Request - State: closed - Opened by pzread over 1 year ago

#28 - Add batch import tool to DB import

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#27 - Fetch preprocessed inputs for benchmarking

Pull Request - State: closed - Opened by pzread over 1 year ago

#26 - Add basic file processing to DB import tool

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#25 - Add JAX T5ForConditionalGeneration

Pull Request - State: closed - Opened by mariecwhite over 1 year ago

#24 - Use platform-specific JAX packages

Pull Request - State: closed - Opened by mariecwhite over 1 year ago

#23 - Make name of T5 model configurable

Pull Request - State: closed - Opened by mariecwhite over 1 year ago

#22 - Add setup job to comparative benchmark workflow

Pull Request - State: closed - Opened by pzread over 1 year ago

#21 - Move InMemoryStorage into separate file

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#20 - [Migration] Upload benchmark results to GCS

Issue - State: closed - Opened by pzread over 1 year ago - 2 comments

#19 - Add LocalStorage and InMemoryDatabase

Pull Request - State: closed - Opened by beckerhe over 1 year ago

#18 - Move build_tools/scripts to devtools

Pull Request - State: closed - Opened by pzread over 1 year ago

#17 - Change Python formatter to Black

Issue - State: open - Opened by pzread over 1 year ago

#16 - Add e2e JAX XLA benchmark workflow

Pull Request - State: closed - Opened by pzread over 1 year ago

#15 - Remove unnecessary __init__.py

Pull Request - State: closed - Opened by pzread over 1 year ago - 2 comments

#14 - Fetch dumped input in framework benchmarks

Issue - State: closed - Opened by pzread over 1 year ago - 1 comment

#13 - Add e2e XLA JAX framework benchmarks and tools

Pull Request - State: closed - Opened by pzread over 1 year ago - 1 comment

#12 - Artifact generation process in the benchmark suite

Issue - State: closed - Opened by pzread over 1 year ago - 6 comments

#11 - Implement output verifier on numpy tensor

Issue - State: closed - Opened by pzread over 1 year ago

#10 - Refactor artifact representations of `Model` and `ModelTestData`

Pull Request - State: closed - Opened by pzread over 1 year ago

#9 - Add JAX T5 large models and test data

Pull Request - State: closed - Opened by pzread over 1 year ago

#8 - Add Jsonnet rule engine for DB import

Pull Request - State: closed - Opened by beckerhe over 1 year ago - 1 comment

#7 - Add a first version of the DB import pipeline

Pull Request - State: closed - Opened by beckerhe over 1 year ago - 3 comments

#6 - Initialize benchmark definitions

Pull Request - State: closed - Opened by pzread over 1 year ago

#5 - Add linter scripts and check in workflow

Pull Request - State: closed - Opened by pzread over 1 year ago

#4 - [DNS] Test workflow

Pull Request - State: closed - Opened by pzread over 1 year ago

#3 - Add `.pylintrc`

Pull Request - State: closed - Opened by pzread over 1 year ago

#2 - Support multi-model framework benchmarks

Issue - State: open - Opened by pzread over 1 year ago

#1 - Initialize benchmark definitions

Pull Request - State: closed - Opened by pzread over 1 year ago