conbench/conbench issues and pull requests

#832 - BenchmarkResult schema: reconcile, simplify `error` vs `validation`

Issue - State: open - Opened by jgehrcke over 2 years ago - 5 comments
Labels: api JSON schemas and spec

#823 - BenchmarkResultCreate schema: rename, add constraints, remove constraints

Issue - State: open - Opened by jgehrcke over 2 years ago - 4 comments
Labels: api JSON schemas and spec

#817 - db schema: Commit object: require hash and repository specifier?

Issue - State: closed - Opened by jgehrcke over 2 years ago - 8 comments
Labels: DB schema, git repository awareness

#815 - ci instability: ReadTimeoutError during pip / installation from PyPI

Issue - State: closed - Opened by jgehrcke over 2 years ago - 1 comment
Labels: ci-instability

#800 - client HTTP tooling: plan for retryable errors, set specific goal: how long to retry before accepting data loss (robustness in view of micro outage, maintenance, etc)

Issue - State: open - Opened by jgehrcke over 2 years ago - 7 comments
Labels: UX - robustness

#794 - ci instability: prometheus-k8s-0 stuck in Pending (timeout waiting for condition on pods/prometheus-k8s-0)

Issue - State: open - Opened by austin3dickey over 2 years ago - 9 comments
Labels: ci-instability

#790 - db schema: store insertion/creation timestamp for entities

Issue - State: open - Opened by jgehrcke over 2 years ago - 2 comments
Labels: DB schema

#789 - DB schema: optimize for tail view, keep track of insertion order for entities (with index, ideally via primary key)

Issue - State: closed - Opened by jgehrcke over 2 years ago - 6 comments
Labels: DB schema

#786 - api: GET /api/benchmarks/ -- improve specification, make it responsive (less DoSsy)

Issue - State: closed - Opened by jgehrcke over 2 years ago - 6 comments
Labels: api latency, api JSON schemas and spec

#781 - docs: list prior art, enumerate differentiators, link to literature

Issue - State: open - Opened by jgehrcke over 2 years ago - 3 comments
Labels: documentation

#774 - benchalerts: allow for conditional pipeline steps

Issue - State: open - Opened by austin3dickey over 2 years ago
Labels: benchalerts

#748 - benchalerts: tests: re-enable integration tests

Issue - State: closed - Opened by austin3dickey over 2 years ago
Labels: benchalerts

#746 - Streamlining API calls for performance (faster&smaller HTTP responses via filters etc)

Issue - State: closed - Opened by boshek over 2 years ago - 7 comments
Labels: api, api latency

#735 - Review API authentication method for machines/robots/automation

Issue - State: open - Opened by jgehrcke over 2 years ago - 1 comment
Labels: api, security & access control

#720 - Add integration tests for libraries

Issue - State: open - Opened by austin3dickey over 2 years ago - 3 comments
Labels: testing

#691 - ui: history plot: distribution change button: document how this affects data analysis

Issue - State: closed - Opened by jgehrcke over 2 years ago - 2 comments
Labels: UI/UX, measurement method & analysis, method: lookback z-score

#682 - Why does the comparison use "forked from default at" instead of "commit (SHA)" for labelling commit SHAs?

Issue - State: open - Opened by jonkeane over 2 years ago - 5 comments
Labels: time evolution & code evolution

#681 - db.py: catch another variant of 'already exists'

Pull Request - State: closed - Opened by jgehrcke over 2 years ago - 2 comments

#674 - ci: use secrets.GITHUB_TOKEN again for auto-assign

Pull Request - State: closed - Opened by jgehrcke over 2 years ago - 5 comments

#669 - populate_local_conbench: add distribution changes to fake data

Issue - State: open - Opened by austin3dickey over 2 years ago - 1 comment

#658 - _inspect_for_multisample(): unexpected argument: case hit

Issue - State: closed - Opened by jgehrcke over 2 years ago - 4 comments
Labels: maintainability

#644 - Introduce regression candidate detection test framework

Issue - State: open - Opened by jgehrcke over 2 years ago - 3 comments
Labels: measurement method & analysis

#640 - Use "best run" as the "raw" data when calculating z-scores

Issue - State: closed - Opened by austin3dickey over 2 years ago - 3 comments
Labels: measurement method & analysis, method: lookback z-score

#633 - discuss pros and cons for various 'history plot' methods

Issue - State: open - Opened by jgehrcke over 2 years ago - 3 comments
Labels: measurement method & analysis, UX - plotting, method: lookback z-score, time evolution & code evolution

#631 - BenchmarkCreate: required field named `optional_info` (hardware)

Issue - State: open - Opened by jgehrcke over 2 years ago
Labels: api JSON schemas and spec

#617 - legacy CLI / webapp entanglement: separate legacy CLI into isolated package

Issue - State: closed - Opened by austin3dickey over 2 years ago - 7 comments
Labels: devprod, maintainability

#611 - history plot: remove slider, tweak spacing & ci: take screenshot of benchmark result view

Pull Request - State: closed - Opened by jgehrcke over 2 years ago - 7 comments

#607 - run_name: deprecate? document intent, adjust DB schema, remove decision-making burden from users

Issue - State: open - Opened by austin3dickey over 2 years ago - 16 comments
Labels: documentation, api JSON schemas and spec, DB schema

#594 - data interpretation: start emphasizing the meaning of best-case outcome (minimum duration)

Issue - State: open - Opened by jgehrcke over 2 years ago - 2 comments
Labels: measurement method & analysis

#578 - think about notification architecture for errors, instabilities, regressions, etc

Issue - State: open - Opened by jgehrcke over 2 years ago - 4 comments
Labels: UI/UX, DB schema, datamodel & caching & perf

#539 - Conbench client > Can methods return objects

Issue - State: closed - Opened by ElenaHenderson over 2 years ago - 2 comments

#538 - conbench cli: show result timing overview

Pull Request - State: closed - Opened by jgehrcke over 2 years ago - 3 comments
Labels: UI - numbers/sigfig/truncation

#533 - input validation: require consistency between `data`, `iterations`, and aggregates (`stdev` etc)

Issue - State: open - Opened by jgehrcke over 2 years ago - 6 comments
Labels: measurement method & analysis, api JSON schemas and spec

#530 - plotting and analysis: derive per-sample uncertainty from N repetitions, show error bars / distr / box plot / violine plot

Issue - State: open - Opened by jgehrcke over 2 years ago - 11 comments
Labels: measurement method & analysis

#462 - OIDC: carry target URL across flow

Pull Request - State: closed - Opened by jgehrcke almost 3 years ago - 3 comments

#460 - benchmark details: auto-render URL lists as `a href`

Issue - State: closed - Opened by jgehrcke almost 3 years ago - 1 comment
Labels: UI/UX

#415 - Adapter for pytest-benchmark

Pull Request - State: closed - Opened by alistaire47 almost 3 years ago - 2 comments
Labels: benchadapt

#407 - Move info from info.tags into new benchmark_results.info field because info field for benchmark results is unique to each benchmark_results record

Issue - State: closed - Opened by ElenaHenderson almost 3 years ago - 5 comments
Labels: DB schema

GitHub / conbench/conbench issues and pull requests