conbench/conbench issues and pull requests

#1419 - benchmark result submission: clarify "iterations", specify allowed unit symbols

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 2 comments

#1418 - benchmark result: after clarified meaning of `iterations` (not repetitions): remove validation

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 1 comment

#1416 - POST benchmark result: do not require `times` and `time_unit` anymore

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 1 comment

#1415 - UI: result-result comparison: show hardware checksum

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 2 comments

#1414 - UI: result-result compare: fix url

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 1 comment

#1413 - UI: result-result comparison: show hardware checksum

Issue - State: closed - Opened by jgehrcke about 2 years ago - 1 comment
Labels: UI/UX

#1412 - hardware considered equal despite obvious differences

Issue - State: open - Opened by jgehrcke about 2 years ago - 3 comments
Labels: bug, measurement method & analysis

#1411 - result-result compare page: right result URL is bad

Issue - State: closed - Opened by jgehrcke about 2 years ago - 1 comment
Labels: bug

#1410 - kube-prom: dynamically set monitored namespaces

Pull Request - State: closed - Opened by jgehrcke about 2 years ago

#1409 - dashboard: saner defaults, new panel for /compare/runs response time distribution

Pull Request - State: closed - Opened by jgehrcke about 2 years ago

#1408 - grafana dash: add 'cluster' and 'velox' support

Pull Request - State: closed - Opened by jgehrcke about 2 years ago

#1407 - UI: c-bench tinyplots: new source of truth for long form of unit

Pull Request - State: closed - Opened by jgehrcke about 2 years ago

#1406 - c-bench UI: trends: increasing/decreasing -> improving/degrading

Issue - State: open - Opened by jgehrcke about 2 years ago

#1405 - benchalerts devprod: add a Makefile target that regenerates the expected_md folder

Issue - State: open - Opened by austin3dickey about 2 years ago - 2 comments
Labels: devprod, benchalerts

#1404 - benchalerts: add benchmark name/language to notifications

Pull Request - State: closed - Opened by austin3dickey about 2 years ago

#1403 - _plots.py: use conbench.units (new source of truth)

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 1 comment

#1402 - Makefile: bump kube-prometheus to d3889807798d

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 1 comment

#1400 - ci: apply k8s ingress, service, servicemon cfg on ursa2

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 1 comment

#1399 - benchmark result stats: make `times` and `time_unit` not required

Issue - State: closed - Opened by jgehrcke about 2 years ago - 1 comment
Labels: api JSON schemas and spec

#1398 - benchmark result: rename `iterations` to `repetitions`

Issue - State: open - Opened by jgehrcke about 2 years ago - 3 comments
Labels: api JSON schemas and spec, UX - terminology

#1397 - history/compare APIs: use new less_is_better source, introduce TUnit type, tweaks/fixes

Pull Request - State: closed - Opened by jgehrcke about 2 years ago

#1396 - UI: tables: dynamically switch to scientific notation, better spacing

Pull Request - State: closed - Opened by jgehrcke about 2 years ago

#1394 - UI/app: obliterate all API layer re-serialization

Issue - State: open - Opened by jgehrcke about 2 years ago
Labels: maintainability

#1393 - UI: Never use `tags.suite` for benchmark display name

Pull Request - State: closed - Opened by alistaire47 about 2 years ago - 1 comment

#1392 - UI: Don't prefer `tags.suite` to `tags.name` for benchmark display name

Issue - State: closed - Opened by alistaire47 about 2 years ago

#1390 - kube-prom deploy: allow for dynamically setting monitored k8s namespaces

Issue - State: closed - Opened by jgehrcke about 2 years ago - 1 comment
Labels: system-monitoring, k8s-and-infra

#1389 - kube-prom: deploy to ursa-2, too

Pull Request - State: closed - Opened by jgehrcke about 2 years ago

#1388 - kube-prom: do not use custom storage class

Pull Request - State: closed - Opened by jgehrcke about 2 years ago

#1386 - http api: compare endpoints: emit 429 when semaphore blocked

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 1 comment

#1385 - http path timeouts: new alignment

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 2 comments

#1384 - http information flow: align timeout constants (client, alb, tcp keepalive, gunicorn worker timeout)

Issue - State: closed - Opened by jgehrcke about 2 years ago - 1 comment
Labels: api latency, k8s-and-infra

#1383 - HTTP API: add endpoint for downloading a timeseries as CSV file

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 1 comment

#1382 - app/api compare endpoint semaphore: return 429 when sem acq fails (immediately or within timeout?)

Issue - State: closed - Opened by jgehrcke about 2 years ago - 3 comments
Labels: api, UX - robustness, error handling

#1381 - compare/run UI: Filter run comparison results by z-score or percent change

Issue - State: open - Opened by alistaire47 about 2 years ago - 2 comments

#1380 - UI: display units for tables (big number treatment)

Issue - State: open - Opened by alistaire47 about 2 years ago - 3 comments
Labels: UI - numbers/sigfig/truncation

#1379 - benchadapt: ArcheryAdapter is reporting suite name for benchmark name

Issue - State: closed - Opened by alistaire47 about 2 years ago - 3 comments

#1378 - hist plot: do not show negative y range

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 5 comments

#1377 - UI: run: make measurement sortable

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 1 comment

#1376 - history plot: generate res-res-cmp URL

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 2 comments

#1374 - history plot: data point hover: show commit msg

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 1 comment

#1373 - benchmark result submission: require one of known units

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 1 comment

#1371 - history plot: link to direct comparison page between two results

Issue - State: closed - Opened by jgehrcke about 2 years ago - 1 comment
Labels: UX - plotting

#1369 - Present result-result comparison neutrally, 'baseline' & 'contender' is confusing

Issue - State: open - Opened by jgehrcke about 2 years ago - 11 comments
Labels: UX - terminology

#1368 - benchalerts: add docstrings to mocked classes

Pull Request - State: closed - Opened by austin3dickey about 2 years ago

#1367 - ci: work around webdriver manager (for issue #1364)

Pull Request - State: closed - Opened by jgehrcke about 2 years ago

#1366 - requirements-webapp: pin gunicorn

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 1 comment

#1365 - moving dependency: gunicorn: `ValueError: Invalid file descriptor: -1`

Issue - State: closed - Opened by jgehrcke about 2 years ago - 3 comments
Labels: bug

#1364 - ci instability: ui-screenshots fails with chromedriver download

Issue - State: closed - Opened by jgehrcke about 2 years ago - 4 comments
Labels: ci-instability

#1362 - benchalerts: switch to new ConbenchClient

Pull Request - State: closed - Opened by austin3dickey about 2 years ago

#1361 - benchalerts: always display all runs in check summary

Pull Request - State: closed - Opened by austin3dickey about 2 years ago - 1 comment

#1360 - z-score calculation: Exclude prior results on same commit

Issue - State: closed - Opened by alistaire47 about 2 years ago - 1 comment

#1359 - result template: Handle null commit

Pull Request - State: closed - Opened by alistaire47 about 2 years ago - 2 comments

#1358 - benchmark result page: error when commit is None

Issue - State: closed - Opened by austin3dickey about 2 years ago

#1357 - UI: c-bench/tinyplots: fix grouping of results into timeseries: use hardware checksum instead of ID

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 3 comments

#1356 - UI: result view: add summary, explain history plot better

Pull Request - State: closed - Opened by jgehrcke about 2 years ago - 2 comments

#1355 - benchmark result view: present measurement result more clearly

Issue - State: closed - Opened by jgehrcke about 2 years ago - 7 comments
Labels: UI - numbers/sigfig/truncation

#1354 - get_or_create(): simplify

Pull Request - State: closed - Opened by austin3dickey about 2 years ago

#1353 - c-bench: tinyplots: use hardware hash for grouping, not hardware ID

Issue - State: closed - Opened by jgehrcke about 2 years ago - 2 comments
Labels: bug

#1351 - benchalerts: always display all runs analyzed

Issue - State: closed - Opened by austin3dickey about 2 years ago

#1350 - benchalerts: consider using tables instead of bulleted lists

Issue - State: open - Opened by austin3dickey about 2 years ago - 3 comments

#1349 - UI: "delete run", "delete benchmark result" buttons are visible even if user is not logged in

Issue - State: closed - Opened by austin3dickey about 2 years ago - 2 comments

#1348 - lookback z-score method: pragmatic changelog: document evolution so far

Issue - State: open - Opened by jgehrcke about 2 years ago - 13 comments

#1347 - UI: table showing results in run: data sorting not done by numerically

Issue - State: closed - Opened by jgehrcke about 2 years ago - 2 comments
Labels: bug

#1345 - compare endpoint: UI, API: make this much faster

Issue - State: open - Opened by jgehrcke about 2 years ago - 5 comments
Labels: api latency, method: lookback z-score, DB schema, datamodel & caching & perf

#1344 - Conbench: refine mission / premise in view of the imperfect nature of the benchmarking/measurement methods

Issue - State: open - Opened by jgehrcke about 2 years ago
Labels: measurement method & analysis

#1341 - UI: show benchmarks in run: remove api layer indirection, use first-class objects

Issue - State: open - Opened by jgehrcke about 2 years ago
Labels: maintainability

#1335 - benchmark result submission: require to pick from a known set of unit strings (for reliable less/more-is-better determination)

Issue - State: closed - Opened by jgehrcke about 2 years ago - 10 comments
Labels: measurement method & analysis

#1253 - UI: hist plot: add "highlight other" feature, compare two benchmark results: show hist plot and highlight other (& misc)

Pull Request - State: closed - Opened by jgehrcke over 2 years ago - 3 comments

GitHub / conbench/conbench issues and pull requests