Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / aai-institute/nnbench issues and pull requests

#146 - Safe setting of CPU Info

Pull Request - State: closed - Opened by maxmynter 2 months ago

#144 - Add nnbench video to readme

Pull Request - State: closed - Opened by maxmynter 2 months ago - 1 comment

#143 - Explore a CLI workflow for usage in ML pipelines and CI

Issue - State: open - Opened by nicholasjng 6 months ago
Labels: enhancement, help wanted

#142 - Stabilize the record IO interface

Issue - State: open - Opened by nicholasjng 6 months ago
Labels: help wanted, refactor

#141 - Add small parameter representation guide for the benchmark runner

Issue - State: open - Opened by nicholasjng 6 months ago
Labels: documentation, enhancement

#140 - Add `transform` slot to record IO types

Issue - State: open - Opened by nicholasjng 6 months ago
Labels: enhancement, refactor

#139 - (fix) Change default == value to is

Pull Request - State: closed - Opened by maxmynter 6 months ago

#138 - Investigate binding params to benchmarks with weakrefs

Issue - State: open - Opened by nicholasjng 6 months ago
Labels: enhancement

#137 - Implement memo garbage collection

Pull Request - State: closed - Opened by nicholasjng 6 months ago
Labels: enhancement

#136 - Update dev requirements, add Memray artifacts to gitignore

Pull Request - State: closed - Opened by nicholasjng 6 months ago

#135 - Organize `nnbench.types.types` into different files

Pull Request - State: closed - Opened by maxmynter 6 months ago

#134 - Add memo followups to the user guide

Issue - State: closed - Opened by nicholasjng 6 months ago
Labels: documentation

#133 - Update HuggingFace example to use memoization

Pull Request - State: closed - Opened by nicholasjng 6 months ago
Labels: documentation

#132 - Hint all `name` arguments as `str` only

Issue - State: closed - Opened by nicholasjng 6 months ago - 1 comment
Labels: refactor

#131 - Supplying `nnbench.Transform`s directly to record IO

Issue - State: open - Opened by nicholasjng 6 months ago
Labels: enhancement, refactor

#130 - Add global memo cache and integrate with the setUp and teardown injection

Pull Request - State: closed - Opened by maxmynter 6 months ago - 1 comment

#129 - Add new memoization guide

Pull Request - State: closed - Opened by nicholasjng 6 months ago - 1 comment
Labels: documentation

#128 - Update Artifact Tutorial

Issue - State: closed - Opened by Maciej818 6 months ago

#127 - State injection into setup and teardown

Pull Request - State: closed - Opened by maxmynter 6 months ago

#126 - Add global memo cache and eviction API

Issue - State: closed - Opened by nicholasjng 6 months ago
Labels: enhancement

#124 - Add transform submodule, parameter compression transform

Pull Request - State: closed - Opened by nicholasjng 6 months ago
Labels: documentation, refactor

#123 - Rollforward of lakefs-spec doc changes

Pull Request - State: closed - Opened by nicholasjng 6 months ago

#122 - Parameter representations instead of parameters in benchmark records

Issue - State: closed - Opened by nicholasjng 6 months ago - 3 comments
Labels: urgent, refactor

#121 - Enhance API Reference Documentation for nnbench

Issue - State: closed - Opened by leonpawelzik 6 months ago - 1 comment
Labels: documentation, enhancement

#120 - Feature: Thunks as an alternative to artifacts

Pull Request - State: closed - Opened by nicholasjng 6 months ago
Labels: refactor

#119 - Check loaded module names before importing a new one from a file

Pull Request - State: closed - Opened by nicholasjng 6 months ago - 1 comment

#118 - Make `Transform` an abstract base class, delete iapply method ...

Pull Request - State: closed - Opened by nicholasjng 6 months ago
Labels: refactor

#117 - Improve base `Transform` interface

Issue - State: closed - Opened by nicholasjng 6 months ago
Labels: refactor

#116 - `functools.partial`ize the `nnbench.parametrize` decorators.

Pull Request - State: closed - Opened by maxmynter 6 months ago - 1 comment

#115 - Module path construction causes typecheck fails

Issue - State: closed - Opened by nicholasjng 6 months ago
Labels: bug

#114 - Redesign artifact concept

Issue - State: closed - Opened by nicholasjng 6 months ago - 1 comment

#113 - Add transform proposal

Pull Request - State: closed - Opened by nicholasjng 6 months ago - 1 comment

#112 - Unique keys

Pull Request - State: closed - Opened by maxmynter 6 months ago

#111 - Inline the `_cleanup` function for the `FilePathArtifactLoader`.

Pull Request - State: closed - Opened by maxmynter 6 months ago - 2 comments

#110 - Add checksum verification to artifacts and loaders

Pull Request - State: closed - Opened by nicholasjng 6 months ago - 3 comments

#109 - Remove the `ArtifactCollection` class

Pull Request - State: closed - Opened by maxmynter 6 months ago

#108 - Remove `ArtifactCollection` Wrapper

Issue - State: closed - Opened by maxmynter 6 months ago

#107 - Add Artifacts guide

Pull Request - State: closed - Opened by maxmynter 6 months ago

#106 - Docs: Versioning considerations

Issue - State: open - Opened by AdrianoKF 6 months ago
Labels: documentation, enhancement

#105 - Set up best practice for easy record serialization + parametrization + reproducibility

Issue - State: closed - Opened by nicholasjng 6 months ago - 1 comment
Labels: urgent

#104 - Run pre-commit autoupdate, bump dependencies

Pull Request - State: closed - Opened by nicholasjng 6 months ago

#103 - Add Parameters to Benchmarks

Pull Request - State: closed - Opened by maxmynter 7 months ago

#102 - Hotfix: Adjust snippet paths

Pull Request - State: closed - Opened by nicholasjng 7 months ago - 1 comment
Labels: documentation

#101 - Benchmark results do not contain parameters

Issue - State: closed - Opened by nicholasjng 7 months ago
Labels: good first issue, urgent

#100 - Implement a FSspec based ArtifactLoader

Pull Request - State: closed - Opened by maxmynter 7 months ago

#99 - Improve ensure unique keys in `Contexts` upon merge.

Issue - State: closed - Opened by maxmynter 7 months ago

#98 - Add a lightning demo to the README.md

Pull Request - State: closed - Opened by nicholasjng 7 months ago - 3 comments

#97 - Add common ArtifactLoaders to make nnbench provide them out of the box

Issue - State: closed - Opened by maxmynter 7 months ago - 1 comment

#96 - Make `nnbench.parametrize` work with `functools.partial`

Issue - State: closed - Opened by maxmynter 7 months ago
Labels: enhancement

#95 - User guide: benchmarking on saved artifacts

Issue - State: closed - Opened by Maciej818 7 months ago

#94 - Implement fsspec artifact loader for nnbench

Issue - State: closed - Opened by Maciej818 7 months ago

#93 - Run `pre-commit autoupdate`, upgrade requirement lockfiles

Pull Request - State: closed - Opened by nicholasjng 7 months ago

#92 - Example Guide: Benchmark on Saved Models

Pull Request - State: closed - Opened by maxmynter 7 months ago - 2 comments

#91 - Add duckDB example

Pull Request - State: closed - Opened by nicholasjng 7 months ago - 1 comment
Labels: documentation

#90 - Restructure test suite and add file IO roundtrip tests

Pull Request - State: closed - Opened by nicholasjng 7 months ago - 1 comment
Labels: tests

#89 - Switch author and maintainer description in example .toml's

Pull Request - State: closed - Opened by maxmynter 7 months ago

#87 - Add parquet, ndjson drivers, change registration mechanisms

Pull Request - State: closed - Opened by nicholasjng 7 months ago

#85 - Add BQ example to showcase cloud streaming of benchmark records

Pull Request - State: closed - Opened by nicholasjng 7 months ago
Labels: documentation

#84 - Add compression algorithms facility to file IO

Pull Request - State: closed - Opened by nicholasjng 7 months ago
Labels: enhancement

#83 - Unify batching in file drivers, remove read/write stubs on base reporter

Pull Request - State: closed - Opened by nicholasjng 7 months ago
Labels: refactor

#81 - Streamlit Integration

Pull Request - State: closed - Opened by maxmynter 7 months ago - 1 comment

#80 - Support compression for file reading and writing

Issue - State: closed - Opened by nicholasjng 7 months ago
Labels: enhancement

#79 - Add BenchmarkContext Class

Pull Request - State: closed - Opened by maxmynter 7 months ago

#78 - Feature/file reporter

Pull Request - State: open - Opened by Hrsj123 7 months ago - 2 comments

#77 - Improve some aspects on benchmark reporters

Pull Request - State: closed - Opened by nicholasjng 7 months ago

#76 - Inline console reporter into BenchmarkReporter.display()

Pull Request - State: closed - Opened by nicholasjng 7 months ago - 1 comment

#75 - Add barebones duckDB reporter

Pull Request - State: closed - Opened by nicholasjng 7 months ago

#74 - Added IOMixin for reporter

Pull Request - State: open - Opened by Hrsj123 7 months ago - 2 comments

#73 - Orchestrator examples

Pull Request - State: closed - Opened by maxmynter 7 months ago

#72 - Add proper `Context` object, manipulation methods

Issue - State: closed - Opened by nicholasjng 7 months ago - 1 comment

#71 - Change `nnbench.reporter` file to submodule, add base reporter

Pull Request - State: closed - Opened by nicholasjng 7 months ago - 1 comment

#70 - Fix runner collection by checking file path first

Pull Request - State: closed - Opened by nicholasjng 7 months ago

#69 - Benchmark discovery fails if "./" is not prepended to a relative path

Issue - State: closed - Opened by nicholasjng 7 months ago
Labels: bug, urgent

#68 - Simplify nullcols helper, flatten nested context

Pull Request - State: closed - Opened by nicholasjng 7 months ago

#67 - Add example: Integration with workflow engines

Issue - State: closed - Opened by Maciej818 7 months ago
Labels: documentation

#66 - Add example: Streaming results to a database

Issue - State: closed - Opened by Maciej818 7 months ago - 2 comments
Labels: documentation

#65 - Group context structs under top-level semantic keys

Pull Request - State: closed - Opened by nicholasjng 8 months ago

#64 - Add git info and Python package info providers

Pull Request - State: closed - Opened by nicholasjng 8 months ago - 1 comment
Labels: enhancement

#63 - Add example: Implement benchmarking on saved artifacts

Issue - State: closed - Opened by Maciej818 8 months ago - 1 comment
Labels: documentation

#61 - Fix dev status classifier

Pull Request - State: closed - Opened by nicholasjng 8 months ago

#60 - Final docs changes in preparation for v0.1.0

Pull Request - State: closed - Opened by nicholasjng 8 months ago
Labels: documentation

#59 - Add more default information to benchmark JSON result

Pull Request - State: closed - Opened by nicholasjng 8 months ago
Labels: enhancement

#58 - Check type of parametrized argument to avoid mismatch

Pull Request - State: closed - Opened by nicholasjng 8 months ago - 1 comment

#57 - Add customization user guide

Pull Request - State: closed - Opened by nicholasjng 8 months ago
Labels: documentation

#56 - Add some default context values and benchmark metadata

Issue - State: closed - Opened by nicholasjng 8 months ago - 1 comment
Labels: enhancement

#55 - Add benchmark runners user guide

Pull Request - State: closed - Opened by maxmynter 8 months ago - 1 comment
Labels: documentation

#54 - Add test coverage

Pull Request - State: closed - Opened by maxmynter 8 months ago
Labels: tests

#53 - Finalize names

Pull Request - State: closed - Opened by maxmynter 8 months ago

#52 - Define and Implement abstract `BenchmarkReporter` Interface.

Pull Request - State: closed - Opened by maxmynter 8 months ago

#51 - Add namegen argument to `@nnbench.{parametrize,product}`

Pull Request - State: closed - Opened by nicholasjng 8 months ago - 2 comments

#50 - Add test coverage for nnbench core abstractions

Issue - State: closed - Opened by nicholasjng 8 months ago
Labels: urgent, tests

#49 - Finalize names of user-facing abstract classes

Issue - State: closed - Opened by nicholasjng 8 months ago
Labels: good first issue, urgent

#48 - Add flexible base reporters for file IO, web, and DBs

Issue - State: open - Opened by nicholasjng 8 months ago - 3 comments
Labels: enhancement

#47 - Define abstract `BenchmarkReporter` interface

Issue - State: closed - Opened by nicholasjng 8 months ago
Labels: enhancement, urgent

#46 - Remove the test PyPI job entirely

Pull Request - State: closed - Opened by nicholasjng 8 months ago