empirical-run/empirical issues and pull requests

#173 - fix: timeout error tests

Pull Request - State: closed - Opened by saikatmitra91 7 months ago - 1 comment

#168 - rag example: add run -> change parameter `model` to `gpt-4` -> script times out but no error shown on the UI

Issue - State: closed - Opened by saikatmitra91 7 months ago - 2 comments
Labels: bug, good first issue

#168 - rag example: add run -> change parameter `model` to `gpt-4` -> script times out but no error shown on the UI

Issue - State: closed - Opened by saikatmitra91 7 months ago - 2 comments
Labels: bug, good first issue

#167 - [chatbot example] - CLI summary doesnt show the second score - `personal`

Issue - State: closed - Opened by saikatmitra91 7 months ago - 3 comments
Labels: bug, good first issue

#167 - [chatbot example] - CLI summary doesnt show the second score - `personal`

Issue - State: open - Opened by saikatmitra91 7 months ago - 3 comments
Labels: bug, good first issue

#166 - Model latency numbers include time taken to retry

Issue - State: closed - Opened by saikatmitra91 7 months ago - 2 comments
Labels: bug, good first issue

#166 - Model latency numbers include time taken to retry

Issue - State: closed - Opened by saikatmitra91 7 months ago - 2 comments
Labels: bug, good first issue

#165 - feat: add spider example with 150 samples

Pull Request - State: closed - Opened by arjunattam 7 months ago - 3 comments

#156 - feat: simplify config to accept provider, scorers as top level keys

Pull Request - State: closed - Opened by KaustubhKumar05 7 months ago - 2 comments

#156 - feat: simplify config to accept provider, scorers as top level keys

Pull Request - State: closed - Opened by KaustubhKumar05 7 months ago - 2 comments

#156 - feat: simplify config to accept provider, scorers as top level keys

Pull Request - State: closed - Opened by KaustubhKumar05 7 months ago - 2 comments

#127 - feat(interactivity): support editing for all samples

Pull Request - State: closed - Opened by arjunattam 7 months ago - 1 comment

#116 - [WIP] feat: add duckdb filtering on the browser

Pull Request - State: open - Opened by saikatmitra91 7 months ago - 1 comment

#115 - chore: (minor) Add better error messaging for CLI and improve quickstart docs

Pull Request - State: closed - Opened by aditya-67 7 months ago - 2 comments

#100 - feat: add support for output metadata

Pull Request - State: closed - Opened by saikatmitra91 7 months ago - 1 comment

#99 - fix: replace tabs in sample card with dropdown selector

Pull Request - State: closed - Opened by KaustubhKumar05 7 months ago - 2 comments

#98 - [wip] js-script scorer function

Pull Request - State: closed - Opened by arjunattam 7 months ago - 1 comment

#97 - chore: generate json schema programmatically

Pull Request - State: closed - Opened by arjunattam 7 months ago - 1 comment

#96 - docs: improvements to quick start

Pull Request - State: closed - Opened by arjunattam 7 months ago - 2 comments

#95 - docs: for configuration file, minor fixes elsewhere

Pull Request - State: closed - Opened by arjunattam 7 months ago - 2 comments

#94 - chore: update package versions for release

Pull Request - State: closed - Opened by github-actions[bot] 7 months ago

#93 - chore: use shields for discord badge

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#92 - fix: run config gets hidden on opening after scroll

Pull Request - State: closed - Opened by KaustubhKumar05 8 months ago - 1 comment

#91 - chore: update package versions for release

Pull Request - State: closed - Opened by github-actions[bot] 8 months ago

#90 - feat: show output errors in ui cells

Pull Request - State: closed - Opened by KaustubhKumar05 8 months ago - 2 comments

#89 - fix: add support for alternate port for web UI

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#88 - test: make mistral test less flaky

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#87 - chore: add repo link to package.json files

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#86 - feat: add parameters in run config ui

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#85 - feat: add interactivity to the UI

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#84 - chore: improve error message copy

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#83 - chore(deps): bump pillow from 10.2.0 to 10.3.0 in /examples/rag

Pull Request - State: closed - Opened by dependabot[bot] 8 months ago - 1 comment
Labels: dependencies, python

#82 - chore(deps-dev): bump vite from 5.1.6 to 5.2.8

Pull Request - State: closed - Opened by dependabot[bot] 8 months ago - 1 comment
Labels: dependencies, javascript

#81 - chore(deps): bump langchain-core from 0.1.33 to 0.1.35 in /examples/rag

Pull Request - State: closed - Opened by dependabot[bot] 8 months ago - 1 comment
Labels: dependencies, python

#80 - chore(deps): bump express from 4.18.3 to 4.19.2

Pull Request - State: closed - Opened by dependabot[bot] 8 months ago
Labels: dependencies, javascript

#79 - chore: fix docs for script based runs

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#78 - fix: humaneval tests broke

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#77 - fix: throw openai error response

Pull Request - State: closed - Opened by KaustubhKumar05 8 months ago - 1 comment

#76 - chore: clean up humaneval example

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#75 - fix: top bar is not sticky

Pull Request - State: closed - Opened by KaustubhKumar05 8 months ago - 1 comment

#74 - feat: add support for passthrough script executor config

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#73 - fix: rename py script executor value property to path

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#72 - fix: openai gets stuck when balance is negative

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#71 - docs: add links to ci/cd guide

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#70 - feat: support model parameters

Pull Request - State: closed - Opened by arjunattam 8 months ago - 2 comments

#69 - feat: add fireworks model provider

Pull Request - State: closed - Opened by arjunattam 8 months ago - 2 comments

#68 - chore: update package versions for release

Pull Request - State: closed - Opened by github-actions[bot] 8 months ago

#67 - fix: python scorer should prefer config.name

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#66 - chore: update package versions for release

Pull Request - State: closed - Opened by github-actions[bot] 8 months ago

#65 - fix: reduce bundle size of cli package

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#64 - chore: update package versions for release

Pull Request - State: closed - Opened by github-actions[bot] 8 months ago

#63 - chore: update npm package licenses

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#62 - chore: update license

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#61 - fix: monaco blocking page scroll

Pull Request - State: closed - Opened by KaustubhKumar05 8 months ago - 1 comment

#60 - chore: stop changesets from publishing release by default

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#59 - chore: update package versions for release

Pull Request - State: closed - Opened by github-actions[bot] 8 months ago

#58 - feat: add run summary to run header

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#57 - feat: add llm-faithfulness scorer

Pull Request - State: open - Opened by arjunattam 8 months ago - 1 comment

#56 - feat: support cli runs on github actions

Pull Request - State: closed - Opened by arjunattam 8 months ago - 2 comments

#55 - chore: update package versions for release

Pull Request - State: closed - Opened by github-actions[bot] 8 months ago

#54 - chore: ignore changeset file changes for gh build step

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#53 - chore: add npmignore file in package generators

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#52 - chore: update package versions for release

Pull Request - State: closed - Opened by github-actions[bot] 8 months ago

#51 - chore: fix npm publish script name

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#50 - chore: add npm publish script

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#49 - docs: fix og image

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#48 - feat: pick env vars from dotenv files

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#47 - feat: default name for run if not specified

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#46 - fix: update prompt in basic example and default init config

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#45 - docs: update docs, examples, schema for scorer interface

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#44 - chore: interrupt watch routine when new change is detected

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#43 - fix: support aliases for claude 3 models

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#42 - feat: runs store for a dataset

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#41 - fix: sample card broke after input schema change

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#40 - chore: update pull request events for pr message lint gh action

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#39 - chore: update package versions for release

Pull Request - State: closed - Opened by github-actions[bot] 8 months ago

#38 - ci: fix release version generation gh action

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#37 - feat: add changeset and version publish process

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#36 - feat: add stats summary

Pull Request - State: closed - Opened by saikatmitra91 8 months ago - 1 comment

#35 - feat: simplify inputs to be an object with kv pairs

Pull Request - State: closed - Opened by arjunattam 8 months ago

#34 - chore: remove force build in dev watch mode

Pull Request - State: closed - Opened by saikatmitra91 8 months ago

#33 - feat: open the ui web app when ready

Pull Request - State: closed - Opened by arjunattam 8 months ago

#32 - feat: dataset loaders and auto-fill for missing ids

Pull Request - State: closed - Opened by arjunattam 8 months ago - 1 comment

#31 - docs: color scheme

Pull Request - State: closed - Opened by arjunattam 8 months ago

#30 - feat: add error handling

Pull Request - State: closed - Opened by saikatmitra91 8 months ago

#29 - docs: initialize docs with mintlify and update readme

Pull Request - State: closed - Opened by arjunattam 8 months ago

#28 - feat: add message tooltip in while showing scores in UI

Pull Request - State: closed - Opened by saikatmitra91 8 months ago

#27 - chore: add pr message lint github action

Pull Request - State: closed - Opened by saikatmitra91 8 months ago

#26 - Add RAG pipeline evaluation example and run script executor

Pull Request - State: closed - Opened by saikatmitra91 8 months ago

#25 - feat: clean up interface to accommodate script runs

Pull Request - State: closed - Opened by saikatmitra91 8 months ago

#24 - feat: anthropic provider

Pull Request - State: closed - Opened by KaustubhKumar05 8 months ago

#23 - chore: prepare script for humaneval dataset

Pull Request - State: closed - Opened by arjunattam 8 months ago

#22 - feat: timeout and error reporting for py-script eval

Pull Request - State: closed - Opened by arjunattam 8 months ago

#21 - chore: add lint to gh and package manager version management

Pull Request - State: closed - Opened by saikatmitra91 8 months ago

#20 - fix: humaneval prompt and eval updated

Pull Request - State: closed - Opened by arjunattam 8 months ago

#19 - feat: add ai package

Pull Request - State: closed - Opened by saikatmitra91 8 months ago

#18 - feat: python eval script for humaneval

Pull Request - State: closed - Opened by arjunattam 8 months ago

#17 - chore: clickable link for ui command

Pull Request - State: closed - Opened by arjunattam 8 months ago

#16 - feat: load dataset from files

Pull Request - State: closed - Opened by arjunattam 8 months ago

#15 - feat: add library code generator command

Pull Request - State: closed - Opened by saikatmitra91 8 months ago

GitHub / empirical-run/empirical issues and pull requests