arthur-ai/bench issues and pull requests

#118 - Bump version to 0.3.2

Pull Request - State: open - Opened by ntatsumi 9 months ago

#117 - dependency issues

Issue - State: open - Opened by ehutt 10 months ago - 3 comments

#116 - update compare table columns for no reference (issue #114)

Pull Request - State: closed - Opened by cheungrowan 10 months ago - 1 comment

#115 - Refactor local client fs operations to new interface

Pull Request - State: closed - Opened by fryz 10 months ago - 1 comment

#114 - Compare test run UI bug for suite with no reference

Issue - State: closed - Opened by max-arthurai 11 months ago - 1 comment

#113 - Add demo scoring Bedrock models for Summarization

Pull Request - State: closed - Opened by fryz 12 months ago

#111 - Bump version to 0.3.1

Pull Request - State: closed - Opened by mcgrawia 12 months ago

#110 - add async methods of llm based scorers

Pull Request - State: closed - Opened by cheungrowan about 1 year ago

#109 - Add async scoring methods

Pull Request - State: closed - Opened by cheungrowan about 1 year ago

#108 - update requirements and bump version

Pull Request - State: closed - Opened by cheungrowan about 1 year ago

#107 - remove duckdb from local client

Pull Request - State: closed - Opened by cheungrowan about 1 year ago

#106 - add sort param to get test run by id

Pull Request - State: closed - Opened by cheungrowan about 1 year ago - 1 comment

#105 - updated score_result field

Pull Request - State: closed - Opened by vivirose about 1 year ago

#104 - patch num test cases fetch

Pull Request - State: closed - Opened by cheungrowan about 1 year ago

#103 - compare test runs

Pull Request - State: closed - Opened by vivirose about 1 year ago

#102 - LLM-818 - Add run id filter to summary endpoint

Pull Request - State: closed - Opened by edinpeter about 1 year ago

#101 - BENCH-148 added support for categorical charts

Pull Request - State: closed - Opened by vivirose about 1 year ago - 1 comment

#100 - Parameters for hedging language are now editable

Pull Request - State: closed - Opened by Mymoza about 1 year ago

#99 - Small fix: broken link for TestSuite SDK in the docs

Pull Request - State: closed - Opened by Mymoza about 1 year ago

#98 - BENCH-316 added scorer info to details modal

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#97 - BENCH-324-styling-fixes

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#96 - add hallucination scorer to docs

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#95 - [BENCH-301] hallucination as scorer in docs

Pull Request - State: closed - Opened by max-arthurai over 1 year ago - 1 comment

#94 - Bench 324 styling fixes

Pull Request - State: closed - Opened by vivirose over 1 year ago - 1 comment

#93 - Bench 316 Scorer Configs

Pull Request - State: closed - Opened by vivirose over 1 year ago - 1 comment

#92 - Update setup.md

Pull Request - State: closed - Opened by PaulMest over 1 year ago

#91 - Release 0.2.3: UI fixes and styling

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#90 - BENCH-305 smooth line chart

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#89 - BENCH-315 styling fixes

Pull Request - State: closed - Opened by vivirose over 1 year ago

#88 - Fix UI breaks when page is refreshed on non-root route

Pull Request - State: closed - Opened by fryz over 1 year ago

#87 - Bench 305 smooth charts

Pull Request - State: closed - Opened by vivirose over 1 year ago - 1 comment

#86 - adding guide for scorer config

Pull Request - State: closed - Opened by hgupta18 over 1 year ago - 1 comment

#85 - fix for scorer type inference on windows os

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#84 - categorical scorer implementation

Pull Request - State: closed - Opened by max-arthurai over 1 year ago - 2 comments

#83 - updated hallucination

Pull Request - State: closed - Opened by max-arthurai over 1 year ago - 1 comment

#82 - updating Discord link

Pull Request - State: closed - Opened by hgupta18 over 1 year ago

#81 - resolve front end dependencies

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#80 - bump version and docs

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#79 - add arthur bench remote logging documentation

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#78 - Added Project.urls to pyproject.toml

Pull Request - State: closed - Opened by packdl over 1 year ago

#77 - UserValueError: cannot reference custom scorer by string. please provide instantiated scorer

Issue - State: closed - Opened by aaprajo over 1 year ago - 2 comments

#76 - add environment variable for toggling remote vs local storage

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#75 - make llm evaluator configurable in qa correctness scorer

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#74 - Word count scorer updates

Pull Request - State: closed - Opened by teresadatta100 over 1 year ago

#73 - Allow summary quality to use custom LLM

Pull Request - State: closed - Opened by maccam912 over 1 year ago - 4 comments

#72 - Styling fixes

Pull Request - State: closed - Opened by vivirose over 1 year ago

#71 - fix broken discord link

Pull Request - State: closed - Opened by max-arthurai over 1 year ago

#70 - Discord Link appears broken

Issue - State: closed - Opened by JoshuaPurtell over 1 year ago - 1 comment

#69 - Add AzureChatOpenAI as an option for summary_quality and other scorers

Issue - State: closed - Opened by MakonDev over 1 year ago - 3 comments

#68 - Type object is not subscriptable

Issue - State: closed - Opened by thinkinsysdev over 1 year ago - 1 comment

#67 - BENCH21- adding context to error messages for test suite creation

Pull Request - State: closed - Opened by teresadatta100 over 1 year ago

#66 - add discord link

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#65 - Added footer and help icon

Pull Request - State: closed - Opened by vivirose over 1 year ago

#64 - increase patch version

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#63 - update read the docs config

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#62 - changed scoring method to scoring

Pull Request - State: closed - Opened by max-arthurai over 1 year ago

#61 - saas language

Pull Request - State: closed - Opened by max-arthurai over 1 year ago

#60 - updating docs requirement

Pull Request - State: closed - Opened by hgupta18 over 1 year ago

#59 - scoring headers properly formatted

Pull Request - State: closed - Opened by max-arthurai over 1 year ago

#58 - add api references and copy buttons, edit theme

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#57 - Docs typo fixes

Pull Request - State: closed - Opened by max-arthurai over 1 year ago

#56 - add warn kwarg to python unit tester

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#55 - release 0.2.0

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#54 - enable pypi and bump version

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#53 - fix tooltip

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#52 - Lint

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#51 - updating links

Pull Request - State: closed - Opened by hgupta18 over 1 year ago

#50 - updating links in readme

Pull Request - State: closed - Opened by hgupta18 over 1 year ago

#49 - adding readthedocs

Pull Request - State: closed - Opened by hgupta18 over 1 year ago

#48 - Black flake 8

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#47 - move exceptions to own module

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#46 - Naming makeover

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#45 - reduce scorer warnings

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#44 - [LLM-496 & LLM-497] Error handling for hallucination and Summary quality clean prompt

Pull Request - State: closed - Opened by max-arthurai over 1 year ago - 1 comment

#43 - remove extra field discrepancy with REST API

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#42 - [Draft] Docs cleanup

Pull Request - State: closed - Opened by max-arthurai over 1 year ago

#41 - Add GitHub pipelines

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#40 - UI patches and automation

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#39 - [LLM-498] Check for existing run

Pull Request - State: closed - Opened by max-arthurai over 1 year ago - 1 comment

#38 - adding hedging language demo

Pull Request - State: closed - Opened by hgupta18 over 1 year ago

#37 - scoring config logging

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#36 - updating gitignore

Pull Request - State: closed - Opened by hgupta18 over 1 year ago

#35 - Getting started ux

Pull Request - State: closed - Opened by cheungrowan over 1 year ago - 1 comment

#34 - Update UI

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#33 - adding specificity metric

Pull Request - State: closed - Opened by teresadatta100 over 1 year ago

#32 - Code cleanup

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#31 - Delete LICENSE

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#30 - beginning support for generation

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#29 - Add context to run result

Pull Request - State: closed - Opened by karthikrao6d over 1 year ago

#28 - Scoring method configurability

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#27 - Merge pull request #26 from arthur-ai/main

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#26 - sync develop with main

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#25 - add link to docs and patch bug if using a directory that doesn't exist

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#24 - release

Pull Request - State: closed - Opened by cheungrowan over 1 year ago

#23 - Adding hedging language scoring method

Pull Request - State: closed - Opened by hgupta18 over 1 year ago

#21 - Python unit testing scoring method

Pull Request - State: closed - Opened by max-arthurai over 1 year ago

#20 - WIP Fix model bugs

Pull Request - State: closed - Opened by andrew-arthurai over 1 year ago

#19 - [LLM-370] Hallucination scoring method

Pull Request - State: closed - Opened by max-arthurai over 1 year ago

#16 - adding Reading Ease and Word Count metrics

Pull Request - State: closed - Opened by teresadatta100 over 1 year ago

#11 - [Draft] removed langchain as dependency from bench scoring methods

Pull Request - State: closed - Opened by max-arthurai over 1 year ago

GitHub / arthur-ai/bench issues and pull requests