Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / athina-ai/athina-evals issues and pull requests

#120 - Chat history being json dumped in custom prompt

Pull Request - State: closed - Opened by vivek-athina 3 days ago - 1 comment
Labels: published

#119 - Added textdistance libraries

Pull Request - State: closed - Opened by vivek-athina 4 days ago - 1 comment
Labels: published

#118 - Added support for structured outputs

Pull Request - State: closed - Opened by vivek-athina 4 days ago - 1 comment
Labels: published

#117 - App as dynamic column

Pull Request - State: closed - Opened by vivek-athina 6 days ago - 1 comment
Labels: published

#116 - Added support for return citations and bumped litellm version

Pull Request - State: closed - Opened by vivek-athina 11 days ago - 1 comment
Labels: published

#115 - Release/1-5-18

Pull Request - State: closed - Opened by vivek-athina 16 days ago - 1 comment
Labels: published

#114 - Release/1-5-16

Pull Request - State: closed - Opened by vivek-athina 16 days ago - 1 comment
Labels: published

#113 - Vivek/ath 2167 dynamic column openai assistant

Pull Request - State: closed - Opened by vivek-athina 17 days ago - 1 comment
Labels: published

#112 - Release/1-5-15

Pull Request - State: closed - Opened by vivek-athina 18 days ago - 1 comment
Labels: published

#111 - Release/1-5-14

Pull Request - State: closed - Opened by vivek-athina 20 days ago - 1 comment
Labels: published

#110 - Release/1-5-13

Pull Request - State: closed - Opened by vivek-athina 23 days ago - 1 comment
Labels: published

#109 - feat: separating concepts for llm service

Pull Request - State: closed - Opened by DanielAthina 24 days ago - 2 comments

#108 - feat: metadata response

Pull Request - State: closed - Opened by DanielAthina 25 days ago - 1 comment

#107 - Release-1.5.12 (perplexity failed to load json from text in evals)

Pull Request - State: closed - Opened by vivek-athina about 1 month ago - 1 comment
Labels: published

#106 - Release-1.5.11

Pull Request - State: closed - Opened by akshat-g about 2 months ago - 1 comment
Labels: published

#105 - Release-1.5.10

Pull Request - State: closed - Opened by vivek-athina about 2 months ago - 1 comment
Labels: published

#104 - Add multimodal support for llm.py in PromptTemplate and PromptExecution

Pull Request - State: open - Opened by arunankathina about 2 months ago - 2 comments

#103 - modify run batch async ATH-1904

Pull Request - State: closed - Opened by akshat-g about 2 months ago - 1 comment

#102 - modify langchain openai versions ATH-1904

Pull Request - State: closed - Opened by akshat-g about 2 months ago - 1 comment

#101 - Release-1.5.9 (Bug fix in api call)

Pull Request - State: closed - Opened by vivek-athina about 2 months ago - 2 comments
Labels: published

#100 - Bug fix in api call

Pull Request - State: closed - Opened by vivek-athina about 2 months ago - 1 comment

#99 - An error occurred while posting eval results

Issue - State: open - Opened by DsDastgheib about 2 months ago
Labels: bug

#98 - Upgrade Ragas, LlamaIndex and Langchain to the latest versions

Pull Request - State: closed - Opened by akshat-g about 2 months ago - 1 comment

#97 - Remove urlencode for params as request library already does it

Pull Request - State: closed - Opened by vivek-athina 2 months ago - 2 comments

#96 - Vivek/ath 1698 fix json validation eval

Pull Request - State: closed - Opened by vivek-athina 2 months ago - 1 comment

#94 - add weaviate retrieval step

Pull Request - State: closed - Opened by akshat-g 2 months ago - 1 comment

#93 - Upgrades to API step and version bump

Pull Request - State: closed - Opened by vivek-athina 2 months ago - 2 comments

#92 - API step improved

Pull Request - State: closed - Opened by vivek-athina 2 months ago - 1 comment

#91 - add support for custom models for ragas ATH-1721

Pull Request - State: closed - Opened by akshat-g 2 months ago - 1 comment

#89 - Error handling in dynamic columns

Pull Request - State: closed - Opened by vivek-athina 3 months ago - 1 comment

#88 - Tools support in evals

Pull Request - State: closed - Opened by vivek-athina 3 months ago - 1 comment

#87 - Minor fix for error message in JSONSchema Eval

Pull Request - State: closed - Opened by vivek-athina 3 months ago - 1 comment

#86 - fix ragas aspect critique issues ATH-1641

Pull Request - State: closed - Opened by akshat-g 3 months ago - 1 comment

#85 - Version fixes

Pull Request - State: closed - Opened by vivek-athina 3 months ago - 1 comment

#84 - Fix/litellm version specified

Pull Request - State: closed - Opened by vivek-athina 3 months ago - 1 comment

#83 - bump json schema version

Pull Request - State: closed - Opened by akshat-g 3 months ago - 1 comment

#82 - non boolean custom evals

Pull Request - State: closed - Opened by akshat-g 3 months ago - 1 comment

#81 - Added re package support in custom code. Bumped version

Pull Request - State: closed - Opened by vivek-athina 3 months ago - 1 comment

#80 - allow 5000 rows

Pull Request - State: open - Opened by akshat-g 3 months ago - 1 comment

#79 - Improved custom code handling

Pull Request - State: closed - Opened by vivek-athina 3 months ago - 1 comment

#78 - Fix ragas aspect critque metrics

Pull Request - State: closed - Opened by akshat-g 3 months ago - 1 comment

#77 - Minor fixes in import

Pull Request - State: closed - Opened by vivek-athina 4 months ago - 1 comment

#76 - JsonEval split into two evals

Pull Request - State: closed - Opened by vivek-athina 4 months ago - 1 comment

#75 - Custom code eval

Pull Request - State: closed - Opened by vivek-athina 4 months ago - 1 comment

#74 - Added code execution and jsonpath extract steps

Pull Request - State: closed - Opened by vivek-athina 4 months ago - 1 comment

#73 - add chat history to faifulness

Pull Request - State: closed - Opened by akshat-g 4 months ago - 1 comment

#72 - fix logger issues

Pull Request - State: closed - Opened by akshat-g 4 months ago - 1 comment

#71 - Fix/add conversation coherence eval

Pull Request - State: closed - Opened by akshat-g 4 months ago - 1 comment

#70 - Added support to pass dataset id in run eval suite flow

Pull Request - State: closed - Opened by vivek-athina 4 months ago - 1 comment

#69 - remove supported models check

Pull Request - State: closed - Opened by akshat-g 4 months ago - 1 comment

#68 - Fix/json eval text validation reason

Pull Request - State: closed - Opened by vivek-athina 4 months ago - 1 comment

#67 - Added reason to JsonEval validations

Pull Request - State: closed - Opened by vivek-athina 4 months ago - 1 comment

#66 - Fix for boolean evals to return 0 or 1

Pull Request - State: closed - Opened by vivek-athina 4 months ago - 1 comment

#65 - Added JsonEval

Pull Request - State: closed - Opened by vivek-athina 4 months ago - 1 comment

#64 - modify prompt execution step

Pull Request - State: closed - Opened by akshat-g 4 months ago - 1 comment

#63 - Added more guardrail evals and a few function evals. Fixed equals eval.

Pull Request - State: closed - Opened by vivek-athina 4 months ago - 1 comment

#62 - Add retrieval nodes

Pull Request - State: closed - Opened by akshat-g 4 months ago - 1 comment

#61 - Support double curly brace in custom prompt

Pull Request - State: closed - Opened by vivek-athina 4 months ago - 1 comment

#60 - Added new marvin nodes for classification and extracting entities

Pull Request - State: closed - Opened by vivek-athina 4 months ago - 1 comment

#59 - added jinja parsing in custom eval prompt templating

Pull Request - State: closed - Opened by vivek-athina 5 months ago - 1 comment

#58 - Added basic classes for steps and chains along with examples

Pull Request - State: closed - Opened by shivsak 5 months ago - 1 comment

#57 - dataset migration

Pull Request - State: closed - Opened by akshat-g 5 months ago - 1 comment

#56 - added intent evaluator

Pull Request - State: open - Opened by vyelluru 5 months ago - 1 comment

#55 - added gpt-4o and gpt4-32k language models

Pull Request - State: closed - Opened by vivek-athina 5 months ago - 1 comment

#54 - added newer llama models

Pull Request - State: closed - Opened by vivek-athina 5 months ago - 1 comment

#53 - Feature/sdk run evals dataset

Pull Request - State: closed - Opened by akshat-g 5 months ago - 1 comment

#52 - add chat history to context contains enough information evaluator ATH-1029

Pull Request - State: closed - Opened by akshat-g 5 months ago - 1 comment

#51 - Bugfix/context similarity eval

Pull Request - State: closed - Opened by vivek-athina 6 months ago - 1 comment

#50 - Vivek/ath 1124 bug fixes in safety evals

Pull Request - State: closed - Opened by vivek-athina 6 months ago - 1 comment

#49 - mistral large model name fixed and version bump

Pull Request - State: closed - Opened by vivek-athina 6 months ago - 1 comment

#48 - added guardrail and safety evals to be run from samplerun

Pull Request - State: closed - Opened by vivek-athina 6 months ago - 1 comment

#47 - added newer models

Pull Request - State: closed - Opened by vivek-athina 6 months ago - 1 comment

#45 - Feature/develop flow refactor

Pull Request - State: closed - Opened by akshat-g 6 months ago - 1 comment

#44 - Feature/develop flow

Pull Request - State: closed - Opened by akshat-g 6 months ago - 2 comments

#43 - fixing athina init

Pull Request - State: closed - Opened by vivek-athina 6 months ago - 1 comment

#42 - adding convenience wrapper for eval suite run

Pull Request - State: closed - Opened by vivek-athina 6 months ago - 1 comment

#41 - add kwargs to grading criteria

Pull Request - State: closed - Opened by akshat-g 6 months ago - 1 comment

#40 - guard rail evals(gibberish, nsfw, sensitive topics)

Pull Request - State: closed - Opened by vivek-athina 6 months ago - 1 comment

#39 - Content Moderation Eval Added

Pull Request - State: closed - Opened by vivek-athina 6 months ago - 1 comment

#38 - Removed additional kwargs in functions

Pull Request - State: closed - Opened by vivek-athina 6 months ago - 1 comment

#37 - renamed regex to pattern

Pull Request - State: closed - Opened by vivek-athina 6 months ago - 1 comment

#36 - Added fix for summarizationHal

Pull Request - State: closed - Opened by frostbite127 6 months ago - 1 comment

#33 - bug fix in preprocess strings

Pull Request - State: closed - Opened by vivek-athina 7 months ago - 1 comment

#32 - Conversation Evals Code Cleanup

Pull Request - State: closed - Opened by shivsak 7 months ago - 1 comment

#31 - fixed api call eval and grounded eval test runs

Pull Request - State: closed - Opened by vivek-athina 7 months ago - 1 comment

#30 - ATH-841 ATH-842

Pull Request - State: closed - Opened by akshat-g 7 months ago - 1 comment

#29 - added missing models

Pull Request - State: closed - Opened by vivek-athina 7 months ago - 1 comment

#28 - fixed api call function args

Pull Request - State: closed - Opened by vivek-athina 7 months ago - 1 comment

#27 - API Call eval updated

Pull Request - State: closed - Opened by vivek-athina 7 months ago - 1 comment

#26 - add failure threshold logic

Pull Request - State: closed - Opened by akshat-g 7 months ago - 1 comment

#25 - Added functions to get evaluator from eval name

Pull Request - State: closed - Opened by frostbite127 7 months ago - 1 comment

#22 - Bug fix in eval suite run

Pull Request - State: closed - Opened by vivek-athina 8 months ago - 1 comment

#21 - added cohere models to Model enum. remove unused dependency

Pull Request - State: closed - Opened by vivek-athina 8 months ago - 2 comments