Ecosyste.ms: Issues
An open API service for providing issue and pull request metadata for open source projects.
GitHub / athina-ai/athina-evals issues and pull requests
#120 - Chat history being json dumped in custom prompt
Pull Request -
State: closed - Opened by vivek-athina 3 days ago
- 1 comment
Labels: published
#119 - Added textdistance libraries
Pull Request -
State: closed - Opened by vivek-athina 4 days ago
- 1 comment
Labels: published
#118 - Added support for structured outputs
Pull Request -
State: closed - Opened by vivek-athina 4 days ago
- 1 comment
Labels: published
#117 - App as dynamic column
Pull Request -
State: closed - Opened by vivek-athina 7 days ago
- 1 comment
Labels: published
#116 - Added support for return citations and bumped litellm version
Pull Request -
State: closed - Opened by vivek-athina 12 days ago
- 1 comment
Labels: published
#115 - Release/1-5-18
Pull Request -
State: closed - Opened by vivek-athina 16 days ago
- 1 comment
Labels: published
#114 - Release/1-5-16
Pull Request -
State: closed - Opened by vivek-athina 16 days ago
- 1 comment
Labels: published
#113 - Vivek/ath 2167 dynamic column openai assistant
Pull Request -
State: closed - Opened by vivek-athina 18 days ago
- 1 comment
Labels: published
#112 - Release/1-5-15
Pull Request -
State: closed - Opened by vivek-athina 19 days ago
- 1 comment
Labels: published
#111 - Release/1-5-14
Pull Request -
State: closed - Opened by vivek-athina 20 days ago
- 1 comment
Labels: published
#110 - Release/1-5-13
Pull Request -
State: closed - Opened by vivek-athina 23 days ago
- 1 comment
Labels: published
#109 - feat: separating concepts for llm service
Pull Request -
State: closed - Opened by DanielAthina 24 days ago
- 2 comments
#108 - feat: metadata response
Pull Request -
State: closed - Opened by DanielAthina 25 days ago
- 1 comment
#107 - Release-1.5.12 (perplexity failed to load json from text in evals)
Pull Request -
State: closed - Opened by vivek-athina about 1 month ago
- 1 comment
Labels: published
#106 - Release-1.5.11
Pull Request -
State: closed - Opened by akshat-g about 2 months ago
- 1 comment
Labels: published
#105 - Release-1.5.10
Pull Request -
State: closed - Opened by vivek-athina about 2 months ago
- 1 comment
Labels: published
#104 - Add multimodal support for llm.py in PromptTemplate and PromptExecution
Pull Request -
State: open - Opened by arunankathina about 2 months ago
- 2 comments
#103 - modify run batch async ATH-1904
Pull Request -
State: closed - Opened by akshat-g about 2 months ago
- 1 comment
#102 - modify langchain openai versions ATH-1904
Pull Request -
State: closed - Opened by akshat-g about 2 months ago
- 1 comment
#101 - Release-1.5.9 (Bug fix in api call)
Pull Request -
State: closed - Opened by vivek-athina about 2 months ago
- 2 comments
Labels: published
#100 - Bug fix in api call
Pull Request -
State: closed - Opened by vivek-athina about 2 months ago
- 1 comment
#99 - An error occurred while posting eval results
Issue -
State: open - Opened by DsDastgheib about 2 months ago
Labels: bug
#98 - Upgrade Ragas, LlamaIndex and Langchain to the latest versions
Pull Request -
State: closed - Opened by akshat-g 2 months ago
- 1 comment
#97 - Remove urlencode for params as request library already does it
Pull Request -
State: closed - Opened by vivek-athina 2 months ago
- 2 comments
#96 - Vivek/ath 1698 fix json validation eval
Pull Request -
State: closed - Opened by vivek-athina 2 months ago
- 1 comment
#95 - Bugs fixes in extract_json_path, API call and improvements in json validation eval
Pull Request -
State: closed - Opened by vivek-athina 2 months ago
- 1 comment
#94 - add weaviate retrieval step
Pull Request -
State: closed - Opened by akshat-g 2 months ago
- 1 comment
#93 - Upgrades to API step and version bump
Pull Request -
State: closed - Opened by vivek-athina 2 months ago
- 2 comments
#92 - API step improved
Pull Request -
State: closed - Opened by vivek-athina 2 months ago
- 1 comment
#91 - add support for custom models for ragas ATH-1721
Pull Request -
State: closed - Opened by akshat-g 2 months ago
- 1 comment
#89 - Error handling in dynamic columns
Pull Request -
State: closed - Opened by vivek-athina 3 months ago
- 1 comment
#88 - Tools support in evals
Pull Request -
State: closed - Opened by vivek-athina 3 months ago
- 1 comment
#87 - Minor fix for error message in JSONSchema Eval
Pull Request -
State: closed - Opened by vivek-athina 3 months ago
- 1 comment
#86 - fix ragas aspect critique issues ATH-1641
Pull Request -
State: closed - Opened by akshat-g 3 months ago
- 1 comment
#85 - Version fixes
Pull Request -
State: closed - Opened by vivek-athina 3 months ago
- 1 comment
#84 - Fix/litellm version specified
Pull Request -
State: closed - Opened by vivek-athina 3 months ago
- 1 comment
#83 - bump json schema version
Pull Request -
State: closed - Opened by akshat-g 3 months ago
- 1 comment
#82 - non boolean custom evals
Pull Request -
State: closed - Opened by akshat-g 3 months ago
- 1 comment
#81 - Added re package support in custom code. Bumped version
Pull Request -
State: closed - Opened by vivek-athina 3 months ago
- 1 comment
#80 - allow 5000 rows
Pull Request -
State: open - Opened by akshat-g 3 months ago
- 1 comment
#79 - Improved custom code handling
Pull Request -
State: closed - Opened by vivek-athina 3 months ago
- 1 comment
#78 - Fix ragas aspect critque metrics
Pull Request -
State: closed - Opened by akshat-g 3 months ago
- 1 comment
#77 - Minor fixes in import
Pull Request -
State: closed - Opened by vivek-athina 4 months ago
- 1 comment
#76 - JsonEval split into two evals
Pull Request -
State: closed - Opened by vivek-athina 4 months ago
- 1 comment
#75 - Custom code eval
Pull Request -
State: closed - Opened by vivek-athina 4 months ago
- 1 comment
#74 - Added code execution and jsonpath extract steps
Pull Request -
State: closed - Opened by vivek-athina 4 months ago
- 1 comment
#73 - add chat history to faifulness
Pull Request -
State: closed - Opened by akshat-g 4 months ago
- 1 comment
#72 - fix logger issues
Pull Request -
State: closed - Opened by akshat-g 4 months ago
- 1 comment
#71 - Fix/add conversation coherence eval
Pull Request -
State: closed - Opened by akshat-g 4 months ago
- 1 comment
#70 - Added support to pass dataset id in run eval suite flow
Pull Request -
State: closed - Opened by vivek-athina 4 months ago
- 1 comment
#69 - remove supported models check
Pull Request -
State: closed - Opened by akshat-g 4 months ago
- 1 comment
#68 - Fix/json eval text validation reason
Pull Request -
State: closed - Opened by vivek-athina 4 months ago
- 1 comment
#67 - Added reason to JsonEval validations
Pull Request -
State: closed - Opened by vivek-athina 4 months ago
- 1 comment
#66 - Fix for boolean evals to return 0 or 1
Pull Request -
State: closed - Opened by vivek-athina 4 months ago
- 1 comment
#65 - Added JsonEval
Pull Request -
State: closed - Opened by vivek-athina 4 months ago
- 1 comment
#64 - modify prompt execution step
Pull Request -
State: closed - Opened by akshat-g 4 months ago
- 1 comment
#63 - Added more guardrail evals and a few function evals. Fixed equals eval.
Pull Request -
State: closed - Opened by vivek-athina 4 months ago
- 1 comment
#62 - Add retrieval nodes
Pull Request -
State: closed - Opened by akshat-g 4 months ago
- 1 comment
#61 - Support double curly brace in custom prompt
Pull Request -
State: closed - Opened by vivek-athina 4 months ago
- 1 comment
#60 - Added new marvin nodes for classification and extracting entities
Pull Request -
State: closed - Opened by vivek-athina 4 months ago
- 1 comment
#59 - added jinja parsing in custom eval prompt templating
Pull Request -
State: closed - Opened by vivek-athina 5 months ago
- 1 comment
#58 - Added basic classes for steps and chains along with examples
Pull Request -
State: closed - Opened by shivsak 5 months ago
- 1 comment
#57 - dataset migration
Pull Request -
State: closed - Opened by akshat-g 5 months ago
- 1 comment
#56 - added intent evaluator
Pull Request -
State: open - Opened by vyelluru 5 months ago
- 1 comment
#55 - added gpt-4o and gpt4-32k language models
Pull Request -
State: closed - Opened by vivek-athina 5 months ago
- 1 comment
#54 - added newer llama models
Pull Request -
State: closed - Opened by vivek-athina 5 months ago
- 1 comment
#53 - Feature/sdk run evals dataset
Pull Request -
State: closed - Opened by akshat-g 5 months ago
- 1 comment
#52 - add chat history to context contains enough information evaluator ATH-1029
Pull Request -
State: closed - Opened by akshat-g 5 months ago
- 1 comment
#51 - Bugfix/context similarity eval
Pull Request -
State: closed - Opened by vivek-athina 6 months ago
- 1 comment
#50 - Vivek/ath 1124 bug fixes in safety evals
Pull Request -
State: closed - Opened by vivek-athina 6 months ago
- 1 comment
#49 - mistral large model name fixed and version bump
Pull Request -
State: closed - Opened by vivek-athina 6 months ago
- 1 comment
#48 - added guardrail and safety evals to be run from samplerun
Pull Request -
State: closed - Opened by vivek-athina 6 months ago
- 1 comment
#47 - added newer models
Pull Request -
State: closed - Opened by vivek-athina 6 months ago
- 1 comment
#46 - Remove SDK print statement "You can view the results at app.athina.ai/develop/request"
Pull Request -
State: closed - Opened by akshat-g 6 months ago
- 1 comment
#45 - Feature/develop flow refactor
Pull Request -
State: closed - Opened by akshat-g 6 months ago
- 1 comment
#44 - Feature/develop flow
Pull Request -
State: closed - Opened by akshat-g 6 months ago
- 2 comments
#43 - fixing athina init
Pull Request -
State: closed - Opened by vivek-athina 6 months ago
- 1 comment
#42 - adding convenience wrapper for eval suite run
Pull Request -
State: closed - Opened by vivek-athina 6 months ago
- 1 comment
#41 - add kwargs to grading criteria
Pull Request -
State: closed - Opened by akshat-g 6 months ago
- 1 comment
#40 - guard rail evals(gibberish, nsfw, sensitive topics)
Pull Request -
State: closed - Opened by vivek-athina 6 months ago
- 1 comment
#39 - Content Moderation Eval Added
Pull Request -
State: closed - Opened by vivek-athina 6 months ago
- 1 comment
#38 - Removed additional kwargs in functions
Pull Request -
State: closed - Opened by vivek-athina 6 months ago
- 1 comment
#37 - renamed regex to pattern
Pull Request -
State: closed - Opened by vivek-athina 6 months ago
- 1 comment
#36 - Added fix for summarizationHal
Pull Request -
State: closed - Opened by frostbite127 6 months ago
- 1 comment
#35 - Real-time guards + New Evals: Prompt Injection + PII Detection + Changed function evals to evaluate text instead of response
Pull Request -
State: closed - Opened by shivsak 6 months ago
- 1 comment
#34 - Cleanup: Renamed conversation_messages to messages for conversation evals and added better examples
Pull Request -
State: closed - Opened by shivsak 7 months ago
- 1 comment
#33 - bug fix in preprocess strings
Pull Request -
State: closed - Opened by vivek-athina 7 months ago
- 1 comment
#32 - Conversation Evals Code Cleanup
Pull Request -
State: closed - Opened by shivsak 7 months ago
- 1 comment
#31 - fixed api call eval and grounded eval test runs
Pull Request -
State: closed - Opened by vivek-athina 7 months ago
- 1 comment
#30 - ATH-841 ATH-842
Pull Request -
State: closed - Opened by akshat-g 7 months ago
- 1 comment
#29 - added missing models
Pull Request -
State: closed - Opened by vivek-athina 7 months ago
- 1 comment
#28 - fixed api call function args
Pull Request -
State: closed - Opened by vivek-athina 7 months ago
- 1 comment
#27 - API Call eval updated
Pull Request -
State: closed - Opened by vivek-athina 7 months ago
- 1 comment
#26 - add failure threshold logic
Pull Request -
State: closed - Opened by akshat-g 7 months ago
- 1 comment
#25 - Added functions to get evaluator from eval name
Pull Request -
State: closed - Opened by frostbite127 7 months ago
- 1 comment
#24 - Conversation Coherence Evaluator: Check if conversation is logically coherent w.r.t previous messages
Pull Request -
State: closed - Opened by shivsak 7 months ago
- 1 comment
#23 - ConversationResolution evaluator: Checks if every message in the conversation was addressed
Pull Request -
State: closed - Opened by shivsak 8 months ago
- 1 comment
#22 - Bug fix in eval suite run
Pull Request -
State: closed - Opened by vivek-athina 8 months ago
- 1 comment
#21 - added cohere models to Model enum. remove unused dependency
Pull Request -
State: closed - Opened by vivek-athina 8 months ago
- 2 comments
#20 - Added the GibberishText evaluator from guardrails hub (as a PoC of using Guardrails evals)
Pull Request -
State: closed - Opened by shivsak 8 months ago
- 1 comment