confident-ai/deepeval issues and pull requests

#1041 - Update getting-started.mdx

Pull Request - State: closed - Opened by fabiofumarola about 2 months ago - 2 comments

#1040 - new release

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1039 - breaking synthesizer changes

Pull Request - State: closed - Opened by kritinv about 2 months ago - 1 comment

#1038 - ChromaDB dependency missing

Issue - State: closed - Opened by fabiannagel about 2 months ago - 1 comment

#1037 - Synthesizer guide

Pull Request - State: closed - Opened by kritinv about 2 months ago - 1 comment

#1036 - new release

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1035 - Fix copied_turns in ConversationalTestCase

Pull Request - State: closed - Opened by jaime-cespedes-sisniega about 2 months ago - 2 comments

#1034 - Results are not in the same order as input test cases

Issue - State: open - Opened by joaopcm1996 about 2 months ago

#1033 - filter synthesizer

Pull Request - State: closed - Opened by kritinv about 2 months ago - 1 comment

#1032 - Typo in docs/docs/benchmarks-introduction.mdx

Issue - State: open - Opened by jtquach1 about 2 months ago

#1031 - new release

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1030 - skip feature

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1029 - g_eval metric implicitly falls back to top score instead of sum of weighted top probabilities

Issue - State: open - Opened by mattany about 2 months ago - 1 comment

#1028 - new release

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1027 - chromadb collection error

Pull Request - State: closed - Opened by kritinv about 2 months ago - 1 comment

#1026 - new release

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1025 - breaking synthesizer changes

Pull Request - State: closed - Opened by kritinv about 2 months ago - 1 comment

#1024 - new release

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1023 - Relaxe grpc-io version

Issue - State: open - Opened by FrancoisMasson1990 about 2 months ago

#1022 - new release

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1021 - Fix metric data logic

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1020 - new release

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1019 - Ability to tie TestResults with TestCases for batch runs

Issue - State: closed - Opened by debdayal-aws about 2 months ago - 3 comments

#1018 - fix(docs): typo in MMLU import statement

Pull Request - State: closed - Opened by jalling97 about 2 months ago - 3 comments

#1017 - fix(synthesizer): fix error caused by formatting during synthesizer generation from docs

Pull Request - State: closed - Opened by chododom about 2 months ago - 2 comments

#1016 - New synthesizer features

Pull Request - State: closed - Opened by kritinv about 2 months ago - 1 comment

#1015 - [Question] What's the difference between Deepeval's RAG Evaluation and RAGAS

Issue - State: closed - Opened by landhu about 2 months ago - 1 comment

#1014 - FIx evaluate function

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1013 - new release

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1012 - new release

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1011 - fix MLLM bugs

Pull Request - State: closed - Opened by kritinv about 2 months ago - 1 comment

#1010 - Quick fixes to Multimodal

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1009 - Fix #997 Issue

Pull Request - State: closed - Opened by kritinv 2 months ago - 1 comment

#1008 - Synthesizer updates

Pull Request - State: closed - Opened by kritinv 2 months ago - 1 comment

#1007 - Support integrating Ollama as a model provider

Issue - State: closed - Opened by benlyazid 2 months ago - 2 comments

#1006 - new release

Pull Request - State: closed - Opened by penguine-ip 2 months ago - 1 comment

#1005 - fix success state

Pull Request - State: closed - Opened by penguine-ip 2 months ago - 1 comment

#1004 - new release

Pull Request - State: closed - Opened by penguine-ip 2 months ago - 1 comment

#1003 - Hotfix/red teamer

Pull Request - State: closed - Opened by penguine-ip 2 months ago - 1 comment

#1002 - fix: fix concation issue using f string

Pull Request - State: closed - Opened by Jinankhan 2 months ago - 2 comments

#1001 - fix: fix concation issue using f string

Pull Request - State: closed - Opened by Jinankhan 2 months ago - 1 comment

#1000 - new release

Pull Request - State: closed - Opened by penguine-ip 2 months ago - 1 comment

#999 - red teaming usage tracking

Pull Request - State: closed - Opened by kritinv 2 months ago - 1 comment

#998 - Multimodal evaluation

Pull Request - State: closed - Opened by kritinv 2 months ago - 1 comment

#997 - [python] Zero test case in benchmark HellaSwag Task of POLISHING_FURNITURE

Issue - State: closed - Opened by Jonathanjordan21 2 months ago - 1 comment

#996 - Local model support for OpenAI compatible API providers like Ollama and LM Studio (LLM and embeddings)

Pull Request - State: closed - Opened by elsatch 2 months ago - 1 comment

#995 - Missing retrieval_context variable in sample code

Pull Request - State: closed - Opened by elsatch 2 months ago - 1 comment

#994 - Give an explicit number to prevent JSON errors.

Pull Request - State: closed - Opened by john-lemmon-lime 2 months ago - 1 comment

#993 - deepeval AttributeError: 'str' object has no attribute 'answer'

Issue - State: open - Opened by habout632 2 months ago - 1 comment

#992 - new release

Pull Request - State: closed - Opened by penguine-ip 2 months ago - 1 comment

#991 - New release

Pull Request - State: closed - Opened by penguine-ip 2 months ago - 1 comment

#990 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#989 - rename

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#988 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#987 - bug: Tools used not showing up in test report

Issue - State: closed - Opened by chip-davis 3 months ago - 3 comments

#986 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#985 - concurrent evaluate

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#984 - timeout mechanism

Issue - State: open - Opened by Ishiki-Iroha 3 months ago

#983 - Unable to generate Red Teaming Data from Synthesizer using Azure as a model

Issue - State: open - Opened by vsowmya105 3 months ago - 12 comments

#982 - ValueError: Evaluation LLM outputted an invalid JSON. Please use a better evaluation model--- while trying to run synthesizer with Ollama model

Issue - State: closed - Opened by vsowmya105 3 months ago - 1 comment

#981 - answer_relevancy_ metric always "None"

Issue - State: open - Opened by cpolcino 3 months ago - 2 comments

#980 - deepeval and api key

Issue - State: open - Opened by cpolcino 3 months ago - 1 comment

#979 - Knowledge retention metric does not work

Issue - State: open - Opened by domciakocan 3 months ago - 1 comment

#978 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#977 - Printing

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#976 - About 'node' in prompt of contextual_precision

Issue - State: closed - Opened by Ishiki-Iroha 3 months ago - 1 comment

#975 - Deepeval does not work on AWS Lambda

Issue - State: open - Opened by atrik-das-razer 3 months ago - 2 comments

#974 - update docs

Pull Request - State: closed - Opened by kritinv 3 months ago - 1 comment

#973 - Fix langchain tracing

Pull Request - State: closed - Opened by kritinv 3 months ago - 1 comment

#972 - Synthesizer - generate from scratch returns only one golden

Issue - State: open - Opened by attilalaszlo 3 months ago - 2 comments

#971 - fix schema

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#970 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#969 - Error on dataset.pull: TypeError: Api.send_request() got an unexpected keyword argument 'params'

Issue - State: closed - Opened by bendominguez0111 3 months ago - 1 comment

#968 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#967 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#966 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#965 - error

Issue - State: closed - Opened by GeorgeDittmar 3 months ago - 1 comment

#964 - Incorrect pydantic schemas cause ValidationErrors when using a custom LLM for evaluation

Issue - State: closed - Opened by harriet-wood 3 months ago

#963 - Update incorrect pydantic schemas for metrics and add tests for these

Pull Request - State: closed - Opened by harriet-wood 3 months ago - 3 comments

#962 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#961 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#960 - Changes to API

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#959 - Bugfix (utils): Assign error string before raising

Pull Request - State: closed - Opened by gCaglia 3 months ago - 2 comments

#958 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#957 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#956 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#955 - Add optional test run manager

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#954 - Custom LLM Example do not Work

Issue - State: open - Opened by Otavio-Parraga 3 months ago - 3 comments

#953 - Minor typing improvements

Pull Request - State: open - Opened by MartinoMensio 3 months ago - 3 comments

#952 - Update opentelemetry to v1.25.0

Pull Request - State: closed - Opened by fedesierr 3 months ago - 3 comments

#951 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#950 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#949 - .

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#948 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#947 - Example Colab notebook is failing. No module named 'deepeval.metrics.conversation_relevancy

Issue - State: open - Opened by arianpasquali 3 months ago - 1 comment

#946 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#945 - Fix tool correctness

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

GitHub / confident-ai/deepeval issues and pull requests