confident-ai/deepeval issues and pull requests

#940 - Tool Correctness documentation page shows using ContextualRelevancyMetric instead of ToolCorrectnessMetric

Issue - State: closed - Opened by vensr 6 months ago - 1 comment

#939 - more relaxed constraints

Pull Request - State: closed - Opened by MartinoMensio 6 months ago - 4 comments

#938 - Red team scanner

Pull Request - State: closed - Opened by kritinv 6 months ago - 1 comment

#937 - SummarizationMetric changing score and reasoning methodology at random

Issue - State: open - Opened by VVinayak 6 months ago - 1 comment

#936 - BiasMetric fails to return verdicts in release >=0.21.73

Issue - State: closed - Opened by VVinayak 6 months ago - 1 comment

#935 - Added conversational messages

Pull Request - State: closed - Opened by penguine-ip 6 months ago - 1 comment

#934 - rename models to schema

Pull Request - State: closed - Opened by penguine-ip 6 months ago - 1 comment

#933 - Recursion Error While Verbose_mode is set to True

Issue - State: open - Opened by vinitkhemka 6 months ago - 1 comment

#932 - Fixed bug in ContextGenerator where it wouldn't generate the correct number of contexts + fix tests pipeline

Pull Request - State: closed - Opened by fschuh 6 months ago - 7 comments

#931 - Add jsonl file support in dataset

Issue - State: open - Opened by princepride 6 months ago - 2 comments

#930 - benchmarks JSON confinement

Pull Request - State: closed - Opened by kritinv 6 months ago - 1 comment

#929 - ValueError: Evaluation LLM outputted an invalid JSON. Please use a better evaluation model.

Issue - State: open - Opened by JingsenZhang 6 months ago - 5 comments

#924 - Why actual_output is required for Context Relevancy?

Issue - State: closed - Opened by isaactl 6 months ago - 1 comment

#917 - Suggestion to modify the "valid_gpt_models" list from gpt_model.py

Issue - State: open - Opened by KisuYang 6 months ago

#916 - data.generate_goldens throw TypeError: Synthesizer.generate_goldens() got an unexpected keyword argument 'enable_breadth_evolve'

Issue - State: open - Opened by gring2 7 months ago - 2 comments

#914 - deepeval v0.21.71 - v 0.21.73 MMLU prediction results are wrong and the task scores and overall score are always 0 for Mistral7B and Gemma2B models

Issue - State: open - Opened by dequanchen 7 months ago - 6 comments

#887 - The Colab notebook left as an example in the LLamaIndex Docs is not working!

Issue - State: open - Opened by reemabdelrazek30 7 months ago - 3 comments

#883 - Patch for issue 882 "deepeval --login" fails

Pull Request - State: closed - Opened by jerrydboonstra 7 months ago - 4 comments

#855 - Disable Update Warnings

Issue - State: closed - Opened by murnitur 8 months ago - 1 comment

#847 - Update to tenacity 8.4.1

Issue - State: open - Opened by FrancoisMasson1990 8 months ago - 4 comments

#844 - AuthenticationError

Issue - State: open - Opened by AprilJoy 8 months ago - 3 comments

#843 - Invalid value for 'top_logprobs': must be less than or equal to 5 for AzureOpenAi Models

Issue - State: open - Opened by Neeraj319 8 months ago - 2 comments

#829 - Add Support for `Gemini` Models

Issue - State: open - Opened by CD-rajveer 8 months ago - 7 comments

#821 - Message about nest_asyncio Should Not Be Printed

Issue - State: open - Opened by hoffm386 8 months ago - 3 comments

#779 - Synthesizer generates goldens but a dataset loads as test cases

Issue - State: open - Opened by penguine-ip 9 months ago - 1 comment

#743 - Cant use HuggingFace Model for evaluation

Issue - State: open - Opened by Kraebs 9 months ago - 14 comments

#738 - Knowledge Retention Metric Fix

Pull Request - State: closed - Opened by AnanyaRaval 9 months ago - 1 comment

#500 - Parallelization of Evaluations

Issue - State: open - Opened by AndresPrez 12 months ago - 10 comments

#453 - NER Metric

Pull Request - State: closed - Opened by RigvedRocks about 1 year ago - 1 comment

#193 - feat: add question generator with llamaindex

Pull Request - State: closed - Opened by chziakas over 1 year ago - 2 comments

#100 - Add LLMEvalMetric

Issue - State: closed - Opened by ColabDog over 1 year ago - 1 comment
Labels: enhancement, help wanted

#99 - Improve typing (Mypy) support

Issue - State: closed - Opened by ColabDog over 1 year ago - 1 comment
Labels: enhancement

#98 - Flowise Integration

Issue - State: closed - Opened by ColabDog over 1 year ago
Labels: help wanted

#97 - Compare Anthropic Claude and ChatGPT in Factual Consistency, Answer Relevancy

Issue - State: closed - Opened by ColabDog over 1 year ago
Labels: help wanted, question

#96 - Cohere Integration

Issue - State: closed - Opened by ColabDog over 1 year ago
Labels: good first issue, help wanted

#95 - Add unstructured integration for synthetic data

Issue - State: closed - Opened by ColabDog over 1 year ago
Labels: good first issue, help wanted

#94 - Feature/add context back to textcase

Pull Request - State: closed - Opened by ColabDog over 1 year ago - 57 comments

#93 - Add GuardRails Tutorial

Issue - State: closed - Opened by ColabDog over 1 year ago

#92 - Update README.md

Pull Request - State: closed - Opened by penguine-ip over 1 year ago - 2 comments

#91 - Feature/add spinner progress

Pull Request - State: closed - Opened by ColabDog over 1 year ago - 2 comments

#90 - Feature/improve writing unit tests

Pull Request - State: closed - Opened by ColabDog over 1 year ago - 4 comments

#89 - Improve support for Jupyter notebook usage

Issue - State: closed - Opened by ColabDog over 1 year ago - 1 comment

#88 - Parallelized Tests

Issue - State: closed - Opened by ColabDog over 1 year ago
Labels: enhancement, help wanted

#87 - Add Ruff Linter

Issue - State: closed - Opened by ColabDog over 1 year ago
Labels: enhancement

#86 - Fix conceptual similarity

Pull Request - State: closed - Opened by ColabDog over 1 year ago - 2 comments

#85 - Hotfix/incorrect issue

Pull Request - State: closed - Opened by ColabDog over 1 year ago - 4 comments

#84 - Why are chunks required in FactualConsistencyMetric

Issue - State: closed - Opened by FloTeu over 1 year ago - 2 comments

#83 - Error logging conceptual similarity

Issue - State: closed - Opened by ColabDog over 1 year ago
Labels: bug, good first issue

#82 - Hotfix/fix project name

Pull Request - State: closed - Opened by ColabDog over 1 year ago - 2 comments

#81 - Feature/fix quickstart

Pull Request - State: closed - Opened by ColabDog over 1 year ago - 3 comments

#80 - Add Evaluation For Structured Output

Issue - State: closed - Opened by ColabDog over 1 year ago

#79 - remove

Pull Request - State: closed - Opened by ColabDog over 1 year ago - 2 comments

#78 - LiteLLM Docs

Issue - State: closed - Opened by krrishdholakia over 1 year ago - 2 comments

#77 - Feature/log context

Pull Request - State: closed - Opened by ColabDog over 1 year ago - 2 comments

#76 - Add CLI for AutoEvals

Issue - State: closed - Opened by ColabDog over 1 year ago

#75 - Fix overall score

Pull Request - State: closed - Opened by ColabDog over 1 year ago - 2 comments

#74 - Add gpt synthetic data

Pull Request - State: closed - Opened by penguine-ip over 1 year ago - 6 comments

#73 - Feature/add test overall score

Pull Request - State: closed - Opened by ColabDog over 1 year ago - 5 comments

#72 - Add Translation Similarity

Issue - State: closed - Opened by ColabDog over 1 year ago

#71 - Add ChatGPT Synthetic Data Creation

Issue - State: closed - Opened by ColabDog over 1 year ago - 1 comment
Labels: enhancement, good first issue, help wanted

#70 - Add functionality to calculate average score

Issue - State: closed - Opened by ColabDog over 1 year ago - 3 comments
Labels: enhancement, help wanted