confident-ai/deepeval issues and pull requests

#1255 - change guardrails URL

Pull Request - State: closed - Opened by kritinv about 1 month ago - 1 comment

#1255 - change guardrails URL

Pull Request - State: closed - Opened by kritinv about 1 month ago - 1 comment

#1254 - new release

Pull Request - State: closed - Opened by penguine-ip about 1 month ago - 1 comment

#1254 - new release

Pull Request - State: closed - Opened by penguine-ip about 1 month ago - 1 comment

#1253 - fix text-to-image metric bug

Pull Request - State: closed - Opened by kritinv about 1 month ago - 1 comment

#1252 - guardrails base url change

Pull Request - State: closed - Opened by kritinv about 1 month ago - 1 comment

#1251 - Text to Image metric incorrect evaluation

Issue - State: open - Opened by shanmukh-sai2003 about 1 month ago - 3 comments

#1250 - update `target_model_callback` in red-teaming docs

Pull Request - State: closed - Opened by kritinv about 1 month ago - 1 comment

#1249 - Issue while running scan

Issue - State: open - Opened by jayjpjays about 1 month ago - 2 comments

#1248 - Issue while running scan

Issue - State: open - Opened by jaysudhakaran about 1 month ago

#1247 - guardrails

Pull Request - State: closed - Opened by kritinv about 1 month ago - 1 comment

#1246 - a few benchmark optimizations

Pull Request - State: closed - Opened by kritinv about 1 month ago - 1 comment

#1245 - new release

Pull Request - State: closed - Opened by penguine-ip about 1 month ago - 1 comment

#1244 - DeepEval, G-Eval logging issues.

Issue - State: open - Opened by j0ntendo about 1 month ago

#1243 - When I set local ollama , but still need the Open api key

Issue - State: open - Opened by lijinxinKing about 1 month ago - 2 comments

#1242 - Changed Error type

Pull Request - State: closed - Opened by Mizuki8783 about 1 month ago - 1 comment

#1241 - Multimodal rag

Pull Request - State: closed - Opened by kritinv about 1 month ago - 1 comment

#1240 - benchmark docs

Pull Request - State: closed - Opened by kritinv about 2 months ago - 1 comment

#1239 - Benchmarks

Pull Request - State: closed - Opened by kritinv about 2 months ago - 1 comment

#1237 - identifier

Pull Request - State: closed - Opened by penguine-ip about 2 months ago - 1 comment

#1236 - red teaming updates

Pull Request - State: closed - Opened by kritinv about 2 months ago - 1 comment

#1235 - CLI improvement: option to display only failed tests

Issue - State: open - Opened by yuriykuzin about 2 months ago - 1 comment

#1234 - ToolCorrectnessMetric raises ZeroDivisionError when expected_tools is empty

Issue - State: open - Opened by j-mesnil about 2 months ago

#1233 - Grammar fix in RAG eval guide

Pull Request - State: closed - Opened by nishant-mahesh about 2 months ago - 1 comment

#1231 - Ensure opentelemetry only loaded if telemetry is not opted outed

Pull Request - State: closed - Opened by chaliy about 2 months ago - 2 comments

#1230 - 3 new MLLM metrics

Pull Request - State: closed - Opened by kritinv about 2 months ago - 2 comments

#1229 - fix: MMLUTemplate.format_subject should be a static method

Pull Request - State: closed - Opened by Jerry-Terrasse about 2 months ago - 2 comments

#1228 - Loosen grpcio dependencies to avoid opentelemetry installing issues

Pull Request - State: closed - Opened by Redna about 2 months ago - 10 comments

#1227 - Avoid unnecessary opentelemetry imports

Issue - State: closed - Opened by volodymyr-lotos about 2 months ago

#1226 - Update pyproject.toml by updating version of tenacity

Pull Request - State: closed - Opened by Anindyadeep about 2 months ago - 10 comments

#1225 - Doc Improvement

Issue - State: open - Opened by SamyAteia about 2 months ago - 4 comments

#1224 - GH-99: include types and check types

Pull Request - State: closed - Opened by ChristianMurphy about 2 months ago - 3 comments

#1223 - GH-1169: sync dependencies between setup py and pyproject toml, remove setup py

Pull Request - State: closed - Opened by ChristianMurphy about 2 months ago - 2 comments

#1222 - Add doc count to the ContextualPrecision template (Fixes #1205)

Pull Request - State: closed - Opened by enrico-stauss about 2 months ago - 2 comments

#1221 - Stop deepeval from storing sensitive information in dotenv files.

Issue - State: open - Opened by thomend about 2 months ago - 3 comments

#1220 - Can't pass trust_remote_code to hallucination_score

Issue - State: open - Opened by mausch about 2 months ago - 2 comments

#1218 - Feature/synthesis cost

Pull Request - State: closed - Opened by vjsliogeris 2 months ago - 4 comments

#1217 - new release

Pull Request - State: closed - Opened by penguine-ip 2 months ago - 1 comment

#1216 - Generate Your First Golden - Using Custom model

Issue - State: open - Opened by pratikchhapolika 2 months ago - 7 comments

#1215 - new release

Pull Request - State: closed - Opened by penguine-ip 2 months ago - 1 comment

#1214 - Update gpt_model.py to include base_url in ChatOpenAI object

Pull Request - State: closed - Opened by cmorris108 2 months ago - 1 comment

#1213 - Metric for Answer Precision/Completeness

Issue - State: closed - Opened by enrico-stauss 2 months ago - 2 comments

#1212 - Documentation for ContextualPrecision

Issue - State: open - Opened by enrico-stauss 2 months ago

#1211 - Commented piece of code for Golden creation

Issue - State: open - Opened by jaroslaw-pawluszka 2 months ago - 1 comment

#1210 - Add image support to LLMTestCase

Issue - State: open - Opened by ethanelasky 2 months ago - 11 comments

#1209 - redteaming 2.0 docs

Pull Request - State: closed - Opened by kritinv 2 months ago - 1 comment

#1208 - Fixes #1207

Pull Request - State: closed - Opened by kbarendrecht 2 months ago - 5 comments

#1207 - ModuleNotFoundError: No module named 'nest_asyncio'

Issue - State: closed - Opened by kbarendrecht 2 months ago

#1206 - Red teaming 2.0

Pull Request - State: closed - Opened by kritinv 2 months ago - 1 comment

#1205 - ContextualPrecision endless generation

Issue - State: closed - Opened by enrico-stauss 2 months ago - 3 comments

#1204 - new release

Pull Request - State: closed - Opened by penguine-ip 2 months ago - 1 comment

#1202 - Fix/fix copy metrics

Pull Request - State: closed - Opened by vjsliogeris 2 months ago - 3 comments

#1201 - copy_metrics from deepeval.metrics.utils does not copy arguments when they're passed via `**kwargs` instead of named arguments.

Issue - State: closed - Opened by vjsliogeris 2 months ago - 2 comments

#1200 - Make TELEMETRY_DATA_FILE located into temp folder or configurable by env var

Issue - State: open - Opened by cisarpavel 2 months ago

#1199 - AssertionError: n_contexts_per_doc must be a positive integer.

Issue - State: open - Opened by tenchanhe 2 months ago - 9 comments

#1198 - Update getting-started.mdx, enhanced for Windows users

Pull Request - State: closed - Opened by bmerkle 2 months ago - 1 comment

#1197 - guardrails docs

Pull Request - State: closed - Opened by kritinv 2 months ago - 1 comment

#1196 - Fix issue #1171

Pull Request - State: closed - Opened by CAW-nz 2 months ago - 2 comments

#1195 - Fix issue #1185

Pull Request - State: closed - Opened by CAW-nz 2 months ago - 2 comments

#1194 - Update grpcio version to 1.68.0

Pull Request - State: closed - Opened by psjamesh 2 months ago - 2 comments

#1193 - Tutorial Updates

Pull Request - State: closed - Opened by kritinv 2 months ago - 1 comment

#1192 - tutorial datasets + other edits

Pull Request - State: closed - Opened by kritinv 2 months ago - 1 comment

#1191 - 2.0 release

Pull Request - State: closed - Opened by penguine-ip 2 months ago - 1 comment

#1190 - new metric

Pull Request - State: closed - Opened by penguine-ip 2 months ago - 1 comment

#1189 - tutorial dataset

Pull Request - State: closed - Opened by kritinv 2 months ago - 1 comment

#1188 - updated guards

Pull Request - State: closed - Opened by kritinv 2 months ago - 1 comment

#1187 - Tutorial evaluations

Pull Request - State: closed - Opened by kritinv 2 months ago - 1 comment

#1186 - new release

Pull Request - State: closed - Opened by penguine-ip 2 months ago - 1 comment

#1185 - Bug: Two Red-Teaming attack enhancement modes are failing (at line 121 of red_teamer.py) when red_teamer.scan() runs with async_mode=False

Issue - State: closed - Opened by CAW-nz 2 months ago - 5 comments

#1184 - Update guides-red-teaming.mdx : Fix object name of load_model()

Pull Request - State: closed - Opened by MANISH007700 3 months ago - 2 comments

#1183 - Tutorial - Metrics

Pull Request - State: closed - Opened by kritinv 3 months ago - 1 comment

#1182 - Fixed g-eval doc.

Pull Request - State: closed - Opened by zyuanlim 3 months ago - 2 comments

#1181 - Synthesizer in deepeval requires an OpenAi key

Issue - State: closed - Opened by alexazarh 3 months ago - 3 comments

#1180 - fix: corrected unaligned attacks enum to match promptfoo

Pull Request - State: closed - Opened by nabeel-chhatri 3 months ago - 2 comments

#1179 - Number of golden records in output is not the same as requested in input during synthesis

Issue - State: closed - Opened by nalin-programmer 3 months ago - 3 comments

#1178 - ISSUE 1173: dataset loading and saving

Pull Request - State: closed - Opened by kritinv 3 months ago - 1 comment

#1177 - fix issue #1174

Pull Request - State: closed - Opened by kritinv 3 months ago - 1 comment

#1176 - Prompt versioning from confident

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#1175 - tutorial

Pull Request - State: closed - Opened by kritinv 3 months ago - 1 comment

#1174 - Is there an initialisation bug in synthesizer/chunking/context_generator.py for class ContextGenerator method generate_contexts() ?

Issue - State: closed - Opened by CAW-nz 3 months ago - 3 comments

#1173 - Issue regarding consistent loading of source_files data (from goldens original values)

Issue - State: open - Opened by CAW-nz 3 months ago - 3 comments

#1172 - Corrections for some of the DeepEval documentation/help pages

Issue - State: closed - Opened by CAW-nz 3 months ago - 3 comments

#1171 - Bug: dataset.py - All 4 Open statements in this file are lacking encoding="utf-8" argument

Issue - State: closed - Opened by CAW-nz 3 months ago - 2 comments

#1170 - new release

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#1169 - Use Poetry build instead of setup.py

Issue - State: closed - Opened by FrancoisMasson1990 3 months ago - 1 comment

#1168 - update guardrails

Pull Request - State: closed - Opened by kritinv 3 months ago - 1 comment

#1167 - Revert "Replace context with retrieval_context in LLMTestCaseParams for HallucinationMetric"

Pull Request - State: closed - Opened by penguine-ip 3 months ago - 1 comment

#1166 - fix tracing

Pull Request - State: closed - Opened by kritinv 3 months ago - 1 comment

#1165 - fix batch processing for large documents

Pull Request - State: closed - Opened by kritinv 3 months ago - 1 comment

#1164 - fix guardrails

Pull Request - State: closed - Opened by kritinv 3 months ago - 1 comment

#1163 - How to integrate llama3 to deepeval?

Issue - State: open - Opened by drbhushan 3 months ago - 1 comment

#1162 - tutorial

Pull Request - State: closed - Opened by kritinv 3 months ago - 1 comment

#1161 - Replace context with retrieval_context in LLMTestCaseParams for HallucinationMetric

Pull Request - State: closed - Opened by louisbrulenaudet 3 months ago - 2 comments

#1160 - Redteam Jailbreak Linear / Tree Fix

Pull Request - State: closed - Opened by nabeel-chhatri 3 months ago - 1 comment

GitHub / confident-ai/deepeval issues and pull requests