Ecosyste.ms: Issues
An open API service for providing issue and pull request metadata for open source projects.
GitHub / confident-ai/deepeval issues and pull requests
#1154 - new release
Pull Request -
State: closed - Opened by penguine-ip 1 day ago
- 1 comment
#1153 - fix import require
Pull Request -
State: closed - Opened by penguine-ip 1 day ago
- 1 comment
#1152 - pass test name to test result
Pull Request -
State: closed - Opened by AugmentMo 1 day ago
- 2 comments
#1151 - a few tracing fixes
Pull Request -
State: closed - Opened by kritinv 3 days ago
- 1 comment
#1149 - GEval not focusing on expected_output & Relying on OpenAI instead
Issue -
State: open - Opened by pavan-growexxer 3 days ago
- 1 comment
#1148 - red-teaming guide
Pull Request -
State: closed - Opened by kritinv 3 days ago
- 1 comment
#1147 - Asynchronous test runs are sometimes not completed correctly
Issue -
State: open - Opened by jmaczan 5 days ago
- 1 comment
#1146 - Hallucination metric score is set to 0.0 in one run and 1.0 in another run, despite having the same input values
Issue -
State: open - Opened by jmaczan 5 days ago
- 1 comment
#1145 - Hallucination metric assigns only either 0.0 or 1.0 score
Issue -
State: open - Opened by jmaczan 5 days ago
- 1 comment
#1144 - guardrails
Pull Request -
State: closed - Opened by kritinv 6 days ago
- 1 comment
#1143 - g.
Pull Request -
State: closed - Opened by penguine-ip 8 days ago
- 1 comment
#1142 - Avoid asking for OPENAI_API_KEY when creating an empty EvaluationDataset
Pull Request -
State: closed - Opened by michieletto 9 days ago
- 1 comment
#1141 - Fix link in getting-started.mdx
Pull Request -
State: closed - Opened by NimJay 10 days ago
- 2 comments
#1140 - EvaluationDataset in deepeval suddenly requires an OpenAi key since update
Issue -
State: closed - Opened by kbarendrecht 10 days ago
- 3 comments
#1139 - Error while calculating Knowledge retention ; Evaluation LLM outputted an invalid JSON. Please use a better evaluation model.
Issue -
State: open - Opened by jaysudhakaran 10 days ago
- 4 comments
#1138 - new release
Pull Request -
State: closed - Opened by penguine-ip 11 days ago
- 1 comment
#1137 - Improving Answer Relevancy Template
Issue -
State: open - Opened by dipanjanS 11 days ago
- 4 comments
#1136 - observability guide
Pull Request -
State: closed - Opened by kritinv 13 days ago
- 1 comment
#1135 - synthesizer docs breakdown
Pull Request -
State: closed - Opened by kritinv 14 days ago
- 1 comment
#1134 - remove extraction limit print
Pull Request -
State: closed - Opened by penguine-ip 14 days ago
- 1 comment
#1133 - reformat
Pull Request -
State: closed - Opened by penguine-ip 14 days ago
- 1 comment
#1132 - OpenAI Rate Limit Exceeds
Issue -
State: closed - Opened by Miriam2040 15 days ago
- 1 comment
#1131 - Support utf8
Pull Request -
State: closed - Opened by kinga-marszalkowska 16 days ago
- 1 comment
#1130 - faithfulness print extraction_limit for each test case
Issue -
State: closed - Opened by rjiangnju 17 days ago
- 1 comment
#1129 - GEval docs: `strict` -> `strict_mode`
Pull Request -
State: closed - Opened by chkimes 17 days ago
- 2 comments
#1128 - utf-8 not supported [with solution]
Issue -
State: closed - Opened by kinga-marszalkowska 17 days ago
#1127 - new release
Pull Request -
State: closed - Opened by penguine-ip 18 days ago
- 1 comment
#1126 - Fix missing relevant_statements in non-async generate_reason
Pull Request -
State: closed - Opened by dendarrion 18 days ago
- 1 comment
#1125 - new release
Pull Request -
State: closed - Opened by penguine-ip 20 days ago
- 1 comment
#1124 - .
Pull Request -
State: closed - Opened by penguine-ip 21 days ago
- 1 comment
#1123 - Image Metrics
Pull Request -
State: closed - Opened by kritinv 22 days ago
- 1 comment
#1122 - Add `tools_called_key_name` and `expected_tools_key_name` to both .add_test_cases_from_json_file() and .add_test_cases_from_csv_file() methods
Issue -
State: closed - Opened by rayenf-mw 22 days ago
- 1 comment
#1121 - confident-link
Pull Request -
State: closed - Opened by kritinv 22 days ago
- 1 comment
#1120 - Black screen after login
Issue -
State: closed - Opened by JLombar 22 days ago
- 1 comment
#1118 - telemetry updates
Pull Request -
State: closed - Opened by kritinv 22 days ago
- 1 comment
#1117 - new release
Pull Request -
State: closed - Opened by penguine-ip 23 days ago
- 1 comment
#1114 - Get tokens for each request
Issue -
State: open - Opened by luqianjiang 23 days ago
- 1 comment
#1113 - ragas metrics
Pull Request -
State: closed - Opened by kritinv 23 days ago
- 1 comment
#1112 - Enable the discussion feature on the repository
Issue -
State: closed - Opened by nprime496 23 days ago
- 1 comment
#1111 - Release v1.4.3
Pull Request -
State: closed - Opened by penguine-ip 23 days ago
- 1 comment
#1110 - update generate methods in dataset
Pull Request -
State: closed - Opened by kritinv 24 days ago
- 2 comments
#1109 - synth docs edit
Pull Request -
State: closed - Opened by kritinv 24 days ago
- 1 comment
#1108 - synthesizer fixes
Pull Request -
State: closed - Opened by kritinv 24 days ago
- 1 comment
#1107 - Red Teaming Docs
Pull Request -
State: closed - Opened by kritinv 24 days ago
- 1 comment
#1106 - Introducing DeepEval Guru on Gurubase.io
Pull Request -
State: closed - Opened by kursataktas 24 days ago
- 2 comments
#1105 - TypeError while evaluating a test case for Role Adherence metric using CustomLlama3_8B.
Issue -
State: open - Opened by Qca-2023 24 days ago
#1104 - fix one more incorrect multistep_arithmetic_two key in schema.py
Pull Request -
State: closed - Opened by NikyParfenov 24 days ago
- 2 comments
#1103 - new release
Pull Request -
State: closed - Opened by penguine-ip 25 days ago
- 1 comment
#1102 - HumanEval doesn't actually runs test
Issue -
State: open - Opened by cfytrok 26 days ago
#1101 - vulnerability map + breaking fixes
Pull Request -
State: closed - Opened by kritinv 26 days ago
- 1 comment
#1100 - The latest versions of deepeval and Instructor can't be installed together
Issue -
State: open - Opened by milest 27 days ago
- 5 comments
#1099 - new vulnerabilities
Pull Request -
State: closed - Opened by kritinv 27 days ago
- 1 comment
#1098 - new release
Pull Request -
State: closed - Opened by penguine-ip 27 days ago
- 1 comment
#1097 - Fix: MMLU loads the old task when a new one should be loaded.
Pull Request -
State: closed - Opened by thohag 27 days ago
- 1 comment
#1096 - new release
Pull Request -
State: closed - Opened by penguine-ip 27 days ago
- 1 comment
#1095 - added error catching
Pull Request -
State: closed - Opened by penguine-ip 27 days ago
- 1 comment
#1094 - APIConnectionError: Connection error.
Issue -
State: open - Opened by paulacanva 28 days ago
- 3 comments
#1093 - new attack enhancements
Pull Request -
State: closed - Opened by kritinv 28 days ago
- 1 comment
#1092 - fix incorrect dyck_languages key in schema.py
Pull Request -
State: closed - Opened by NikyParfenov 28 days ago
- 1 comment
#1091 - bug
Pull Request -
State: closed - Opened by kritinv 28 days ago
- 1 comment
#1090 - Accuracy Issue - accuracy is 0.00
Issue -
State: open - Opened by Pitambartech 28 days ago
#1089 - Incompatibility with ragas v0.2.x causing TypeError in measure function
Issue -
State: open - Opened by lagom0327 28 days ago
#1088 - synthesizer docs
Pull Request -
State: closed - Opened by kritinv 29 days ago
- 1 comment
#1087 - Typo in scorer/scorer.py
Issue -
State: open - Opened by jtquach1 29 days ago
#1086 - Missing API key and endpoint configuration...(etc) in OpenAI() and support for Azure OpenAI
Issue -
State: open - Opened by NaraSyuhei 29 days ago
- 1 comment
#1085 - evolutions argument should be of type Dict[Evolution, float] instead of List[Evolution]
Issue -
State: closed - Opened by NaraSyuhei 29 days ago
- 1 comment
#1084 - new release
Pull Request -
State: closed - Opened by penguine-ip 30 days ago
- 1 comment
#1083 - fix algorithm
Pull Request -
State: closed - Opened by penguine-ip 30 days ago
- 1 comment
#1082 - Wrong interpretation of the result of Hallucination
Issue -
State: closed - Opened by anotine10 about 1 month ago
- 3 comments
#1081 - new release
Pull Request -
State: closed - Opened by penguine-ip about 1 month ago
- 1 comment
#1080 - Synthesizer customizations
Pull Request -
State: closed - Opened by kritinv about 1 month ago
- 1 comment
#1079 - Contextual Relevancy is not implemented the way it's documented
Issue -
State: closed - Opened by linzzzzzz about 1 month ago
- 2 comments
#1078 - new release
Pull Request -
State: closed - Opened by penguine-ip about 1 month ago
- 1 comment
#1077 - dataset.generated_goldens buggy because of wrong type of evolutions
Issue -
State: closed - Opened by amgonzalezd about 1 month ago
- 2 comments
#1076 - new release
Pull Request -
State: closed - Opened by penguine-ip about 1 month ago
- 1 comment
#1075 - cleanup
Pull Request -
State: closed - Opened by penguine-ip about 1 month ago
- 1 comment
#1074 - Pass back test_run info with evaluate
Pull Request -
State: closed - Opened by john-lemmon-lime about 1 month ago
- 2 comments
#1073 - fix: update parameters for dataset's synthesizer's goldens generation from docs method
Pull Request -
State: closed - Opened by chododom about 1 month ago
- 1 comment
#1072 - Upon failing a certain test, deepeval could print the LLM's output
Issue -
State: closed - Opened by adesso-dominik-chodounsky about 1 month ago
- 1 comment
#1071 - Missing max_goldens_per_document argument in synthesizer.generate_goldens_from_docs
Issue -
State: closed - Opened by NaraSyuhei about 1 month ago
- 2 comments
#1070 - quality control + cleanup context
Pull Request -
State: closed - Opened by kritinv about 1 month ago
- 1 comment
#1069 - Generation of goldens from docs in Synthesizer is not possible unless
Issue -
State: closed - Opened by adesso-dominik-chodounsky about 1 month ago
- 2 comments
#1068 - Question About The Syntesizer
Issue -
State: open - Opened by Miriam2040 about 1 month ago
#1067 - docs: RAG evaluation example
Pull Request -
State: closed - Opened by Anush008 about 1 month ago
- 1 comment
#1066 - new release
Pull Request -
State: closed - Opened by penguine-ip about 1 month ago
- 1 comment
#1065 - Fix missing "skip_on_missing_params"
Pull Request -
State: closed - Opened by AdrienDuff about 1 month ago
- 1 comment
#1064 - Why are the scorer tests commented out?
Issue -
State: closed - Opened by jtquach1 about 1 month ago
- 1 comment
#1063 - red-teaming fix
Pull Request -
State: closed - Opened by kritinv about 1 month ago
- 1 comment
#1062 - TypeError: object AIMessage can't be used in 'await' expression
Issue -
State: closed - Opened by kofboy2000 about 1 month ago
- 1 comment
#1061 - new release
Pull Request -
State: closed - Opened by penguine-ip about 1 month ago
- 1 comment
#1060 - made functional
Pull Request -
State: closed - Opened by penguine-ip about 1 month ago
- 1 comment
#1059 - clean api
Pull Request -
State: closed - Opened by penguine-ip about 1 month ago
- 1 comment
#1058 - fix(deepeval/metrics): catch attribute error on metric async evaluation
Pull Request -
State: closed - Opened by ottingbob about 1 month ago
- 7 comments
#1057 - Support function calling fields (`tools_called` and `expected_tools`) in Golden
Pull Request -
State: closed - Opened by aandyw about 1 month ago
- 1 comment
#1056 - Adding variable-length arguments and keyword arguments support for `load_benchmark_dataset`
Pull Request -
State: closed - Opened by aandyw about 1 month ago
- 1 comment
#1053 - Update `load_benchmark_dataset` to ensure correct typings for method implementations
Issue -
State: closed - Opened by aandyw about 1 month ago
#1052 - Support function calling fields `expected_tools` and `tools_called` in`Golden`
Issue -
State: closed - Opened by aandyw about 1 month ago
- 1 comment
#1046 - can we disable telemetry by default in python client?
Issue -
State: closed - Opened by pritamsoni-hsr about 1 month ago
- 3 comments
#1043 - Added Semaphore for restricting amount of concurrent LLM calls
Pull Request -
State: open - Opened by KolodziejczykWaldemar about 1 month ago
- 1 comment
#1042 - Unable to create API keys, website reloading again and again.
Issue -
State: closed - Opened by aarushiksk about 2 months ago
- 1 comment