Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / gkamradt/LLMTest_NeedleInAHaystack issues and pull requests

#54 - Feature/add anthropic evaluator

Pull Request - State: closed - Opened by fufufukakaka 3 months ago - 1 comment

#53 - Add support for google gemini

Pull Request - State: open - Opened by vijaykyr 4 months ago - 1 comment

#52 - Bug in Multi_needle and Single_Needle

Pull Request - State: open - Opened by TachikakaMin 5 months ago - 2 comments

#51 - feat: support openai use proxy address

Pull Request - State: open - Opened by csunny 5 months ago

#48 - How can we cite the Needle-in-a-Haystack?

Issue - State: open - Opened by rozyang 6 months ago - 1 comment

#47 - Can I use local LLM as the evaluator and provider?

Issue - State: open - Opened by hijkzzz 7 months ago - 6 comments

#46 - Added Databricks provider

Pull Request - State: open - Opened by akhelou 7 months ago

#45 - Possibility to specify custom API endpoint address?

Issue - State: open - Opened by ElliottDyson 7 months ago - 4 comments

#44 - Question: Can the Haystack have variations?

Issue - State: open - Opened by BradKML 7 months ago - 3 comments

#43 - does it run at all? Basic commands failed to run as per the README.

Issue - State: open - Opened by iitrsamrat 8 months ago - 1 comment

#42 - Add open-source model support & random multi-needle insertion

Pull Request - State: open - Opened by gauss5930 8 months ago - 1 comment
Labels: New provider

#41 - [Feature Proposal] Multi-needle in a haystack

Issue - State: open - Opened by jsharf 8 months ago - 2 comments

#40 - Add cohere model provider

Pull Request - State: closed - Opened by arkadyark-cohere 8 months ago - 2 comments
Labels: New provider

#39 - I was wondering about the evaluation method

Issue - State: closed - Opened by gauss5930 8 months ago - 2 comments

#38 - adding azure openai evaluator

Pull Request - State: open - Opened by LazaroHurtado 8 months ago
Labels: New provider

#37 - fixed minor typos

Pull Request - State: closed - Opened by Wilsven 8 months ago
Labels: enhancement

#36 - Updated Anthropic to use message API (instead of completion API) - resolves #13

Pull Request - State: open - Opened by prabha-git 8 months ago
Labels: enhancement

#35 - feat: Amazon Bedrock provider support for MNIAH testing

Pull Request - State: open - Opened by JGalego 8 months ago
Labels: New provider

#34 - multi-needle-eval-pizza-3 dataset not found

Issue - State: closed - Opened by gkamradt 8 months ago - 1 comment

#33 - PyPi packaging and environment variables for API keys

Pull Request - State: closed - Opened by kedarchandrayan 8 months ago

#31 - Convert the repository to a PyPi package

Issue - State: closed - Opened by kedarchandrayan 8 months ago - 1 comment

#30 - Finalize / update multi-needle viz

Pull Request - State: closed - Opened by rlancemartin 8 months ago - 1 comment

#29 - Fixed a typo in evaluation function

Pull Request - State: open - Opened by BastienZim 8 months ago - 5 comments
Labels: enhancement

#28 - Refactoring needles tester

Pull Request - State: open - Opened by LazaroHurtado 8 months ago - 6 comments
Labels: enhancement

#27 - Add viz ntbk, fix needle insertion, fix prompts

Pull Request - State: closed - Opened by rlancemartin 8 months ago - 3 comments

#26 - Implement Docker and unit tests

Pull Request - State: closed - Opened by pavelkraleu 8 months ago

#25 - Standard Tokenizer

Issue - State: closed - Opened by prabha-git 8 months ago - 12 comments

#24 - added make command to reset test run

Pull Request - State: closed - Opened by LazaroHurtado 8 months ago - 1 comment

#23 - Add Makefile target for resetting run results

Issue - State: closed - Opened by LazaroHurtado 8 months ago - 1 comment

#22 - added model kwargs support

Pull Request - State: closed - Opened by LazaroHurtado 8 months ago - 1 comment

#21 - Model kwargs support

Issue - State: closed - Opened by LazaroHurtado 8 months ago - 1 comment

#20 - Code optimizations

Issue - State: open - Opened by LazaroHurtado 8 months ago

#19 - Add multi needle and LangSmith evaluator

Pull Request - State: closed - Opened by rlancemartin 8 months ago - 12 comments

#18 - set the correct default model name for anthropic

Pull Request - State: closed - Opened by prabha-git 8 months ago - 1 comment

#17 - Implement Docker for testing

Issue - State: open - Opened by pavelkraleu 8 months ago - 1 comment
Labels: enhancement

#16 - using class alias for anthropic provider

Pull Request - State: closed - Opened by LazaroHurtado 8 months ago - 2 comments

#15 - Anthropic Naming Conflict Error

Issue - State: closed - Opened by prabha-git 8 months ago - 2 comments

#14 - Optimizations and readability improvements

Pull Request - State: open - Opened by LazaroHurtado 8 months ago - 7 comments
Labels: enhancement

#13 - Update package Anthropic

Issue - State: open - Opened by pavelkraleu 8 months ago - 2 comments
Labels: enhancement

#12 - Replace os.path with Pathlib

Issue - State: open - Opened by pavelkraleu 8 months ago
Labels: enhancement

#11 - Install pre-commit with end-of-file-fixer

Issue - State: open - Opened by pavelkraleu 8 months ago
Labels: enhancement

#10 - hard coding of 'gpt-4' for evaluation

Issue - State: open - Opened by prabha-git 9 months ago

#9 - Fixed a typo in evaluation function

Pull Request - State: closed - Opened by BastienZim 9 months ago - 8 comments

#8 - dependency injection for models and evaluators

Pull Request - State: closed - Opened by LazaroHurtado 9 months ago - 13 comments

#7 - Code Refactoring

Pull Request - State: closed - Opened by prabha-git 10 months ago - 10 comments

#6 - Phoenix integration

Pull Request - State: closed - Opened by harrisonchu 11 months ago

#5 - Phoenix integration

Pull Request - State: closed - Opened by jlopatec 11 months ago

#4 - Add license file

Issue - State: closed - Opened by haesleinhuepf 11 months ago - 4 comments

#3 - Azure OpenAI key

Issue - State: closed - Opened by cobraheleah 11 months ago - 2 comments

#2 - Update README.md

Pull Request - State: closed - Opened by eltociear 12 months ago

#1 - Can you share the PowerPoint where you mapped the final result?

Issue - State: closed - Opened by jarheadjoe 12 months ago - 1 comment