Ecosyste.ms: Issues
An open API service for providing issue and pull request metadata for open source projects.
GitHub / gkamradt/LLMTest_NeedleInAHaystack issues and pull requests
#54 - Feature/add anthropic evaluator
Pull Request -
State: closed - Opened by fufufukakaka 3 months ago
- 1 comment
#53 - Add support for google gemini
Pull Request -
State: open - Opened by vijaykyr 4 months ago
- 1 comment
#52 - Bug in Multi_needle and Single_Needle
Pull Request -
State: open - Opened by TachikakaMin 5 months ago
- 2 comments
#51 - feat: support openai use proxy address
Pull Request -
State: open - Opened by csunny 5 months ago
#50 - Different prompts in providers - I just wonder why cohere don't have "Don't give information outside the document or repeat your findings" and does it make a bit difference?
Issue -
State: closed - Opened by radarFudan 6 months ago
#49 - add base_url env in openai provider - to support OpenAI compatibility local inference like - ollama, tgi, etc
Issue -
State: open - Opened by backroom-coder 6 months ago
#48 - How can we cite the Needle-in-a-Haystack?
Issue -
State: open - Opened by rozyang 6 months ago
- 1 comment
#47 - Can I use local LLM as the evaluator and provider?
Issue -
State: open - Opened by hijkzzz 7 months ago
- 6 comments
#46 - Added Databricks provider
Pull Request -
State: open - Opened by akhelou 7 months ago
#45 - Possibility to specify custom API endpoint address?
Issue -
State: open - Opened by ElliottDyson 7 months ago
- 4 comments
#44 - Question: Can the Haystack have variations?
Issue -
State: open - Opened by BradKML 7 months ago
- 3 comments
#43 - does it run at all? Basic commands failed to run as per the README.
Issue -
State: open - Opened by iitrsamrat 8 months ago
- 1 comment
#42 - Add open-source model support & random multi-needle insertion
Pull Request -
State: open - Opened by gauss5930 8 months ago
- 1 comment
Labels: New provider
#41 - [Feature Proposal] Multi-needle in a haystack
Issue -
State: open - Opened by jsharf 8 months ago
- 2 comments
#40 - Add cohere model provider
Pull Request -
State: closed - Opened by arkadyark-cohere 8 months ago
- 2 comments
Labels: New provider
#39 - I was wondering about the evaluation method
Issue -
State: closed - Opened by gauss5930 8 months ago
- 2 comments
#38 - adding azure openai evaluator
Pull Request -
State: open - Opened by LazaroHurtado 8 months ago
Labels: New provider
#37 - fixed minor typos
Pull Request -
State: closed - Opened by Wilsven 8 months ago
Labels: enhancement
#36 - Updated Anthropic to use message API (instead of completion API) - resolves #13
Pull Request -
State: open - Opened by prabha-git 8 months ago
Labels: enhancement
#35 - feat: Amazon Bedrock provider support for MNIAH testing
Pull Request -
State: open - Opened by JGalego 8 months ago
Labels: New provider
#34 - multi-needle-eval-pizza-3 dataset not found
Issue -
State: closed - Opened by gkamradt 8 months ago
- 1 comment
#33 - PyPi packaging and environment variables for API keys
Pull Request -
State: closed - Opened by kedarchandrayan 8 months ago
#32 - Remove passing of API keys as parameters and read them from environment variables
Issue -
State: closed - Opened by kedarchandrayan 8 months ago
- 1 comment
#31 - Convert the repository to a PyPi package
Issue -
State: closed - Opened by kedarchandrayan 8 months ago
- 1 comment
#30 - Finalize / update multi-needle viz
Pull Request -
State: closed - Opened by rlancemartin 8 months ago
- 1 comment
#29 - Fixed a typo in evaluation function
Pull Request -
State: open - Opened by BastienZim 8 months ago
- 5 comments
Labels: enhancement
#28 - Refactoring needles tester
Pull Request -
State: open - Opened by LazaroHurtado 8 months ago
- 6 comments
Labels: enhancement
#27 - Add viz ntbk, fix needle insertion, fix prompts
Pull Request -
State: closed - Opened by rlancemartin 8 months ago
- 3 comments
#26 - Implement Docker and unit tests
Pull Request -
State: closed - Opened by pavelkraleu 8 months ago
#25 - Standard Tokenizer
Issue -
State: closed - Opened by prabha-git 8 months ago
- 12 comments
#24 - added make command to reset test run
Pull Request -
State: closed - Opened by LazaroHurtado 8 months ago
- 1 comment
#23 - Add Makefile target for resetting run results
Issue -
State: closed - Opened by LazaroHurtado 8 months ago
- 1 comment
#22 - added model kwargs support
Pull Request -
State: closed - Opened by LazaroHurtado 8 months ago
- 1 comment
#21 - Model kwargs support
Issue -
State: closed - Opened by LazaroHurtado 8 months ago
- 1 comment
#20 - Code optimizations
Issue -
State: open - Opened by LazaroHurtado 8 months ago
#19 - Add multi needle and LangSmith evaluator
Pull Request -
State: closed - Opened by rlancemartin 8 months ago
- 12 comments
#18 - set the correct default model name for anthropic
Pull Request -
State: closed - Opened by prabha-git 8 months ago
- 1 comment
#17 - Implement Docker for testing
Issue -
State: open - Opened by pavelkraleu 8 months ago
- 1 comment
Labels: enhancement
#16 - using class alias for anthropic provider
Pull Request -
State: closed - Opened by LazaroHurtado 8 months ago
- 2 comments
#15 - Anthropic Naming Conflict Error
Issue -
State: closed - Opened by prabha-git 8 months ago
- 2 comments
#14 - Optimizations and readability improvements
Pull Request -
State: open - Opened by LazaroHurtado 8 months ago
- 7 comments
Labels: enhancement
#13 - Update package Anthropic
Issue -
State: open - Opened by pavelkraleu 8 months ago
- 2 comments
Labels: enhancement
#12 - Replace os.path with Pathlib
Issue -
State: open - Opened by pavelkraleu 8 months ago
Labels: enhancement
#11 - Install pre-commit with end-of-file-fixer
Issue -
State: open - Opened by pavelkraleu 8 months ago
Labels: enhancement
#10 - hard coding of 'gpt-4' for evaluation
Issue -
State: open - Opened by prabha-git 9 months ago
#9 - Fixed a typo in evaluation function
Pull Request -
State: closed - Opened by BastienZim 9 months ago
- 8 comments
#8 - dependency injection for models and evaluators
Pull Request -
State: closed - Opened by LazaroHurtado 9 months ago
- 13 comments
#7 - Code Refactoring
Pull Request -
State: closed - Opened by prabha-git 10 months ago
- 10 comments
#6 - Phoenix integration
Pull Request -
State: closed - Opened by harrisonchu 11 months ago
#5 - Phoenix integration
Pull Request -
State: closed - Opened by jlopatec 11 months ago
#4 - Add license file
Issue -
State: closed - Opened by haesleinhuepf 11 months ago
- 4 comments
#3 - Azure OpenAI key
Issue -
State: closed - Opened by cobraheleah 11 months ago
- 2 comments
#2 - Update README.md
Pull Request -
State: closed - Opened by eltociear 12 months ago
#1 - Can you share the PowerPoint where you mapped the final result?
Issue -
State: closed - Opened by jarheadjoe 12 months ago
- 1 comment