Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / embeddings-benchmark/arena issues and pull requests

#45 - [WIP] Personal Leaderboard

Pull Request - State: open - Opened by SaitejaUtpala 4 months ago

#43 - complete adding download button

Pull Request - State: open - Opened by SaitejaUtpala 5 months ago - 4 comments

#42 - allow downloading query embedding

Issue - State: open - Opened by Muennighoff 5 months ago

#41 - Breakdown Across Corpus

Issue - State: open - Opened by zanussbaum 6 months ago

#40 - Diff between retrieved documents

Issue - State: open - Opened by Muennighoff 6 months ago

#39 - Top1 vs Top2 of same model

Issue - State: open - Opened by Muennighoff 6 months ago

#38 - Fix HF repo error when loading bm25 index

Pull Request - State: open - Opened by isaac-chung 6 months ago

#37 - WIP: feat: Add ColBERT

Pull Request - State: open - Opened by bclavie 6 months ago

#36 - Do more to prevent ties

Issue - State: open - Opened by KennethEnevoldsen 6 months ago - 2 comments

#35 - Add normalization kwarg for encode

Pull Request - State: closed - Opened by violenil 7 months ago

#34 - Normalization during retrieval scores computation

Issue - State: open - Opened by violenil 7 months ago - 2 comments

#32 - Support for Retrieval-only models (ColBERT?)

Issue - State: open - Opened by bclavie 7 months ago - 7 comments

#31 - Check "both are bad" / "tie" votes & consider displaying top-K

Issue - State: open - Opened by Muennighoff 7 months ago - 2 comments

#30 - Why is Alibaba-NLP/gte-Qwen2-7B-instruct so bad?

Issue - State: closed - Opened by Muennighoff 7 months ago - 2 comments

#29 - Connect leaderboard with crowdsourcing platform

Issue - State: open - Opened by Muennighoff 7 months ago

#28 - Skip responses when passages are the same?

Issue - State: open - Opened by orionw 7 months ago - 1 comment

#27 - Use the CI in determining ranking numbers

Issue - State: open - Opened by orionw 7 months ago

#26 - fix: nomic embed prefix, v1 to arena

Pull Request - State: closed - Opened by zanussbaum 7 months ago - 1 comment

#25 - Turn MTEB-Arena-logs into HF dataset?

Issue - State: open - Opened by Muennighoff 7 months ago - 4 comments

#24 - KeyError: '### Model A: voyage-multilingual-2'

Issue - State: open - Opened by Muennighoff 7 months ago

#23 - Classification tab

Issue - State: open - Opened by Muennighoff 7 months ago - 1 comment

#22 - Make Clustering more intuitive

Issue - State: open - Opened by Muennighoff 7 months ago

#21 - Nomic Models Not Using Query (or Document) Instructions

Issue - State: closed - Opened by zanussbaum 7 months ago - 9 comments

#20 - Track time people take to vote

Issue - State: open - Opened by Muennighoff 7 months ago - 3 comments

#19 - Add clustering samples

Pull Request - State: closed - Opened by orionw 7 months ago - 1 comment

#18 - Allow non-ascii characters in jsonl

Issue - State: closed - Opened by Muennighoff 7 months ago - 4 comments

#17 - Add leaderboard categories

Issue - State: open - Opened by Muennighoff 7 months ago

#16 - Corpus improvements

Issue - State: open - Opened by Muennighoff 7 months ago

#15 - Wikipedia corpus tables ignored

Issue - State: open - Opened by Muennighoff 7 months ago - 1 comment

#14 - Add personal leaderboard

Issue - State: open - Opened by Muennighoff 7 months ago - 8 comments

#13 - Separate Side-by-Side results and Battle results; Update Arxiv text

Pull Request - State: closed - Opened by orionw 7 months ago - 5 comments

#12 - Final launch TODOs

Issue - State: closed - Opened by Muennighoff 7 months ago - 54 comments

#11 - [WIP] Add bm25s

Pull Request - State: closed - Opened by isaac-chung 8 months ago - 1 comment

#10 - Remove Git LFS from repo

Pull Request - State: open - Opened by orionw 8 months ago

#9 - WIP: Create indices

Pull Request - State: closed - Opened by Muennighoff 8 months ago - 4 comments

#8 - Add chunking for arxiv and stackexchange

Pull Request - State: closed - Opened by orionw 8 months ago

#7 - Wikipedia parsing and chunking

Pull Request - State: closed - Opened by orionw 8 months ago - 3 comments

#6 - Models to support for launch

Issue - State: closed - Opened by Muennighoff 8 months ago - 14 comments

#5 - Allow selecting different corpora in Retrieval

Issue - State: closed - Opened by Muennighoff 8 months ago - 27 comments

#4 - Add share button to Clustering arena

Issue - State: open - Opened by Muennighoff 8 months ago

#3 - Add share button to STS arena

Issue - State: open - Opened by Muennighoff 8 months ago

#2 - Add GCP Vertex search index

Pull Request - State: closed - Opened by isaac-chung 8 months ago - 32 comments

#1 - General feedback

Issue - State: closed - Opened by KennethEnevoldsen 8 months ago - 16 comments