Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / bigcode-project/bigcodebench issues and pull requests

#50 - Pass@100 not supported?

Issue - State: open - Opened by JohnLins 14 days ago - 1 comment

#48 - Await futures in progress checker

Pull Request - State: open - Opened by hvaara 15 days ago

#46 - Batch generation (to be released)

Issue - State: open - Opened by terryyz 19 days ago

#45 - [Roadmap] BigCodeBench Q3 2024 Roadmap

Issue - State: open - Opened by terryyz 19 days ago

#44 - Upcoming Leaderboard Model Update

Issue - State: closed - Opened by terryyz 19 days ago - 1 comment

#43 - 🤗 [REQUEST] - Reflection-Llama-3.1-70B

Issue - State: closed - Opened by sammcj 24 days ago - 1 comment

#42 - [bug] Lingering processes after eval run completion

Issue - State: open - Opened by hvaara 25 days ago - 1 comment

#41 - [docs] Clarify risks of running evals locally.

Issue - State: open - Opened by hvaara 26 days ago

#40 - [docs] Add working end-to-end example for a local model

Issue - State: open - Opened by hvaara 26 days ago - 1 comment

#39 - Add option to report general issues

Issue - State: closed - Opened by hvaara 26 days ago - 2 comments

#38 - Remove extra period in task BigCodeBench/16

Pull Request - State: closed - Opened by hvaara 28 days ago - 3 comments

#37 - add multiprocessing support for sanitization step

Pull Request - State: closed - Opened by sk-g about 2 months ago - 1 comment

#36 - Batch size configuration when using greedy decoding

Issue - State: closed - Opened by YSLIU627 about 2 months ago - 4 comments

#35 - 🐛 [TaskRepair] - 760 Imports

Issue - State: closed - Opened by terryyz 2 months ago - 1 comment

#34 - [Question] Evaluation by docker does not work

Issue - State: closed - Opened by VichyTong 2 months ago - 5 comments
Labels: bug

#33 - 🐛 [TestRemoval/TestRepair] - 211, 215- include status code in mock response

Issue - State: open - Opened by dmelcer9 2 months ago - 3 comments
Labels: bug

#32 - 🤗 [REQUEST] - InternLM2.5 7B

Issue - State: open - Opened by ethanc8 2 months ago - 1 comment

#31 - Problem Categories

Issue - State: open - Opened by normster 2 months ago - 4 comments
Labels: question

#30 - 🐛 [TaskRemoval/TaskRepair] - 16 Extra period in tests vs prompt

Issue - State: closed - Opened by dmelcer9 2 months ago - 2 comments
Labels: bug

#29 - 🤗 [REQUEST] - Mathstral

Issue - State: closed - Opened by ethanc8 2 months ago - 3 comments

#28 - 🤗 [REQUEST] - Mistral NeMO

Issue - State: closed - Opened by ethanc8 2 months ago - 1 comment

#27 - 🤗 [REQUEST] - Codestral Mamba

Issue - State: closed - Opened by ethanc8 2 months ago - 4 comments

#26 - 🤗 [REQUEST] - InverseCoder

Issue - State: closed - Opened by terryyz 2 months ago - 4 comments

#25 - Update on v0.1.8

Pull Request - State: closed - Opened by terryyz 3 months ago

#24 - 🐛 [TaskRemoval/TaskRepair] - 178 import unused library

Issue - State: closed - Opened by imamnurby 3 months ago - 4 comments
Labels: bug

#23 - Add network free set

Pull Request - State: closed - Opened by imamnurby 3 months ago - 2 comments

#22 - 🤗 [REQUEST] - NTQAI/Nxcode-CQ-7B-orpo

Issue - State: closed - Opened by bino282 3 months ago - 4 comments

#21 - (Tokenization) Performance Degradation Starting from Transformers v4.42.*

Issue - State: closed - Opened by takkyu2 3 months ago - 17 comments

#20 - Save pass@k result & use custom tokenizer

Pull Request - State: closed - Opened by marianna13 3 months ago - 2 comments

#19 - 🤗 [REQUEST] - CodeGeeX-4-9B

Issue - State: closed - Opened by ethanc8 3 months ago - 6 comments

#18 - 🤗 [REQUEST] - Gemma 2

Issue - State: closed - Opened by ethanc8 3 months ago - 4 comments

#17 - [WIP] v0.1.8 Release

Pull Request - State: closed - Opened by terryyz 3 months ago

#16 - 🤗 [REQUEST] - multi-token-prediction

Issue - State: open - Opened by ethanc8 3 months ago

#15 - 🤗 [REQUEST] - Artigenz-Coder-DS-6.7B

Issue - State: closed - Opened by ethanc8 3 months ago - 1 comment

#14 - [Feature] BigCodeBench-Hard is coming...

Issue - State: closed - Opened by terryyz 3 months ago

#13 - Very low pass@1

Issue - State: closed - Opened by marianna13 3 months ago - 15 comments

#11 - 🤗 [REQUEST] - ReflectionCoder

Issue - State: closed - Opened by ethanc8 3 months ago - 6 comments

#10 - 🤗 [REQUEST] - AutoCoder

Issue - State: closed - Opened by ethanc8 3 months ago - 2 comments

#9 - release: v0.1.6

Pull Request - State: closed - Opened by terryyz 3 months ago

#8 - ValueError: not allowed to raise maximum limit

Issue - State: closed - Opened by dmelcer9 3 months ago - 12 comments
Labels: enhancement, good first issue

#7 - 🤗 Evaluation - microsoft/wavecoder-ultra-6.7b

Issue - State: closed - Opened by iam-samasif 3 months ago - 2 comments

#6 - Fix links to github.io

Pull Request - State: closed - Opened by egor-bogomolov 3 months ago - 1 comment

#5 - 🤗 [REQUEST] - LLaMa2 7B HHRLHF QLoRA

Issue - State: closed - Opened by Sharan1712 3 months ago - 4 comments

#4 - fix: update the generations to v0.1.5

Pull Request - State: closed - Opened by terryyz 3 months ago

#3 - fix: change issues

Pull Request - State: closed - Opened by terryyz 3 months ago

#2 - Release BigCodeBench v0.1.5

Pull Request - State: closed - Opened by terryyz 3 months ago

#1 - Add generation docker and some common sense options

Pull Request - State: closed - Opened by iNeil77 4 months ago