Ecosyste.ms: Issues
An open API service for providing issue and pull request metadata for open source projects.
GitHub / bigcode-project/bigcodebench issues and pull requests
#50 - Pass@100 not supported?
Issue -
State: open - Opened by JohnLins 14 days ago
- 1 comment
#49 - Add status code to mock response for BigCodeBench tasks 211 and 215
Pull Request -
State: open - Opened by hvaara 15 days ago
#48 - Await futures in progress checker
Pull Request -
State: open - Opened by hvaara 15 days ago
#47 - No instructions for evaluating for pass@k where k is greator than 1
Issue -
State: closed - Opened by JohnLins 16 days ago
#46 - Batch generation (to be released)
Issue -
State: open - Opened by terryyz 19 days ago
#45 - [Roadmap] BigCodeBench Q3 2024 Roadmap
Issue -
State: open - Opened by terryyz 19 days ago
#44 - Upcoming Leaderboard Model Update
Issue -
State: closed - Opened by terryyz 19 days ago
- 1 comment
#43 - 🤗 [REQUEST] - Reflection-Llama-3.1-70B
Issue -
State: closed - Opened by sammcj 24 days ago
- 1 comment
#42 - [bug] Lingering processes after eval run completion
Issue -
State: open - Opened by hvaara 25 days ago
- 1 comment
#41 - [docs] Clarify risks of running evals locally.
Issue -
State: open - Opened by hvaara 26 days ago
#40 - [docs] Add working end-to-end example for a local model
Issue -
State: open - Opened by hvaara 26 days ago
- 1 comment
#39 - Add option to report general issues
Issue -
State: closed - Opened by hvaara 26 days ago
- 2 comments
#38 - Remove extra period in task BigCodeBench/16
Pull Request -
State: closed - Opened by hvaara 28 days ago
- 3 comments
#37 - add multiprocessing support for sanitization step
Pull Request -
State: closed - Opened by sk-g about 2 months ago
- 1 comment
#36 - Batch size configuration when using greedy decoding
Issue -
State: closed - Opened by YSLIU627 about 2 months ago
- 4 comments
#35 - 🐛 [TaskRepair] - 760 Imports
Issue -
State: closed - Opened by terryyz 2 months ago
- 1 comment
#34 - [Question] Evaluation by docker does not work
Issue -
State: closed - Opened by VichyTong 2 months ago
- 5 comments
Labels: bug
#33 - 🐛 [TestRemoval/TestRepair] - 211, 215- include status code in mock response
Issue -
State: open - Opened by dmelcer9 2 months ago
- 3 comments
Labels: bug
#32 - 🤗 [REQUEST] - InternLM2.5 7B
Issue -
State: open - Opened by ethanc8 2 months ago
- 1 comment
#31 - Problem Categories
Issue -
State: open - Opened by normster 2 months ago
- 4 comments
Labels: question
#30 - 🐛 [TaskRemoval/TaskRepair] - 16 Extra period in tests vs prompt
Issue -
State: closed - Opened by dmelcer9 2 months ago
- 2 comments
Labels: bug
#29 - 🤗 [REQUEST] - Mathstral
Issue -
State: closed - Opened by ethanc8 2 months ago
- 3 comments
#28 - 🤗 [REQUEST] - Mistral NeMO
Issue -
State: closed - Opened by ethanc8 2 months ago
- 1 comment
#27 - 🤗 [REQUEST] - Codestral Mamba
Issue -
State: closed - Opened by ethanc8 2 months ago
- 4 comments
#26 - 🤗 [REQUEST] - InverseCoder
Issue -
State: closed - Opened by terryyz 2 months ago
- 4 comments
#25 - Update on v0.1.8
Pull Request -
State: closed - Opened by terryyz 3 months ago
#24 - 🐛 [TaskRemoval/TaskRepair] - 178 import unused library
Issue -
State: closed - Opened by imamnurby 3 months ago
- 4 comments
Labels: bug
#23 - Add network free set
Pull Request -
State: closed - Opened by imamnurby 3 months ago
- 2 comments
#22 - 🤗 [REQUEST] - NTQAI/Nxcode-CQ-7B-orpo
Issue -
State: closed - Opened by bino282 3 months ago
- 4 comments
#21 - (Tokenization) Performance Degradation Starting from Transformers v4.42.*
Issue -
State: closed - Opened by takkyu2 3 months ago
- 17 comments
#20 - Save pass@k result & use custom tokenizer
Pull Request -
State: closed - Opened by marianna13 3 months ago
- 2 comments
#19 - 🤗 [REQUEST] - CodeGeeX-4-9B
Issue -
State: closed - Opened by ethanc8 3 months ago
- 6 comments
#18 - 🤗 [REQUEST] - Gemma 2
Issue -
State: closed - Opened by ethanc8 3 months ago
- 4 comments
#17 - [WIP] v0.1.8 Release
Pull Request -
State: closed - Opened by terryyz 3 months ago
#16 - 🤗 [REQUEST] - multi-token-prediction
Issue -
State: open - Opened by ethanc8 3 months ago
#15 - 🤗 [REQUEST] - Artigenz-Coder-DS-6.7B
Issue -
State: closed - Opened by ethanc8 3 months ago
- 1 comment
#14 - [Feature] BigCodeBench-Hard is coming...
Issue -
State: closed - Opened by terryyz 3 months ago
#13 - Very low pass@1
Issue -
State: closed - Opened by marianna13 3 months ago
- 15 comments
#12 - Update evaluate.py - Calculating gt_pass_rate befre evaluating the expression "gt_pass_rate > 0.99"
Pull Request -
State: closed - Opened by lapidshay 3 months ago
- 2 comments
#11 - 🤗 [REQUEST] - ReflectionCoder
Issue -
State: closed - Opened by ethanc8 3 months ago
- 6 comments
#10 - 🤗 [REQUEST] - AutoCoder
Issue -
State: closed - Opened by ethanc8 3 months ago
- 2 comments
#9 - release: v0.1.6
Pull Request -
State: closed - Opened by terryyz 3 months ago
#8 - ValueError: not allowed to raise maximum limit
Issue -
State: closed - Opened by dmelcer9 3 months ago
- 12 comments
Labels: enhancement, good first issue
#7 - 🤗 Evaluation - microsoft/wavecoder-ultra-6.7b
Issue -
State: closed - Opened by iam-samasif 3 months ago
- 2 comments
#6 - Fix links to github.io
Pull Request -
State: closed - Opened by egor-bogomolov 3 months ago
- 1 comment
#5 - 🤗 [REQUEST] - LLaMa2 7B HHRLHF QLoRA
Issue -
State: closed - Opened by Sharan1712 3 months ago
- 4 comments
#4 - fix: update the generations to v0.1.5
Pull Request -
State: closed - Opened by terryyz 3 months ago
#3 - fix: change issues
Pull Request -
State: closed - Opened by terryyz 3 months ago
#2 - Release BigCodeBench v0.1.5
Pull Request -
State: closed - Opened by terryyz 3 months ago
#1 - Add generation docker and some common sense options
Pull Request -
State: closed - Opened by iNeil77 4 months ago