Ecosyste.ms: Issues
An open API service for providing issue and pull request metadata for open source projects.
GitHub / google/BIG-bench issues and pull requests
#100 - StrategyQA
Pull Request -
State: closed - Opened by mega002 over 3 years ago
- 15 comments
Labels: task submission
#99 - [idea for task] gender sensitivity test in your language
Issue -
State: open - Opened by XudongOliverShen over 3 years ago
Labels: Idea for task
#98 - gender sensitivity test Chinese
Pull Request -
State: closed - Opened by XudongOliverShen over 3 years ago
- 12 comments
Labels: task submission
#97 - 2 tasks based on atoms / the periodic table
Pull Request -
State: closed - Opened by lukemetz over 3 years ago
- 15 comments
Labels: task submission
#96 - Added Question-Answer creation and self-evaluation task.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
- 15 comments
Labels: task submission
#95 - Add repeat copy logic task (in json).
Pull Request -
State: closed - Opened by nfiedel over 3 years ago
- 8 comments
Labels: task submission
#94 - gender sensitivity test English
Pull Request -
State: closed - Opened by XudongOliverShen over 3 years ago
- 18 comments
Labels: task submission
#93 - Roots optimization and games
Pull Request -
State: closed - Opened by daniellevy over 3 years ago
- 7 comments
Labels: task submission
#92 - gender sensitivity test English
Pull Request -
State: closed - Opened by XudongOliverShen over 3 years ago
- 2 comments
#91 - Add cifar10-test-base64 (Author)
Pull Request -
State: closed - Opened by liaopeiyuan over 3 years ago
- 18 comments
#90 - add twenty questions task
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
- 11 comments
Labels: task submission
#89 - Created the BIG-bench version of the WinoWhy task
Pull Request -
State: closed - Opened by colinzhaoust over 3 years ago
- 11 comments
Labels: task submission
#88 - Normalize targets per input
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#87 - wrong normalisation for model.cond_log_prob()
Issue -
State: closed - Opened by XudongOliverShen over 3 years ago
- 1 comment
Labels: bug
#86 - Update README.md
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#85 - Add cifar10-test-base64
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
- 5 comments
Labels: task submission
#84 - Revert "Added cifar10-test-base64"
Pull Request -
State: closed - Opened by timothyn617 over 3 years ago
#83 - Add ambiguous sentences and clarifying questions task
Pull Request -
State: closed - Opened by mapmeld over 3 years ago
- 14 comments
Labels: task submission
#82 - Permute choices when presenting examples.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
#81 - [idea for task] Can a language model drive a car?
Issue -
State: open - Opened by RomanPlusPlus over 3 years ago
- 2 comments
Labels: Idea for task
#80 - improve `task_prefix` JSON field description
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#79 - Update doc.md
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#78 - Fixed formatting in several files
Pull Request -
State: closed - Opened by cdfreeman-google over 3 years ago
#77 - Formatting fixes
Pull Request -
State: closed - Opened by cdfreeman-google over 3 years ago
- 1 comment
#76 - Update README.md
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#75 - Add documentation for stop_string and output_regex
Pull Request -
State: closed - Opened by guygurari over 3 years ago
#74 - Update human_model.py
Pull Request -
State: closed - Opened by cdfreeman-google over 3 years ago
#73 - Improvements to description of task review process
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#72 - [idea for task] Script-independent language identification for 1522 languages
Issue -
State: closed - Opened by RomanPlusPlus over 3 years ago
- 1 comment
#71 - [idea for task] Guess the chemical properties of a protein given its sequence of amino acids
Issue -
State: open - Opened by RomanPlusPlus over 3 years ago
Labels: Idea for task
#70 - Update human_model.py
Pull Request -
State: closed - Opened by cdfreeman-google over 3 years ago
#69 - Adding Checkmate in One Task
Pull Request -
State: closed - Opened by keskarnitish over 3 years ago
- 16 comments
Labels: task submission
#68 - Adding the ASCII word recognition task
Pull Request -
State: closed - Opened by keskarnitish over 3 years ago
- 10 comments
Labels: task submission
#67 - [idea for task] language models for (super-)forecasting
Issue -
State: open - Opened by duck-master over 3 years ago
- 2 comments
Labels: Idea for task
#66 - A suggestion: a crowdsourced list of novel benchmark ideas
Issue -
State: closed - Opened by RomanPlusPlus over 3 years ago
- 6 comments
#65 - Change in formatting json task.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
- 1 comment
#64 - Added code_line_description_json_task
Pull Request -
State: closed - Opened by gokyori over 3 years ago
- 15 comments
Labels: task submission
#63 - Add Alignment Questionnaire task
Pull Request -
State: closed - Opened by RomanPlusPlus over 3 years ago
- 21 comments
Labels: task submission
#62 - handle non-str model outputs by converting to str
Pull Request -
State: closed - Opened by guygurari over 3 years ago
#61 - Added support for multiple targets in json tasks.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
- 1 comment
#60 - Added support for `output_regex` and `stop_string` as json keys.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
- 2 comments
#59 - Text-to-text task, multiple responses
Issue -
State: closed - Opened by mapmeld over 3 years ago
- 2 comments
#58 - nit language fixes
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#57 - Added cifar10-test-base64
Pull Request -
State: closed - Opened by liaopeiyuan over 3 years ago
- 3 comments
Labels: task submission
#56 - Added discourse_marker_prediction
Pull Request -
State: closed - Opened by sileod over 3 years ago
- 10 comments
Labels: task submission
#55 - Update doc.md
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#54 - Added einstein_puzzle
Pull Request -
State: closed - Opened by ADMoreau over 3 years ago
- 9 comments
Labels: task submission
#53 - Add odd-one-out multiple choice JSON task
Pull Request -
State: closed - Opened by mosegui over 3 years ago
- 15 comments
Labels: task submission
#52 - include english idioms task. tests passed
Pull Request -
State: closed - Opened by mosegui over 3 years ago
- 12 comments
Labels: task submission
#51 - Add FAQ section to the readme
Pull Request -
State: closed - Opened by guygurari over 3 years ago
#50 - Make scripts work independently of execution directory
Pull Request -
State: closed - Opened by guygurari over 3 years ago
#49 - Test multiple choice with varying option count.
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
- 2 comments
Labels: bug
#48 - Minor formatting fix in docstring
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#47 - Update task.py
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#46 - Update test_tasks.py
Pull Request -
State: closed - Opened by cdfreeman-google over 3 years ago
#45 - test task behavior for zero-probability sequences
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#44 - Provide template README.md file, improve review criteria
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#43 - Added interactive human evaluation.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
#42 - Added My_Task
Pull Request -
State: closed - Opened by DataCrusade1999 over 3 years ago
- 1 comment
#41 - Long context length example task.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
#40 - Updated json_task to allow easier inheritance of class.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
#39 - Clarify novelty criterion for task submission
Pull Request -
State: closed - Opened by guygurari over 3 years ago
#38 - Aslone/gpu docker
Pull Request -
State: closed - Opened by ajslone over 3 years ago
- 1 comment
#37 - Does a task submission have to be novel?
Issue -
State: closed - Opened by yuchenlin over 3 years ago
- 5 comments
#36 - Enforce standardized formatting
Pull Request -
State: closed - Opened by vedant over 3 years ago
- 5 comments
#35 - Add photo of Alan Turing sitting on a bench
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#34 - Fix bug in evaluate_task.py and refactoring
Pull Request -
State: closed - Opened by guygurari over 3 years ago
- 1 comment
#33 - Updated colab for tf to be compatible with CUDA.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
#32 - change absl version to 0.10 to fix version mismatch
Pull Request -
State: closed - Opened by guygurari over 3 years ago
#31 - Update to evaluate_model() API
Pull Request -
State: closed - Opened by guygurari over 3 years ago
- 2 comments
#30 - Update conditional probability function
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#29 - Added tests and fixed some naming issues.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
#28 - Move location of colab notebook
Pull Request -
State: closed - Opened by jaehlee over 3 years ago
#27 - Update Readme.md to include reference to the colab
Pull Request -
State: closed - Opened by jaehlee over 3 years ago
#26 - setting scipy, scikit-learn versions
Pull Request -
State: closed - Opened by ajslone over 3 years ago
#25 - remove spaces from task, example prefixes and change default prefix
Pull Request -
State: closed - Opened by guygurari over 3 years ago
#24 - Fixed some metric inconsistencies.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
#23 - Moving duplicated requirements to shared file.
Pull Request -
State: closed - Opened by mathopus over 3 years ago
#22 - Reinitialization of random.seed before task eval.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
#21 - Major docs revision
Pull Request -
State: closed - Opened by guygurari over 3 years ago
- 3 comments
#20 - Guyga docs 1
Pull Request -
State: closed - Opened by guygurari over 3 years ago
#19 - Add test to verify all keywords in tasks are also present in keywords.md.
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
- 1 comment
#18 - change to canary test
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
- 1 comment
#17 - simplified deterministic pseudorandom tie breaking
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#16 - Add docker support and task evaluation script
Pull Request -
State: closed - Opened by ajslone over 3 years ago
- 1 comment
#15 - Test improvements
Pull Request -
State: closed - Opened by guygurari over 3 years ago
#14 - Mock PR, do not merge
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
#13 - Tests cleanup
Pull Request -
State: closed - Opened by guygurari over 3 years ago
- 1 comment
#12 - Added standardized post-processing for model outputs, as well as support for regex matching. Misc small fixes.
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
- 1 comment
#11 - Changes the generation of few shot examples to be more sample efficient.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
#10 - added missing license headers
Pull Request -
State: closed - Opened by ajslone over 3 years ago
#9 - Make categorical accuracy pseudorandom but deterministic in case of ties.
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
- 1 comment
#8 - Adding example colab
Pull Request -
State: closed - Opened by jaehlee over 3 years ago
- 4 comments
#7 - make tie-breaking deterministic for categorical accuracy
Pull Request -
State: closed - Opened by Sohl-Dickstein over 3 years ago
#6 - Update task_metrics.py with random tiebreaker argmax.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
#5 - Aslone/hf sanity test
Pull Request -
State: closed - Opened by ajslone over 3 years ago
#4 - Timothyn617 beta test
Pull Request -
State: closed - Opened by timothyn617 over 3 years ago
- 2 comments
#3 - Fixed bug in metrics. Added example multiple choice task.
Pull Request -
State: closed - Opened by lewkowycz over 3 years ago
#2 - Changes to Python Arithmetic Task
Pull Request -
State: closed - Opened by jaehlee over 3 years ago
#1 - Create __init__.py
Pull Request -
State: closed - Opened by jaehlee over 3 years ago