Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / google/BIG-bench issues and pull requests

#100 - StrategyQA

Pull Request - State: closed - Opened by mega002 over 3 years ago - 15 comments
Labels: task submission

#99 - [idea for task] gender sensitivity test in your language

Issue - State: open - Opened by XudongOliverShen over 3 years ago
Labels: Idea for task

#98 - gender sensitivity test Chinese

Pull Request - State: closed - Opened by XudongOliverShen over 3 years ago - 12 comments
Labels: task submission

#97 - 2 tasks based on atoms / the periodic table

Pull Request - State: closed - Opened by lukemetz over 3 years ago - 15 comments
Labels: task submission

#96 - Added Question-Answer creation and self-evaluation task.

Pull Request - State: closed - Opened by lewkowycz over 3 years ago - 15 comments
Labels: task submission

#95 - Add repeat copy logic task (in json).

Pull Request - State: closed - Opened by nfiedel over 3 years ago - 8 comments
Labels: task submission

#94 - gender sensitivity test English

Pull Request - State: closed - Opened by XudongOliverShen over 3 years ago - 18 comments
Labels: task submission

#93 - Roots optimization and games

Pull Request - State: closed - Opened by daniellevy over 3 years ago - 7 comments
Labels: task submission

#92 - gender sensitivity test English

Pull Request - State: closed - Opened by XudongOliverShen over 3 years ago - 2 comments

#91 - Add cifar10-test-base64 (Author)

Pull Request - State: closed - Opened by liaopeiyuan over 3 years ago - 18 comments

#90 - add twenty questions task

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago - 11 comments
Labels: task submission

#89 - Created the BIG-bench version of the WinoWhy task

Pull Request - State: closed - Opened by colinzhaoust over 3 years ago - 11 comments
Labels: task submission

#88 - Normalize targets per input

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#87 - wrong normalisation for model.cond_log_prob()

Issue - State: closed - Opened by XudongOliverShen over 3 years ago - 1 comment
Labels: bug

#86 - Update README.md

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#85 - Add cifar10-test-base64

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago - 5 comments
Labels: task submission

#84 - Revert "Added cifar10-test-base64"

Pull Request - State: closed - Opened by timothyn617 over 3 years ago

#83 - Add ambiguous sentences and clarifying questions task

Pull Request - State: closed - Opened by mapmeld over 3 years ago - 14 comments
Labels: task submission

#82 - Permute choices when presenting examples.

Pull Request - State: closed - Opened by lewkowycz over 3 years ago

#81 - [idea for task] Can a language model drive a car?

Issue - State: open - Opened by RomanPlusPlus over 3 years ago - 2 comments
Labels: Idea for task

#80 - improve `task_prefix` JSON field description

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#79 - Update doc.md

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#78 - Fixed formatting in several files

Pull Request - State: closed - Opened by cdfreeman-google over 3 years ago

#77 - Formatting fixes

Pull Request - State: closed - Opened by cdfreeman-google over 3 years ago - 1 comment

#76 - Update README.md

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#75 - Add documentation for stop_string and output_regex

Pull Request - State: closed - Opened by guygurari over 3 years ago

#74 - Update human_model.py

Pull Request - State: closed - Opened by cdfreeman-google over 3 years ago

#73 - Improvements to description of task review process

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#70 - Update human_model.py

Pull Request - State: closed - Opened by cdfreeman-google over 3 years ago

#69 - Adding Checkmate in One Task

Pull Request - State: closed - Opened by keskarnitish over 3 years ago - 16 comments
Labels: task submission

#68 - Adding the ASCII word recognition task

Pull Request - State: closed - Opened by keskarnitish over 3 years ago - 10 comments
Labels: task submission

#67 - [idea for task] language models for (super-)forecasting

Issue - State: open - Opened by duck-master over 3 years ago - 2 comments
Labels: Idea for task

#66 - A suggestion: a crowdsourced list of novel benchmark ideas

Issue - State: closed - Opened by RomanPlusPlus over 3 years ago - 6 comments

#65 - Change in formatting json task.

Pull Request - State: closed - Opened by lewkowycz over 3 years ago - 1 comment

#64 - Added code_line_description_json_task

Pull Request - State: closed - Opened by gokyori over 3 years ago - 15 comments
Labels: task submission

#63 - Add Alignment Questionnaire task

Pull Request - State: closed - Opened by RomanPlusPlus over 3 years ago - 21 comments
Labels: task submission

#62 - handle non-str model outputs by converting to str

Pull Request - State: closed - Opened by guygurari over 3 years ago

#61 - Added support for multiple targets in json tasks.

Pull Request - State: closed - Opened by lewkowycz over 3 years ago - 1 comment

#60 - Added support for `output_regex` and `stop_string` as json keys.

Pull Request - State: closed - Opened by lewkowycz over 3 years ago - 2 comments

#59 - Text-to-text task, multiple responses

Issue - State: closed - Opened by mapmeld over 3 years ago - 2 comments

#58 - nit language fixes

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#57 - Added cifar10-test-base64

Pull Request - State: closed - Opened by liaopeiyuan over 3 years ago - 3 comments
Labels: task submission

#56 - Added discourse_marker_prediction

Pull Request - State: closed - Opened by sileod over 3 years ago - 10 comments
Labels: task submission

#55 - Update doc.md

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#54 - Added einstein_puzzle

Pull Request - State: closed - Opened by ADMoreau over 3 years ago - 9 comments
Labels: task submission

#53 - Add odd-one-out multiple choice JSON task

Pull Request - State: closed - Opened by mosegui over 3 years ago - 15 comments
Labels: task submission

#52 - include english idioms task. tests passed

Pull Request - State: closed - Opened by mosegui over 3 years ago - 12 comments
Labels: task submission

#51 - Add FAQ section to the readme

Pull Request - State: closed - Opened by guygurari over 3 years ago

#50 - Make scripts work independently of execution directory

Pull Request - State: closed - Opened by guygurari over 3 years ago

#49 - Test multiple choice with varying option count.

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago - 2 comments
Labels: bug

#48 - Minor formatting fix in docstring

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#47 - Update task.py

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#46 - Update test_tasks.py

Pull Request - State: closed - Opened by cdfreeman-google over 3 years ago

#45 - test task behavior for zero-probability sequences

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#44 - Provide template README.md file, improve review criteria

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#43 - Added interactive human evaluation.

Pull Request - State: closed - Opened by lewkowycz over 3 years ago

#42 - Added My_Task

Pull Request - State: closed - Opened by DataCrusade1999 over 3 years ago - 1 comment

#41 - Long context length example task.

Pull Request - State: closed - Opened by lewkowycz over 3 years ago

#40 - Updated json_task to allow easier inheritance of class.

Pull Request - State: closed - Opened by lewkowycz over 3 years ago

#39 - Clarify novelty criterion for task submission

Pull Request - State: closed - Opened by guygurari over 3 years ago

#38 - Aslone/gpu docker

Pull Request - State: closed - Opened by ajslone over 3 years ago - 1 comment

#37 - Does a task submission have to be novel?

Issue - State: closed - Opened by yuchenlin over 3 years ago - 5 comments

#36 - Enforce standardized formatting

Pull Request - State: closed - Opened by vedant over 3 years ago - 5 comments

#35 - Add photo of Alan Turing sitting on a bench

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#34 - Fix bug in evaluate_task.py and refactoring

Pull Request - State: closed - Opened by guygurari over 3 years ago - 1 comment

#33 - Updated colab for tf to be compatible with CUDA.

Pull Request - State: closed - Opened by lewkowycz over 3 years ago

#32 - change absl version to 0.10 to fix version mismatch

Pull Request - State: closed - Opened by guygurari over 3 years ago

#31 - Update to evaluate_model() API

Pull Request - State: closed - Opened by guygurari over 3 years ago - 2 comments

#30 - Update conditional probability function

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#29 - Added tests and fixed some naming issues.

Pull Request - State: closed - Opened by lewkowycz over 3 years ago

#28 - Move location of colab notebook

Pull Request - State: closed - Opened by jaehlee over 3 years ago

#27 - Update Readme.md to include reference to the colab

Pull Request - State: closed - Opened by jaehlee over 3 years ago

#26 - setting scipy, scikit-learn versions

Pull Request - State: closed - Opened by ajslone over 3 years ago

#25 - remove spaces from task, example prefixes and change default prefix

Pull Request - State: closed - Opened by guygurari over 3 years ago

#24 - Fixed some metric inconsistencies.

Pull Request - State: closed - Opened by lewkowycz over 3 years ago

#23 - Moving duplicated requirements to shared file.

Pull Request - State: closed - Opened by mathopus over 3 years ago

#22 - Reinitialization of random.seed before task eval.

Pull Request - State: closed - Opened by lewkowycz over 3 years ago

#21 - Major docs revision

Pull Request - State: closed - Opened by guygurari over 3 years ago - 3 comments

#20 - Guyga docs 1

Pull Request - State: closed - Opened by guygurari over 3 years ago

#19 - Add test to verify all keywords in tasks are also present in keywords.md.

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago - 1 comment

#18 - change to canary test

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago - 1 comment

#17 - simplified deterministic pseudorandom tie breaking

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#16 - Add docker support and task evaluation script

Pull Request - State: closed - Opened by ajslone over 3 years ago - 1 comment

#15 - Test improvements

Pull Request - State: closed - Opened by guygurari over 3 years ago

#14 - Mock PR, do not merge

Pull Request - State: closed - Opened by lewkowycz over 3 years ago

#13 - Tests cleanup

Pull Request - State: closed - Opened by guygurari over 3 years ago - 1 comment

#10 - added missing license headers

Pull Request - State: closed - Opened by ajslone over 3 years ago

#9 - Make categorical accuracy pseudorandom but deterministic in case of ties.

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago - 1 comment

#8 - Adding example colab

Pull Request - State: closed - Opened by jaehlee over 3 years ago - 4 comments

#7 - make tie-breaking deterministic for categorical accuracy

Pull Request - State: closed - Opened by Sohl-Dickstein over 3 years ago

#6 - Update task_metrics.py with random tiebreaker argmax.

Pull Request - State: closed - Opened by lewkowycz over 3 years ago

#5 - Aslone/hf sanity test

Pull Request - State: closed - Opened by ajslone over 3 years ago

#4 - Timothyn617 beta test

Pull Request - State: closed - Opened by timothyn617 over 3 years ago - 2 comments

#3 - Fixed bug in metrics. Added example multiple choice task.

Pull Request - State: closed - Opened by lewkowycz over 3 years ago

#2 - Changes to Python Arithmetic Task

Pull Request - State: closed - Opened by jaehlee over 3 years ago

#1 - Create __init__.py

Pull Request - State: closed - Opened by jaehlee over 3 years ago