Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / EleutherAI/lm-evaluation-harness issues and pull requests

#298 - Add `language` argument

Issue - State: closed - Opened by hyunwoongko over 2 years ago

#293 - How to use T5 related models for evaluation?

Issue - State: closed - Opened by YU-Anthony over 2 years ago - 3 comments

#251 - Fix blimp tests

Issue - State: closed - Opened by leogao2 over 2 years ago - 4 comments

#248 - Review all GLUE + SuperGLUE tasks.

Issue - State: closed - Opened by zphang over 2 years ago - 2 comments
Labels: opinions wanted

#241 - Multilingual StoryCloze

Issue - State: closed - Opened by StellaAthena over 2 years ago - 4 comments
Labels: feature request, good first issue

#238 - Error upon running coqa

Issue - State: closed - Opened by robinsongh381 over 2 years ago - 1 comment

#233 - Megatron DS (hacky)

Pull Request - State: open - Opened by zphang over 2 years ago - 4 comments

#231 - Implement GPT-3 style contamination study

Issue - State: closed - Opened by StellaAthena over 2 years ago - 10 comments
Labels: feature request

#200 - Add XQuAD + MLQA

Pull Request - State: open - Opened by sdtblck about 3 years ago - 1 comment

#199 - Add XNLI

Issue - State: closed - Opened by sdtblck about 3 years ago
Labels: feature request

#193 - Implement TyDiQA

Issue - State: open - Opened by sdtblck about 3 years ago
Labels: help wanted, feature request, good first issue

#192 - Implement MLQA

Issue - State: open - Opened by sdtblck about 3 years ago
Labels: help wanted, feature request, good first issue

#191 - Implement XQuAD

Issue - State: open - Opened by sdtblck about 3 years ago - 3 comments
Labels: help wanted, feature request, good first issue

#165 - fewshot for datasets where there's only one set

Issue - State: closed - Opened by leogao2 over 3 years ago

#22 - Implement the SuperGLUE evaluation

Issue - State: open - Opened by StellaAthena almost 4 years ago - 9 comments
Labels: feature request

#18 - Implement the QuAC evaluation

Issue - State: closed - Opened by StellaAthena almost 4 years ago - 6 comments
Labels: feature request, good first issue

#9 - Implement the Natural Questions evaluation

Issue - State: closed - Opened by StellaAthena almost 4 years ago - 19 comments
Labels: feature request, good first issue