Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / huggingface/evaluate issues and pull requests

#650 - docs: correct param name in visualization docstring🐛

Pull Request - State: open - Opened by KarelZe 1 day ago

#649 - Evaluation took so much disk

Issue - State: open - Opened by uluumbch 4 days ago

#647 - TypeError: 'NoneType' object is not callable

Issue - State: open - Opened by Yichangfa 26 days ago - 1 comment

#646 - Fix typo: Remove redundant trailing quote

Pull Request - State: open - Opened by ftnext about 1 month ago

#644 - [FR] Binoculars as a new metric

Issue - State: open - Opened by MayankChaturvedi about 1 month ago

#644 - [FR] Binoculars as a new metric

Issue - State: open - Opened by MayankChaturvedi about 1 month ago

#643 - Perplexity for quantized models

Issue - State: open - Opened by Hongao0611 about 1 month ago - 1 comment

#643 - Perplexity for quantized models

Issue - State: open - Opened by Hongao0611 about 1 month ago - 1 comment

#641 - Optionally stop data deletion after compute

Issue - State: open - Opened by sizhky 2 months ago

#638 - Evaluate fails to load all metrics.

Issue - State: closed - Opened by filbeofITK 2 months ago - 2 comments

#637 - Add ROUGE detailed option

Pull Request - State: open - Opened by eranhirs 2 months ago

#636 - Perplexity for Left Padded Models

Issue - State: open - Opened by Lawhy 2 months ago

#635 - Add required hf_token secret to build main documentation

Pull Request - State: closed - Opened by albertvillanova 2 months ago

#634 - Main documentation building is not triggered

Issue - State: closed - Opened by albertvillanova 2 months ago

#633 - Pass trust_remote_code to fix test_token_classification_parity

Pull Request - State: closed - Opened by albertvillanova 2 months ago - 1 comment

#631 - Pin numpy<2 as required by tensorflow to fix doc building

Pull Request - State: closed - Opened by albertvillanova 2 months ago - 1 comment

#629 - Support nltk>=3.9 to fix vulnerability

Pull Request - State: closed - Opened by albertvillanova 2 months ago

#628 - Support nltk>=3.9 to fix vulnerability

Issue - State: closed - Opened by albertvillanova 2 months ago

#627 - set dev version

Pull Request - State: closed - Opened by lhoestq 3 months ago

#626 - version 0.4.3

Pull Request - State: closed - Opened by lhoestq 3 months ago

#625 - fix run_ner test

Pull Request - State: closed - Opened by lhoestq 3 months ago - 1 comment

#624 - remove ignore_url_params

Pull Request - State: closed - Opened by lhoestq 3 months ago

#623 - Fix CI with temporary pin nltk<3.9

Pull Request - State: closed - Opened by albertvillanova 3 months ago

#621 - Replace deprecated use_auth_token with token

Pull Request - State: closed - Opened by albertvillanova 3 months ago

#619 - Correct the warning information of loader.py

Pull Request - State: open - Opened by HelloWorldLTY 3 months ago - 2 comments

#618 - add tip in docs and readme referring to lighteval

Pull Request - State: closed - Opened by MoritzLaurer 3 months ago

#616 - `list_evaluation_modules` returns empty list

Issue - State: open - Opened by MohamedAliRashad 3 months ago - 2 comments

#615 - Benchmark evaluation for language models.

Issue - State: open - Opened by mina58 3 months ago - 1 comment

#614 - Fix: message for lack of sklearn

Pull Request - State: open - Opened by ArkiZh 3 months ago

#613 - add vqa_accuracy and CIDEr

Pull Request - State: open - Opened by Kami-chanw 3 months ago - 1 comment

#611 - How to customize my own evaluator and metrics?

Issue - State: closed - Opened by Kami-chanw 4 months ago

#609 - AttributeError: module 'evaluate' has no attribute 'load'

Issue - State: open - Opened by Adesoji1 4 months ago - 7 comments

#608 - fix import error message for sklearn

Pull Request - State: open - Opened by bingwork 4 months ago

#606 - Add `zero_division` to F1 metric

Pull Request - State: open - Opened by TimRepke 5 months ago

#605 - Update MAUVE's readme and citations

Pull Request - State: open - Opened by krishnap25 5 months ago

#604 - Support tuple format in `combine` evaluations

Pull Request - State: open - Opened by shunk031 5 months ago

#602 - Gradio dependency issue

Issue - State: open - Opened by bnaman50 5 months ago - 2 comments

#600 - feat(ci): add trufflehog secrets detection

Pull Request - State: closed - Opened by McPatate 6 months ago

#599 - Addition of SummEval Metric to `evaluate` Library

Pull Request - State: open - Opened by penguinwang96825 6 months ago

#598 - New JSON encoder for results containing numpy types

Pull Request - State: open - Opened by jpodivin 6 months ago

#595 - Add tokenizer initialization to the documented example

Pull Request - State: open - Opened by jpodivin 6 months ago

#592 - SyntaxError: closing parenthesis '}'

Issue - State: open - Opened by wangxiuwen 7 months ago - 3 comments

#591 - Silence NLTK download info

Pull Request - State: open - Opened by bobbywlindsey 7 months ago

#590 - Problems during run initial step

Issue - State: open - Opened by simplelifetime 7 months ago - 12 comments

#589 - Can't load exist dataset for evaluation

Issue - State: open - Opened by IsmaelMousa 7 months ago - 1 comment

#588 - Super tiny fix typo

Pull Request - State: open - Opened by fzyzcjy 7 months ago

#587 - set dev version

Pull Request - State: open - Opened by lhoestq 7 months ago

#585 - Metrics for multilabel problems don't match the expected format.

Issue - State: closed - Opened by adamamer20 7 months ago - 2 comments

#584 - Add raw pyarrow type check message for `EvaluationModule`

Pull Request - State: closed - Opened by shenxiangzhuang 7 months ago

#583 - fix: install cmd for extra pkg

Pull Request - State: closed - Opened by shenxiangzhuang 7 months ago

#581 - [FR] Confidence intervals for metrics

Issue - State: open - Opened by NightMachinery 7 months ago

#580 - Apply deprecated `evaluation_strategy`

Pull Request - State: open - Opened by muellerzr 7 months ago - 1 comment

#579 - Release: 0.4.2

Pull Request - State: closed - Opened by lhoestq 7 months ago - 3 comments

#578 - Fix FileFreeLock

Pull Request - State: closed - Opened by lhoestq 7 months ago

#577 - Fix wrong lib name in ImportError mesage

Pull Request - State: open - Opened by milistu 7 months ago

#576 - Unable to run pip install evaluate[template]

Issue - State: open - Opened by saicharan2804 7 months ago - 1 comment

#575 - Fix example doc in load function

Pull Request - State: closed - Opened by alexrs 7 months ago

#574 - Module 'glue' doesn't exist on the Hugging Face Hub either.

Issue - State: closed - Opened by enori 7 months ago - 1 comment

#572 - METEOR has no option to return unaggregated results

Issue - State: open - Opened by ashtonomy 8 months ago - 1 comment

#571 - Update python to 3.8

Pull Request - State: closed - Opened by qubvel 8 months ago - 5 comments

#569 - Speeding up mean_iou metric computation

Pull Request - State: closed - Opened by qubvel 8 months ago

#567 - Cannot use it offline!

Issue - State: open - Opened by SirryChen 8 months ago - 1 comment

#566 - Shouldn't perplexity range from [1 to inf)?

Issue - State: closed - Opened by ivanmkc 8 months ago - 2 comments

#565 - Can't use the BLEU offline.

Issue - State: open - Opened by Zhuxing01 8 months ago - 4 comments

#564 - Does Rouge score support the multilingual language?

Issue - State: closed - Opened by sanjeev-bhandari 8 months ago - 1 comment

#561 - It seems like evaluate.load doesnt use

Issue - State: open - Opened by anhq-nguyen 8 months ago

#560 - Is perplexity correctly computed?

Issue - State: open - Opened by halixness 8 months ago - 5 comments

#559 - evaluate consuming Memory and slow down the process

Issue - State: closed - Opened by Redix8 9 months ago

#558 - the difference of your bleu and sacrebleu

Issue - State: open - Opened by cooper12121 9 months ago - 1 comment

#557 - Add Diarization Error Rate (DER) metric

Pull Request - State: open - Opened by MedAhmedKrichen 9 months ago

#556 - Add Frechet Inception Distance (FID) Score

Pull Request - State: open - Opened by MedAhmedKrichen 9 months ago

#554 - Add Precision@k and Recall@k metrics

Issue - State: open - Opened by Andron00e 9 months ago

#553 - Evaluation not working with `microsoft/deberta-v3-large`

Issue - State: open - Opened by msinha251 9 months ago - 2 comments