Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / huggingface/evaluate issues and pull requests

#641 - Optionally stop data deletion after compute

Issue - State: open - Opened by sizhky 7 days ago

#638 - Evaluate fails to load all metrics.

Issue - State: closed - Opened by filbeofITK 14 days ago - 2 comments

#637 - Add ROUGE detailed option

Pull Request - State: open - Opened by eranhirs 14 days ago

#636 - Perplexity for Left Padded Models

Issue - State: open - Opened by Lawhy 16 days ago

#635 - Add required hf_token secret to build main documentation

Pull Request - State: closed - Opened by albertvillanova 19 days ago

#634 - Main documentation building is not triggered

Issue - State: closed - Opened by albertvillanova 19 days ago

#633 - Pass trust_remote_code to fix test_token_classification_parity

Pull Request - State: closed - Opened by albertvillanova 19 days ago - 1 comment

#631 - Pin numpy<2 as required by tensorflow to fix doc building

Pull Request - State: closed - Opened by albertvillanova 19 days ago - 1 comment

#629 - Support nltk>=3.9 to fix vulnerability

Pull Request - State: closed - Opened by albertvillanova 19 days ago

#628 - Support nltk>=3.9 to fix vulnerability

Issue - State: closed - Opened by albertvillanova 19 days ago

#627 - set dev version

Pull Request - State: closed - Opened by lhoestq 20 days ago

#626 - version 0.4.3

Pull Request - State: closed - Opened by lhoestq 20 days ago

#625 - fix run_ner test

Pull Request - State: closed - Opened by lhoestq 20 days ago - 1 comment

#624 - remove ignore_url_params

Pull Request - State: closed - Opened by lhoestq 20 days ago

#623 - Fix CI with temporary pin nltk<3.9

Pull Request - State: closed - Opened by albertvillanova 20 days ago

#621 - Replace deprecated use_auth_token with token

Pull Request - State: closed - Opened by albertvillanova 20 days ago

#619 - Correct the warning information of loader.py

Pull Request - State: open - Opened by HelloWorldLTY 20 days ago - 1 comment

#618 - add tip in docs and readme referring to lighteval

Pull Request - State: closed - Opened by MoritzLaurer 26 days ago

#616 - `list_evaluation_modules` returns empty list

Issue - State: open - Opened by MohamedAliRashad about 1 month ago - 2 comments

#615 - Benchmark evaluation for language models.

Issue - State: open - Opened by mina58 about 1 month ago - 1 comment

#614 - Fix: message for lack of sklearn

Pull Request - State: open - Opened by ArkiZh about 2 months ago

#613 - add vqa_accuracy and CIDEr

Pull Request - State: open - Opened by Kami-chanw about 2 months ago - 1 comment

#612 - Add note in repo and docs that library is no longer actively maintained

Issue - State: closed - Opened by MoritzLaurer about 2 months ago - 6 comments

#611 - How to customize my own evaluator and metrics?

Issue - State: closed - Opened by Kami-chanw about 2 months ago

#609 - AttributeError: module 'evaluate' has no attribute 'load'

Issue - State: open - Opened by Adesoji1 2 months ago - 7 comments

#608 - fix import error message for sklearn

Pull Request - State: open - Opened by bingwork 2 months ago

#606 - Add `zero_division` to F1 metric

Pull Request - State: open - Opened by TimRepke 3 months ago

#605 - Update MAUVE's readme and citations

Pull Request - State: open - Opened by krishnap25 3 months ago

#604 - Support tuple format in `combine` evaluations

Pull Request - State: open - Opened by shunk031 3 months ago

#602 - Gradio dependency issue

Issue - State: open - Opened by bnaman50 3 months ago - 2 comments

#600 - feat(ci): add trufflehog secrets detection

Pull Request - State: closed - Opened by McPatate 4 months ago

#599 - Addition of SummEval Metric to `evaluate` Library

Pull Request - State: open - Opened by penguinwang96825 4 months ago

#598 - New JSON encoder for results containing numpy types

Pull Request - State: open - Opened by jpodivin 4 months ago

#595 - Add tokenizer initialization to the documented example

Pull Request - State: open - Opened by jpodivin 4 months ago

#592 - SyntaxError: closing parenthesis '}'

Issue - State: open - Opened by wangxiuwen 5 months ago - 3 comments

#591 - Silence NLTK download info

Pull Request - State: open - Opened by bobbywlindsey 5 months ago

#590 - Problems during run initial step

Issue - State: open - Opened by simplelifetime 5 months ago - 12 comments

#589 - Can't load exist dataset for evaluation

Issue - State: open - Opened by IsmaelMousa 5 months ago - 1 comment

#588 - Super tiny fix typo

Pull Request - State: open - Opened by fzyzcjy 5 months ago

#587 - set dev version

Pull Request - State: open - Opened by lhoestq 5 months ago

#585 - Metrics for multilabel problems don't match the expected format.

Issue - State: closed - Opened by adamamer20 5 months ago - 2 comments

#584 - Add raw pyarrow type check message for `EvaluationModule`

Pull Request - State: closed - Opened by shenxiangzhuang 5 months ago

#583 - fix: install cmd for extra pkg

Pull Request - State: closed - Opened by shenxiangzhuang 5 months ago

#581 - [FR] Confidence intervals for metrics

Issue - State: open - Opened by NightMachinery 5 months ago

#580 - Apply deprecated `evaluation_strategy`

Pull Request - State: open - Opened by muellerzr 6 months ago - 1 comment

#579 - Release: 0.4.2

Pull Request - State: closed - Opened by lhoestq 6 months ago - 3 comments

#578 - Fix FileFreeLock

Pull Request - State: closed - Opened by lhoestq 6 months ago

#577 - Fix wrong lib name in ImportError mesage

Pull Request - State: open - Opened by milistu 6 months ago

#576 - Unable to run pip install evaluate[template]

Issue - State: open - Opened by saicharan2804 6 months ago - 1 comment

#575 - Fix example doc in load function

Pull Request - State: closed - Opened by alexrs 6 months ago

#574 - Module 'glue' doesn't exist on the Hugging Face Hub either.

Issue - State: closed - Opened by enori 6 months ago - 1 comment

#572 - METEOR has no option to return unaggregated results

Issue - State: open - Opened by ashtonomy 6 months ago - 1 comment

#571 - Update python to 3.8

Pull Request - State: closed - Opened by qubvel 6 months ago - 5 comments

#569 - Speeding up mean_iou metric computation

Pull Request - State: closed - Opened by qubvel 6 months ago

#567 - Cannot use it offline!

Issue - State: open - Opened by SirryChen 6 months ago - 1 comment

#566 - Shouldn't perplexity range from [1 to inf)?

Issue - State: closed - Opened by ivanmkc 6 months ago - 2 comments

#565 - Can't use the BLEU offline.

Issue - State: open - Opened by Zhuxing01 6 months ago - 3 comments

#564 - Does Rouge score support the multilingual language?

Issue - State: closed - Opened by sanjeev-bhandari 6 months ago - 1 comment

#561 - It seems like evaluate.load doesnt use

Issue - State: open - Opened by anhq-nguyen 7 months ago

#560 - Is perplexity correctly computed?

Issue - State: open - Opened by halixness 7 months ago - 4 comments

#559 - evaluate consuming Memory and slow down the process

Issue - State: closed - Opened by Redix8 7 months ago

#558 - the difference of your bleu and sacrebleu

Issue - State: open - Opened by cooper12121 7 months ago - 1 comment

#557 - Add Diarization Error Rate (DER) metric

Pull Request - State: open - Opened by MedAhmedKrichen 7 months ago

#556 - Add Frechet Inception Distance (FID) Score

Pull Request - State: open - Opened by MedAhmedKrichen 7 months ago

#554 - Add Precision@k and Recall@k metrics

Issue - State: open - Opened by Andron00e 7 months ago

#553 - Evaluation not working with `microsoft/deberta-v3-large`

Issue - State: open - Opened by msinha251 7 months ago - 2 comments

#551 - Add geometric mean of per-token-Perplexities

Issue - State: open - Opened by LuggiStruggi 7 months ago

#550 - MSE not accepting (n_samples, n_ouptuts) despite docs stating so

Issue - State: open - Opened by bauersimon 8 months ago - 2 comments

#549 - Add draft progress bar for 3 tasks

Pull Request - State: open - Opened by boyleconnor 8 months ago

#544 - Style fixes for main

Pull Request - State: open - Opened by ashahba 8 months ago

#543 - missing dependency

Pull Request - State: open - Opened by stas00 8 months ago

#542 - several breakages due to recent `datasets`

Issue - State: closed - Opened by stas00 8 months ago - 16 comments

#540 - Evaluate fails to acquire lock interrupting single-node multi-GPU training

Issue - State: closed - Opened by iNeil77 8 months ago - 2 comments