Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / huggingface/evaluate issues and pull requests

#539 - [metric] macro and weighted average metrics for seqeval

Pull Request - State: open - Opened by imvladikon 8 months ago

#538 - Remove an unused dependency

Pull Request - State: closed - Opened by yasufumy 8 months ago

#537 - Is TextGenerationEvaluator incomplete?

Issue - State: open - Opened by inf3rnus 9 months ago - 5 comments

#534 - How should i cite this repository?

Issue - State: open - Opened by perlitz 9 months ago - 2 comments

#533 - support multilabel confusion matrix

Pull Request - State: open - Opened by 0ssamaak0 9 months ago - 7 comments

#532 - `perplexity` yields `IndexError: index out of range in self`

Issue - State: open - Opened by pminervini 9 months ago - 1 comment

#531 - Update requirements.txt

Pull Request - State: open - Opened by pminervini 9 months ago

#530 - Fix typos ; Update gradio.py

Pull Request - State: open - Opened by furkanakkurt1335 9 months ago

#528 - Add confusion matrix

Pull Request - State: closed - Opened by osanseviero 10 months ago - 5 comments

#524 - Added index.html

Pull Request - State: closed - Opened by tybrs 10 months ago

#521 - Added the ability to return the dictionary to the WER metric.

Pull Request - State: open - Opened by blademoon 10 months ago

#520 - Add custom device selection such as 'cuda:1' for perplexity

Pull Request - State: open - Opened by thatmee 10 months ago

#519 - `evaluate-cli create` fails with `subprocess.CalledProcessError`

Issue - State: open - Opened by Tai-Mai 10 months ago - 1 comment

#518 - Importing Error for evaluator module

Issue - State: open - Opened by DreF174 10 months ago

#517 - Cannot list evaluation modules types

Issue - State: open - Opened by trajepl 10 months ago

#516 - Can't calculate combined metric (WER + CER)

Issue - State: open - Opened by blademoon 11 months ago - 2 comments

#515 - Allow BLEURT optimisation parameters

Issue - State: open - Opened by Limess 11 months ago

#512 - about evaluate/perplexity

Issue - State: open - Opened by KangkangStu 11 months ago - 2 comments

#511 - Loading metrics that are shipped with the evaluate package is slow

Issue - State: open - Opened by harmbuisman 11 months ago - 2 comments

#510 - fix: remove useless token

Pull Request - State: closed - Opened by rtrompier 12 months ago

#509 - [metric] CRPS metric

Pull Request - State: open - Opened by kashif 12 months ago

#508 - docs: specify micro average in seqeval README

Pull Request - State: open - Opened by WojciechKusa 12 months ago

#507 - Remove unused dependency

Pull Request - State: closed - Opened by daskol 12 months ago - 2 comments

#506 - set dev version

Pull Request - State: closed - Opened by lhoestq 12 months ago

#505 - Release: 0.4.1

Pull Request - State: closed - Opened by lhoestq 12 months ago

#504 - Compatibility Issue with datasets library

Issue - State: open - Opened by adldotori 12 months ago - 3 comments

#503 - Move away from `Repository` to `HfApi` upload files

Issue - State: open - Opened by Wauplin 12 months ago

#502 - Perplexity Space Broken

Issue - State: closed - Opened by alyserecord 12 months ago

#500 - Wrong accuracy on the validation set!!!

Issue - State: open - Opened by utility-aagrawal 12 months ago - 1 comment

#499 - removed unnecessary `module_type` attribute

Pull Request - State: open - Opened by leo-gan 12 months ago

#498 - fix scikit-learn package name suggestion

Pull Request - State: closed - Opened by bzz about 1 year ago - 1 comment

#497 - Multi-label with f1

Issue - State: closed - Opened by jingmingzhuo about 1 year ago - 2 comments

#495 - CER.py error in class CERConfig

Issue - State: open - Opened by danhertztech about 1 year ago - 1 comment

#494 - is it possible to evaluate a proprietary model without uploading it to HF

Issue - State: open - Opened by sermolin about 1 year ago - 7 comments

#493 - Feature request for metric result keys in `MetricInfo`

Issue - State: open - Opened by tybrs about 1 year ago

#492 - Matthew's Correlation Coefficient (MCC) always zero

Issue - State: closed - Opened by mirix about 1 year ago - 1 comment

#490 - Fix `datasets` import in Meteor metric

Pull Request - State: closed - Opened by mariosasko about 1 year ago

#488 - Add support for glue metric in `EvaluationSuite`

Pull Request - State: open - Opened by tybrs about 1 year ago - 1 comment

#487 - Evaluate a quantize model

Issue - State: open - Opened by Psancs05 about 1 year ago - 2 comments

#486 - evaluate.load("wer") is returning Filenotfound error

Issue - State: open - Opened by valmetisrinivas about 1 year ago - 3 comments

#484 - Metric returned instead of the actual COMET scores

Issue - State: open - Opened by LeonardoEmili about 1 year ago - 6 comments

#483 - `.compute()` not compatible with `perplexity` for `text-generation`

Issue - State: open - Opened by hans-ekbrand about 1 year ago - 1 comment

#482 - Adding tokenizer_id to perplexity evaluation

Pull Request - State: open - Opened by Albertoimpl about 1 year ago - 3 comments

#480 - meteor don't works with latest datasets

Issue - State: closed - Opened by leoleoasd about 1 year ago - 13 comments

#479 - Fix formatting of model names in radar plot

Pull Request - State: open - Opened by potamides about 1 year ago

#478 - Trainer Custom_loss - Empty metrics in Evaluate

Issue - State: closed - Opened by AkimParis about 1 year ago - 2 comments

#474 - Fix SQuAD v2 metric docs on `references` format

Pull Request - State: open - Opened by bryant1410 about 1 year ago

#473 - evaluate.load loads indefinitely offline

Issue - State: open - Opened by Virgile-Richard1 about 1 year ago - 1 comment

#472 - [BUG] Loading metrics is extremely slow

Issue - State: open - Opened by NightMachinery about 1 year ago - 6 comments

#471 - Fix mean computation in Mahalanobis distance

Pull Request - State: open - Opened by j0ma about 1 year ago

#470 - Mahalanobis distance computes X_minus_mu incorrectly

Issue - State: open - Opened by j0ma about 1 year ago

#468 - [doc build] Use secrets

Pull Request - State: open - Opened by mishig25 over 1 year ago

#467 - Trigger CI on ci-* branches

Pull Request - State: closed - Opened by Wauplin over 1 year ago - 1 comment

#466 - [don't merge] Test hfh v0.15.0.rc0

Pull Request - State: closed - Opened by Wauplin over 1 year ago - 2 comments

#464 - does rouge_score = evaluate.load("rouge") support other languages?

Issue - State: open - Opened by lixnvege over 1 year ago - 1 comment

#463 - Render Text2TextGenerationEvaluators' docstring examples

Pull Request - State: closed - Opened by mariosasko over 1 year ago - 1 comment

#461 - How to disable NLTK download information output?

Issue - State: open - Opened by yiyualt over 1 year ago - 1 comment

#460 - Evaluate SyntaxError: invalid syntax

Issue - State: open - Opened by NLBONU1 over 1 year ago - 1 comment

#459 - feat: custom evaluator and metric name to support llm evaluation #433

Pull Request - State: open - Opened by fecet over 1 year ago

#458 - Removed unused and restrictive `responses` requirement

Pull Request - State: closed - Opened by twerkmeister over 1 year ago - 2 comments

#457 - Quick tour typos

Issue - State: open - Opened by trajepl over 1 year ago

#456 - Module 'accuracy' doesn't exist on the Hugging Face Hub either.

Issue - State: open - Opened by AceCHQ over 1 year ago - 19 comments

#455 - AttributeError: module 'sacrebleu' has no attribute '__version__'

Issue - State: closed - Opened by dsj96 over 1 year ago - 4 comments

#454 - Toxicity Measurement Should Allow GPU Usage

Issue - State: open - Opened by dni138 over 1 year ago

#453 - SQuAD v2 is missing the NoAns metrics

Issue - State: open - Opened by GitMew over 1 year ago

#452 - unable to import evaluate

Issue - State: closed - Opened by Ananthzeke over 1 year ago - 1 comment

#451 - Huggingface bertscore metric GPU leak?

Issue - State: open - Opened by will-maclean over 1 year ago

#450 - Adding new metrics locally without using the evaluate-cli command

Issue - State: closed - Opened by sameeravithana over 1 year ago - 1 comment

#449 - Metrics that use DownloadConfig not working any more

Issue - State: closed - Opened by BramVanroy over 1 year ago - 5 comments

#447 - replace evaluate DownloadConfig with datasets

Pull Request - State: closed - Opened by lvwerra over 1 year ago - 2 comments

#446 - Add kendall tau distance (not as a submodule)

Pull Request - State: open - Opened by unna97 over 1 year ago - 3 comments

#445 - Understanding metric compute in DDP setup

Issue - State: closed - Opened by Natooz over 1 year ago - 1 comment

#444 - combined metrics in evaluators/Suite subtasks

Issue - State: open - Opened by Vipitis over 1 year ago

#443 - Update comet

Pull Request - State: closed - Opened by ricardorei over 1 year ago - 6 comments

#442 - Progress bar when using evaluator

Issue - State: open - Opened by zxcej over 1 year ago - 1 comment

#441 - Problem about code_eval

Issue - State: closed - Opened by rongaoli over 1 year ago