Ecosyste.ms: Issues
An open API service for providing issue and pull request metadata for open source projects.
GitHub / huggingface/evaluate issues and pull requests
#642 - ImportError: To be able to use evaluate-metric/glue, you need to install the following dependencies['scipy', 'scikit-learn'] using 'pip install scipy sklearn' for instance'
Issue -
State: open - Opened by JINO-ROHIT 7 days ago
- 3 comments
#641 - Optionally stop data deletion after compute
Issue -
State: open - Opened by sizhky 7 days ago
#640 - Typo fixed in `Types of Evaluations in 🤗 Evaluate` documentation
Pull Request -
State: open - Opened by sergiopaniego 13 days ago
#639 - Typo in `Types of Evaluations in 🤗 Evaluate` documentation
Issue -
State: open - Opened by sergiopaniego 13 days ago
#638 - Evaluate fails to load all metrics.
Issue -
State: closed - Opened by filbeofITK 14 days ago
- 2 comments
#637 - Add ROUGE detailed option
Pull Request -
State: open - Opened by eranhirs 14 days ago
#636 - Perplexity for Left Padded Models
Issue -
State: open - Opened by Lawhy 16 days ago
#635 - Add required hf_token secret to build main documentation
Pull Request -
State: closed - Opened by albertvillanova 19 days ago
#634 - Main documentation building is not triggered
Issue -
State: closed - Opened by albertvillanova 19 days ago
#633 - Pass trust_remote_code to fix test_token_classification_parity
Pull Request -
State: closed - Opened by albertvillanova 19 days ago
- 1 comment
#632 - CI parity test is broken due to FileNotFoundError
Issue -
State: open - Opened by albertvillanova 19 days ago
#631 - Pin numpy<2 as required by tensorflow to fix doc building
Pull Request -
State: closed - Opened by albertvillanova 19 days ago
- 1 comment
#630 - Documentation building is broken due to numpy-2 and tensorflow
Issue -
State: closed - Opened by albertvillanova 19 days ago
#629 - Support nltk>=3.9 to fix vulnerability
Pull Request -
State: closed - Opened by albertvillanova 19 days ago
#628 - Support nltk>=3.9 to fix vulnerability
Issue -
State: closed - Opened by albertvillanova 19 days ago
#627 - set dev version
Pull Request -
State: closed - Opened by lhoestq 20 days ago
#626 - version 0.4.3
Pull Request -
State: closed - Opened by lhoestq 20 days ago
#625 - fix run_ner test
Pull Request -
State: closed - Opened by lhoestq 20 days ago
- 1 comment
#624 - remove ignore_url_params
Pull Request -
State: closed - Opened by lhoestq 20 days ago
#623 - Fix CI with temporary pin nltk<3.9
Pull Request -
State: closed - Opened by albertvillanova 20 days ago
#622 - CI is broken due to nltk: Resource punkt_tab not found
Issue -
State: closed - Opened by albertvillanova 20 days ago
#621 - Replace deprecated use_auth_token with token
Pull Request -
State: closed - Opened by albertvillanova 20 days ago
#620 - Evaluate uses deprecated use_auth_token and will break with datasets-3.0
Issue -
State: closed - Opened by albertvillanova 20 days ago
#619 - Correct the warning information of loader.py
Pull Request -
State: open - Opened by HelloWorldLTY 20 days ago
- 1 comment
#618 - add tip in docs and readme referring to lighteval
Pull Request -
State: closed - Opened by MoritzLaurer 26 days ago
#616 - `list_evaluation_modules` returns empty list
Issue -
State: open - Opened by MohamedAliRashad about 1 month ago
- 2 comments
#615 - Benchmark evaluation for language models.
Issue -
State: open - Opened by mina58 about 1 month ago
- 1 comment
#614 - Fix: message for lack of sklearn
Pull Request -
State: open - Opened by ArkiZh about 2 months ago
#613 - add vqa_accuracy and CIDEr
Pull Request -
State: open - Opened by Kami-chanw about 2 months ago
- 1 comment
#612 - Add note in repo and docs that library is no longer actively maintained
Issue -
State: closed - Opened by MoritzLaurer about 2 months ago
- 6 comments
#611 - How to customize my own evaluator and metrics?
Issue -
State: closed - Opened by Kami-chanw about 2 months ago
#610 - Unable to compute f1 score - Throwing Value Error trying to convert a string in non english Language to integer
Issue -
State: open - Opened by alans3321 2 months ago
#609 - AttributeError: module 'evaluate' has no attribute 'load'
Issue -
State: open - Opened by Adesoji1 2 months ago
- 7 comments
#608 - fix import error message for sklearn
Pull Request -
State: open - Opened by bingwork 2 months ago
#607 - [Metrics] ValueError: Expected to find locked file from process x but it doesn't exist.
Issue -
State: open - Opened by raghavm1 3 months ago
#606 - Add `zero_division` to F1 metric
Pull Request -
State: open - Opened by TimRepke 3 months ago
#605 - Update MAUVE's readme and citations
Pull Request -
State: open - Opened by krishnap25 3 months ago
#604 - Support tuple format in `combine` evaluations
Pull Request -
State: open - Opened by shunk031 3 months ago
#603 - AttributeError: 'CombinedEvaluations' object has no attribute 'evaluation_modules'
Issue -
State: open - Opened by shunk031 3 months ago
- 2 comments
#602 - Gradio dependency issue
Issue -
State: open - Opened by bnaman50 3 months ago
- 2 comments
#601 - Evaluation of form feed symbol with BLEU results in error
Issue -
State: open - Opened by lowlypalace 4 months ago
#600 - feat(ci): add trufflehog secrets detection
Pull Request -
State: closed - Opened by McPatate 4 months ago
#599 - Addition of SummEval Metric to `evaluate` Library
Pull Request -
State: open - Opened by penguinwang96825 4 months ago
#598 - New JSON encoder for results containing numpy types
Pull Request -
State: open - Opened by jpodivin 4 months ago
#597 - LocalModuleTest.test_load_metric_code_eval fails with "The "code_eval" metric executes untrusted model-generated code in Python."
Issue -
State: open - Opened by jpodivin 4 months ago
#596 - Enhancing `TextClassificationEvaluator` to Support Averaged Metrics
Pull Request -
State: open - Opened by ilyesdjerfaf 4 months ago
#595 - Add tokenizer initialization to the documented example
Pull Request -
State: open - Opened by jpodivin 4 months ago
#594 - Execution of example from the Using the evaluator docs fails due to unspecified tokenizer
Issue -
State: open - Opened by jpodivin 4 months ago
#593 - Evaluation of empty strings with MAUVE results in error
Issue -
State: open - Opened by lowlypalace 4 months ago
#592 - SyntaxError: closing parenthesis '}'
Issue -
State: open - Opened by wangxiuwen 5 months ago
- 3 comments
#591 - Silence NLTK download info
Pull Request -
State: open - Opened by bobbywlindsey 5 months ago
#590 - Problems during run initial step
Issue -
State: open - Opened by simplelifetime 5 months ago
- 12 comments
#589 - Can't load exist dataset for evaluation
Issue -
State: open - Opened by IsmaelMousa 5 months ago
- 1 comment
#588 - Super tiny fix typo
Pull Request -
State: open - Opened by fzyzcjy 5 months ago
#587 - set dev version
Pull Request -
State: open - Opened by lhoestq 5 months ago
#586 - [Question]Shall we adding a faster BLEU score calculator?
Issue -
State: open - Opened by shenxiangzhuang 5 months ago
#585 - Metrics for multilabel problems don't match the expected format.
Issue -
State: closed - Opened by adamamer20 5 months ago
- 2 comments
#584 - Add raw pyarrow type check message for `EvaluationModule`
Pull Request -
State: closed - Opened by shenxiangzhuang 5 months ago
#583 - fix: install cmd for extra pkg
Pull Request -
State: closed - Opened by shenxiangzhuang 5 months ago
#582 - How to pass generation_kwargs to the TextGeneration evaluator ?
Issue -
State: open - Opened by swarnava112 5 months ago
#581 - [FR] Confidence intervals for metrics
Issue -
State: open - Opened by NightMachinery 5 months ago
#580 - Apply deprecated `evaluation_strategy`
Pull Request -
State: open - Opened by muellerzr 6 months ago
- 1 comment
#579 - Release: 0.4.2
Pull Request -
State: closed - Opened by lhoestq 6 months ago
- 3 comments
#578 - Fix FileFreeLock
Pull Request -
State: closed - Opened by lhoestq 6 months ago
#577 - Fix wrong lib name in ImportError mesage
Pull Request -
State: open - Opened by milistu 6 months ago
#576 - Unable to run pip install evaluate[template]
Issue -
State: open - Opened by saicharan2804 6 months ago
- 1 comment
#575 - Fix example doc in load function
Pull Request -
State: closed - Opened by alexrs 6 months ago
#574 - Module 'glue' doesn't exist on the Hugging Face Hub either.
Issue -
State: closed - Opened by enori 6 months ago
- 1 comment
#573 - Perplexity metric does not apply batching correctly to tokenization
Issue -
State: open - Opened by ChengSashankh 6 months ago
- 1 comment
#572 - METEOR has no option to return unaggregated results
Issue -
State: open - Opened by ashtonomy 6 months ago
- 1 comment
#571 - Update python to 3.8
Pull Request -
State: closed - Opened by qubvel 6 months ago
- 5 comments
#570 - [Question] How to have no preset values sent into `.compute()`
Issue -
State: open - Opened by alvations 6 months ago
#569 - Speeding up mean_iou metric computation
Pull Request -
State: closed - Opened by qubvel 6 months ago
#568 - Allow for specify coda device in perplexity evaluation
Issue -
State: open - Opened by manuelbrack 6 months ago
#567 - Cannot use it offline!
Issue -
State: open - Opened by SirryChen 6 months ago
- 1 comment
#566 - Shouldn't perplexity range from [1 to inf)?
Issue -
State: closed - Opened by ivanmkc 6 months ago
- 2 comments
#565 - Can't use the BLEU offline.
Issue -
State: open - Opened by Zhuxing01 6 months ago
- 3 comments
#564 - Does Rouge score support the multilingual language?
Issue -
State: closed - Opened by sanjeev-bhandari 6 months ago
- 1 comment
#563 - ValueError: Predictions and/or references don't match the expected format.
Issue -
State: open - Opened by antopost 6 months ago
- 1 comment
#562 - ImportError: To be able to use evaluate-metric/rouge, you need to install the following dependencies['nltk'] using 'pip install # Here to have a nice missing dependency error message early on' for instance'
Issue -
State: closed - Opened by BAEK26 7 months ago
- 2 comments
#561 - It seems like evaluate.load doesnt use
Issue -
State: open - Opened by anhq-nguyen 7 months ago
#560 - Is perplexity correctly computed?
Issue -
State: open - Opened by halixness 7 months ago
- 4 comments
#559 - evaluate consuming Memory and slow down the process
Issue -
State: closed - Opened by Redix8 7 months ago
#558 - the difference of your bleu and sacrebleu
Issue -
State: open - Opened by cooper12121 7 months ago
- 1 comment
#557 - Add Diarization Error Rate (DER) metric
Pull Request -
State: open - Opened by MedAhmedKrichen 7 months ago
#556 - Add Frechet Inception Distance (FID) Score
Pull Request -
State: open - Opened by MedAhmedKrichen 7 months ago
#555 - After fine-tuning Gemma and want to evaluate performance: AttributeError: module 'keras._tf_keras.keras' has no attribute '__internal__'
Issue -
State: open - Opened by XinyueZ 7 months ago
#554 - Add Precision@k and Recall@k metrics
Issue -
State: open - Opened by Andron00e 7 months ago
#553 - Evaluation not working with `microsoft/deberta-v3-large`
Issue -
State: open - Opened by msinha251 7 months ago
- 2 comments
#552 - need evaluator code reference for object detector & zero shot object dector task
Issue -
State: open - Opened by solomonmanuelraj 7 months ago
#551 - Add geometric mean of per-token-Perplexities
Issue -
State: open - Opened by LuggiStruggi 7 months ago
#550 - MSE not accepting (n_samples, n_ouptuts) despite docs stating so
Issue -
State: open - Opened by bauersimon 8 months ago
- 2 comments
#549 - Add draft progress bar for 3 tasks
Pull Request -
State: open - Opened by boyleconnor 8 months ago
#548 - Try to import evaluate but get "Library not loaded: @rpath/libwebp.7.dylib" error
Issue -
State: closed - Opened by czxttkl 8 months ago
- 2 comments
#547 - `DatasetColumn` and `DatasetColumnPair` should have `__repr__()` methods defined
Issue -
State: open - Opened by boyleconnor 8 months ago
#544 - Style fixes for main
Pull Request -
State: open - Opened by ashahba 8 months ago
#543 - missing dependency
Pull Request -
State: open - Opened by stas00 8 months ago
#542 - several breakages due to recent `datasets`
Issue -
State: closed - Opened by stas00 8 months ago
- 16 comments
#541 - Is there a plan to translate the documentation for this repository?
Issue -
State: open - Opened by 2404589803 8 months ago
#540 - Evaluate fails to acquire lock interrupting single-node multi-GPU training
Issue -
State: closed - Opened by iNeil77 8 months ago
- 2 comments