Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / mlabonne/llm-autoeval issues and pull requests

#34 - GPT4All Average: Error: File does not exist%

Issue - State: open - Opened by mann1x 3 months ago - 1 comment

#33 - How to set a Git Repo?

Issue - State: open - Opened by qzc438 4 months ago

#32 - File does not exist% on all of every eval

Issue - State: closed - Opened by Pietro00x 4 months ago - 5 comments

#31 - token checks

Issue - State: open - Opened by CrispStrobe 6 months ago - 1 comment

#30 - Fix broken links in README.md

Pull Request - State: closed - Opened by simekent 7 months ago - 1 comment

#29 - Novel errors when running notebook

Issue - State: open - Opened by ann-brown 7 months ago

#28 - Add functionality for evaluating model safety/toxicity

Issue - State: open - Opened by m-newhauser 7 months ago - 1 comment

#26 - Eqbench

Pull Request - State: closed - Opened by mlabonne 8 months ago

#25 - Legalbench

Pull Request - State: closed - Opened by Malikeh97 8 months ago

#24 - Openllm fix

Pull Request - State: closed - Opened by mlabonne 8 months ago

#23 - Bug with lighteval on running from notebook

Issue - State: closed - Opened by ann-brown 8 months ago - 1 comment

#22 - Issues with BF16 models

Issue - State: closed - Opened by anakin87 8 months ago - 3 comments

#21 - Ch dev

Pull Request - State: open - Opened by chenhaodev 8 months ago

#20 - Proposal to Make LLM-AutoEval Repository Multi-Cloud Compatible

Issue - State: open - Opened by adithya-s-k 9 months ago - 1 comment

#19 - Use lighteval for benchmarking

Pull Request - State: closed - Opened by burtenshaw 9 months ago - 1 comment

#18 - Use Colab notebook with private models

Issue - State: closed - Opened by burtenshaw 9 months ago

#17 - assert isinstance(pretrained, str)

Issue - State: closed - Opened by majacinka 9 months ago - 2 comments

#16 - How to run it locally?

Issue - State: open - Opened by JamesKnight0001 9 months ago - 2 comments

#15 - Only one GPU is used during the autoeval

Issue - State: open - Opened by LMSPaul 10 months ago - 2 comments

#14 - Runpod.sh Refactor and new implementations.

Pull Request - State: open - Opened by Steel-skull 10 months ago - 4 comments

#13 - Revert "Update runpod.sh"

Pull Request - State: closed - Opened by mlabonne 10 months ago

#12 - Revert "Update runpod.sh"

Pull Request - State: closed - Opened by mlabonne 10 months ago

#11 - Update runpod.sh

Pull Request - State: closed - Opened by CultriX-Github 10 months ago

#10 - Update runpod.sh

Pull Request - State: closed - Opened by CultriX-Github 10 months ago

#9 - Update runpod.sh

Pull Request - State: closed - Opened by CultriX-Github 10 months ago

#8 - Update runpod.sh

Pull Request - State: closed - Opened by Steel-skull 10 months ago - 1 comment

#7 - Feature request - Local GPUs

Issue - State: open - Opened by maziyarpanahi 10 months ago - 1 comment

#6 - multiple GPUs works in colab

Issue - State: open - Opened by tdolan21 10 months ago - 1 comment

#5 - Docker setup for runpod

Issue - State: closed - Opened by johannhartmann 11 months ago - 2 comments

#4 - Why is `mmlu` benchmark commented out?

Issue - State: closed - Opened by sugatoray 11 months ago - 3 comments

#3 - Runpod says 'no longer any instance available'

Issue - State: closed - Opened by ncs-gobubble 11 months ago - 4 comments

#2 - Feature/time

Pull Request - State: closed - Opened by mlabonne 11 months ago

#1 - log running time?

Issue - State: closed - Opened by gblazex 11 months ago - 3 comments