Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / EvolvingLMMs-Lab/lmms-eval issues and pull requests

#75 - An issue during evaluating MathVista

Issue - State: open - Opened by simplelifetime 9 months ago - 1 comment

#75 - An issue during evaluating MathVista

Issue - State: open - Opened by simplelifetime 9 months ago - 1 comment

#75 - An issue during evaluating MathVista

Issue - State: open - Opened by simplelifetime 9 months ago - 1 comment

#75 - An issue during evaluating MathVista

Issue - State: open - Opened by simplelifetime 9 months ago - 1 comment

#74 - Support for llava-hf/llava-v1.6-x

Issue - State: open - Opened by justinphan3110cais 10 months ago - 3 comments

#74 - Support for llava-hf/llava-v1.6-x

Issue - State: open - Opened by justinphan3110cais 10 months ago - 3 comments

#74 - Support for llava-hf/llava-v1.6-x

Issue - State: open - Opened by justinphan3110cais 10 months ago - 3 comments

#74 - Support for llava-hf/llava-v1.6-x

Issue - State: open - Opened by justinphan3110cais 10 months ago - 3 comments

#73 - [Feat] Add qwen vl api

Pull Request - State: closed - Opened by kcz358 10 months ago - 1 comment

#73 - [Feat] Add qwen vl api

Pull Request - State: closed - Opened by kcz358 10 months ago - 1 comment

#73 - [Feat] Add qwen vl api

Pull Request - State: closed - Opened by kcz358 10 months ago - 1 comment

#73 - [Feat] Add qwen vl api

Pull Request - State: closed - Opened by kcz358 10 months ago - 1 comment

#72 - Multinode Evaluation

Issue - State: open - Opened by yqy2001 10 months ago - 2 comments

#72 - Multinode Evaluation

Issue - State: open - Opened by yqy2001 10 months ago - 2 comments

#72 - Multinode Evaluation

Issue - State: open - Opened by yqy2001 10 months ago - 2 comments

#72 - Multinode Evaluation

Issue - State: open - Opened by yqy2001 10 months ago - 2 comments

#71 - While this lmms-eval support V*Bench?

Issue - State: open - Opened by xmu-xiaoma666 10 months ago

#71 - While this lmms-eval support V*Bench?

Issue - State: open - Opened by xmu-xiaoma666 10 months ago

#71 - While this lmms-eval support V*Bench?

Issue - State: open - Opened by xmu-xiaoma666 10 months ago

#71 - While this lmms-eval support V*Bench?

Issue - State: open - Opened by xmu-xiaoma666 10 months ago

#70 - Bugfix: WebSRC should be token-level F1 NOT character-level

Pull Request - State: closed - Opened by hunterheiden 10 months ago

#70 - Bugfix: WebSRC should be token-level F1 NOT character-level

Pull Request - State: closed - Opened by hunterheiden 10 months ago

#70 - Bugfix: WebSRC should be token-level F1 NOT character-level

Pull Request - State: closed - Opened by hunterheiden 10 months ago

#70 - Bugfix: WebSRC should be token-level F1 NOT character-level

Pull Request - State: closed - Opened by hunterheiden 10 months ago

#69 - [New Task] WebSRC (multimodal Q&A on web screenshots)

Pull Request - State: closed - Opened by hunterheiden 10 months ago - 1 comment

#69 - [New Task] WebSRC (multimodal Q&A on web screenshots)

Pull Request - State: closed - Opened by hunterheiden 10 months ago - 1 comment

#69 - [New Task] WebSRC (multimodal Q&A on web screenshots)

Pull Request - State: closed - Opened by hunterheiden 10 months ago - 1 comment

#69 - [New Task] WebSRC (multimodal Q&A on web screenshots)

Pull Request - State: closed - Opened by hunterheiden 10 months ago - 1 comment

#68 - confusion about llava-next for multi-docvqa

Issue - State: open - Opened by yayafengzi 10 months ago - 1 comment

#68 - confusion about llava-next for multi-docvqa

Issue - State: open - Opened by yayafengzi 10 months ago - 1 comment

#68 - confusion about llava-next for multi-docvqa

Issue - State: open - Opened by yayafengzi 10 months ago - 1 comment

#68 - confusion about llava-next for multi-docvqa

Issue - State: open - Opened by yayafengzi 10 months ago - 1 comment

#67 - Model will be loaded on different devices when using multiple gpus.

Issue - State: closed - Opened by baichuanzhou 10 months ago - 6 comments

#67 - Model will be loaded on different devices when using multiple gpus.

Issue - State: closed - Opened by baichuanzhou 10 months ago - 6 comments

#67 - Model will be loaded on different devices when using multiple gpus.

Issue - State: closed - Opened by baichuanzhou 10 months ago - 6 comments

#67 - Model will be loaded on different devices when using multiple gpus.

Issue - State: closed - Opened by baichuanzhou 10 months ago - 6 comments

#66 - How to add keywords stopping criterion during evaluation?

Issue - State: closed - Opened by baichuanzhou 10 months ago - 1 comment

#66 - How to add keywords stopping criterion during evaluation?

Issue - State: closed - Opened by baichuanzhou 10 months ago - 1 comment

#66 - How to add keywords stopping criterion during evaluation?

Issue - State: closed - Opened by baichuanzhou 10 months ago - 1 comment

#66 - How to add keywords stopping criterion during evaluation?

Issue - State: closed - Opened by baichuanzhou 10 months ago - 1 comment

#65 - incomplete tasks problem

Issue - State: open - Opened by adrianwestmoon 10 months ago - 1 comment

#65 - incomplete tasks problem

Issue - State: open - Opened by adrianwestmoon 10 months ago - 1 comment

#65 - incomplete tasks problem

Issue - State: open - Opened by adrianwestmoon 10 months ago - 1 comment

#65 - incomplete tasks problem

Issue - State: open - Opened by adrianwestmoon 10 months ago - 1 comment

#64 - Unreasonably high scores in llava_in_the_wild benchmark

Issue - State: open - Opened by wusize 10 months ago - 3 comments

#64 - Unreasonably high scores in llava_in_the_wild benchmark

Issue - State: open - Opened by wusize 10 months ago - 3 comments

#64 - Unreasonably high scores in llava_in_the_wild benchmark

Issue - State: open - Opened by wusize 10 months ago - 3 comments

#64 - Unreasonably high scores in llava_in_the_wild benchmark

Issue - State: open - Opened by wusize 10 months ago - 3 comments

#63 - New Task: ScreenSpot - Grounding (REC) and instruction generation (REG) on screens

Pull Request - State: closed - Opened by hunterheiden 10 months ago - 2 comments

#63 - New Task: ScreenSpot - Grounding (REC) and instruction generation (REG) on screens

Pull Request - State: closed - Opened by hunterheiden 10 months ago - 2 comments

#63 - New Task: ScreenSpot - Grounding (REC) and instruction generation (REG) on screens

Pull Request - State: closed - Opened by hunterheiden 10 months ago - 2 comments

#63 - New Task: ScreenSpot - Grounding (REC) and instruction generation (REG) on screens

Pull Request - State: closed - Opened by hunterheiden 10 months ago - 2 comments

#62 - token: False does not work in gqa.yaml

Issue - State: closed - Opened by baiyuting 10 months ago - 4 comments

#62 - token: False does not work in gqa.yaml

Issue - State: closed - Opened by baiyuting 10 months ago - 4 comments

#62 - token: False does not work in gqa.yaml

Issue - State: closed - Opened by baiyuting 10 months ago - 4 comments

#62 - token: False does not work in gqa.yaml

Issue - State: closed - Opened by baiyuting 10 months ago - 4 comments

#61 - Fix typo in Qwen-VL that was causing "reference before assignment"

Pull Request - State: closed - Opened by tupini07 10 months ago - 1 comment

#61 - Fix typo in Qwen-VL that was causing "reference before assignment"

Pull Request - State: closed - Opened by tupini07 10 months ago - 1 comment

#61 - Fix typo in Qwen-VL that was causing "reference before assignment"

Pull Request - State: closed - Opened by tupini07 10 months ago - 1 comment

#61 - Fix typo in Qwen-VL that was causing "reference before assignment"

Pull Request - State: closed - Opened by tupini07 10 months ago - 1 comment

#60 - Add MathVerse

Pull Request - State: closed - Opened by CaraJ7 10 months ago - 2 comments

#60 - Add MathVerse

Pull Request - State: closed - Opened by CaraJ7 10 months ago - 2 comments

#60 - Add MathVerse

Pull Request - State: closed - Opened by CaraJ7 10 months ago - 2 comments

#60 - Add MathVerse

Pull Request - State: closed - Opened by CaraJ7 10 months ago - 2 comments

#59 - add idefics2

Pull Request - State: closed - Opened by jzhang38 10 months ago - 1 comment

#59 - add idefics2

Pull Request - State: closed - Opened by jzhang38 10 months ago - 1 comment

#59 - add idefics2

Pull Request - State: closed - Opened by jzhang38 10 months ago - 1 comment

#59 - add idefics2

Pull Request - State: closed - Opened by jzhang38 10 months ago - 1 comment

#58 - How to add the new dataset?

Issue - State: closed - Opened by rese1f 10 months ago - 2 comments

#58 - How to add the new dataset?

Issue - State: closed - Opened by rese1f 10 months ago - 2 comments

#58 - How to add the new dataset?

Issue - State: closed - Opened by rese1f 10 months ago - 2 comments

#58 - How to add the new dataset?

Issue - State: closed - Opened by rese1f 10 months ago - 2 comments

#57 - [Benchmarks] RealWorldQA

Pull Request - State: closed - Opened by pufanyi 10 months ago

#57 - [Benchmarks] RealWorldQA

Pull Request - State: closed - Opened by pufanyi 10 months ago

#57 - [Benchmarks] RealWorldQA

Pull Request - State: closed - Opened by pufanyi 10 months ago

#57 - [Benchmarks] RealWorldQA

Pull Request - State: closed - Opened by pufanyi 10 months ago

#56 - Multilingual LLava bench

Pull Request - State: closed - Opened by gagan3012 10 months ago - 1 comment

#56 - Multilingual LLava bench

Pull Request - State: closed - Opened by gagan3012 10 months ago - 1 comment

#56 - Multilingual LLava bench

Pull Request - State: closed - Opened by gagan3012 10 months ago - 1 comment

#56 - Multilingual LLava bench

Pull Request - State: closed - Opened by gagan3012 10 months ago - 1 comment

#55 - Fix types to allow nullables in `llava_hf.py`

Pull Request - State: closed - Opened by lewtun 10 months ago

#55 - Fix types to allow nullables in `llava_hf.py`

Pull Request - State: closed - Opened by lewtun 10 months ago

#55 - Fix types to allow nullables in `llava_hf.py`

Pull Request - State: closed - Opened by lewtun 10 months ago

#55 - Fix types to allow nullables in `llava_hf.py`

Pull Request - State: closed - Opened by lewtun 10 months ago

#54 - add Llava-SGlang

Pull Request - State: closed - Opened by jzhang38 10 months ago - 5 comments

#54 - add Llava-SGlang

Pull Request - State: closed - Opened by jzhang38 10 months ago - 5 comments

#54 - add Llava-SGlang

Pull Request - State: closed - Opened by jzhang38 10 months ago - 5 comments

#54 - add Llava-SGlang

Pull Request - State: closed - Opened by jzhang38 10 months ago - 5 comments

#51 - different result between lmms-eval and llava-eval on text-vqa

Issue - State: open - Opened by yiyexy 10 months ago - 4 comments

#51 - different result between lmms-eval and llava-eval on text-vqa

Issue - State: open - Opened by yiyexy 10 months ago - 4 comments

#51 - different result between lmms-eval and llava-eval on text-vqa

Issue - State: open - Opened by yiyexy 10 months ago - 4 comments

#51 - different result between lmms-eval and llava-eval on text-vqa

Issue - State: open - Opened by yiyexy 10 months ago - 4 comments