Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / EvolvingLMMs-Lab/lmms-eval issues and pull requests

#100 - add Conbench

Pull Request - State: closed - Opened by Gumpest 8 months ago - 1 comment

#100 - add Conbench

Pull Request - State: closed - Opened by Gumpest 8 months ago - 1 comment

#100 - add Conbench

Pull Request - State: closed - Opened by Gumpest 8 months ago - 1 comment

#100 - add Conbench

Pull Request - State: closed - Opened by Gumpest 8 months ago - 1 comment

#99 - How to contribute a new dataset?

Issue - State: open - Opened by JohnTang93 8 months ago - 1 comment

#99 - How to contribute a new dataset?

Issue - State: open - Opened by JohnTang93 8 months ago - 1 comment

#99 - How to contribute a new dataset?

Issue - State: open - Opened by JohnTang93 8 months ago - 1 comment

#99 - How to contribute a new dataset?

Issue - State: open - Opened by JohnTang93 8 months ago - 1 comment

#98 - qwenvl-7b evaluate refcoco|+|g cider and IOU are all None,

Issue - State: open - Opened by AderonHuang 9 months ago - 24 comments

#98 - qwenvl-7b evaluate refcoco|+|g cider and IOU are all None,

Issue - State: open - Opened by AderonHuang 9 months ago - 24 comments

#98 - qwenvl-7b evaluate refcoco|+|g cider and IOU are all None,

Issue - State: open - Opened by AderonHuang 9 months ago - 24 comments

#98 - qwenvl-7b evaluate refcoco|+|g cider and IOU are all None,

Issue - State: open - Opened by AderonHuang 9 months ago - 24 comments

#97 - Add MathVerse in README.md

Pull Request - State: closed - Opened by CaraJ7 9 months ago

#97 - Add MathVerse in README.md

Pull Request - State: closed - Opened by CaraJ7 9 months ago

#97 - Add MathVerse in README.md

Pull Request - State: closed - Opened by CaraJ7 9 months ago

#97 - Add MathVerse in README.md

Pull Request - State: closed - Opened by CaraJ7 9 months ago

#96 - Any plan for supporting Lora peft loading

Issue - State: closed - Opened by hxhcreate 9 months ago - 1 comment

#96 - Any plan for supporting Lora peft loading

Issue - State: closed - Opened by hxhcreate 9 months ago - 1 comment

#96 - Any plan for supporting Lora peft loading

Issue - State: closed - Opened by hxhcreate 9 months ago - 1 comment

#96 - Any plan for supporting Lora peft loading

Issue - State: closed - Opened by hxhcreate 9 months ago - 1 comment

#95 - add MM-UPD

Pull Request - State: closed - Opened by AtsuMiyai 9 months ago - 6 comments

#95 - add MM-UPD

Pull Request - State: closed - Opened by AtsuMiyai 9 months ago - 6 comments

#95 - add MM-UPD

Pull Request - State: closed - Opened by AtsuMiyai 9 months ago - 6 comments

#95 - add MM-UPD

Pull Request - State: closed - Opened by AtsuMiyai 9 months ago - 6 comments

#94 - Error when import InstructBLIP

Issue - State: open - Opened by hanqiu-hq 9 months ago - 1 comment

#94 - Error when import InstructBLIP

Issue - State: open - Opened by hanqiu-hq 9 months ago - 1 comment

#94 - Error when import InstructBLIP

Issue - State: open - Opened by hanqiu-hq 9 months ago - 1 comment

#94 - Error when import InstructBLIP

Issue - State: open - Opened by hanqiu-hq 9 months ago - 1 comment

#93 - Add m3exam

Pull Request - State: closed - Opened by Jiawei-Guo 9 months ago
Labels: question

#93 - Add m3exam

Pull Request - State: closed - Opened by Jiawei-Guo 9 months ago
Labels: question

#93 - Add m3exam

Pull Request - State: closed - Opened by Jiawei-Guo 9 months ago
Labels: question

#93 - Add m3exam

Pull Request - State: closed - Opened by Jiawei-Guo 9 months ago
Labels: question

#92 - add multi-lingual MMMU tasks

Pull Request - State: closed - Opened by Junpliu 9 months ago

#92 - add multi-lingual MMMU tasks

Pull Request - State: closed - Opened by Junpliu 9 months ago

#92 - add multi-lingual MMMU tasks

Pull Request - State: closed - Opened by Junpliu 9 months ago

#92 - add multi-lingual MMMU tasks

Pull Request - State: closed - Opened by Junpliu 9 months ago

#91 - how to use “few shot”

Issue - State: open - Opened by woshidengweimo 9 months ago - 1 comment

#91 - how to use “few shot”

Issue - State: open - Opened by woshidengweimo 9 months ago - 1 comment

#91 - how to use “few shot”

Issue - State: open - Opened by woshidengweimo 9 months ago - 1 comment

#91 - how to use “few shot”

Issue - State: open - Opened by woshidengweimo 9 months ago - 1 comment

#90 - Prompt for MathVista

Issue - State: closed - Opened by yqy2001 9 months ago - 2 comments

#90 - Prompt for MathVista

Issue - State: closed - Opened by yqy2001 9 months ago - 2 comments

#90 - Prompt for MathVista

Issue - State: closed - Opened by yqy2001 9 months ago - 2 comments

#90 - Prompt for MathVista

Issue - State: closed - Opened by yqy2001 9 months ago - 2 comments

#89 - W&B Logging Issue on MMMU & Wrong parsed_pred

Issue - State: open - Opened by AlekseyKorshuk 9 months ago - 1 comment

#89 - W&B Logging Issue on MMMU & Wrong parsed_pred

Issue - State: open - Opened by AlekseyKorshuk 9 months ago - 1 comment

#89 - W&B Logging Issue on MMMU & Wrong parsed_pred

Issue - State: open - Opened by AlekseyKorshuk 9 months ago - 1 comment

#89 - W&B Logging Issue on MMMU & Wrong parsed_pred

Issue - State: open - Opened by AlekseyKorshuk 9 months ago - 1 comment

#88 - No "llava_llama_3" template

Issue - State: open - Opened by yqy2001 9 months ago - 2 comments

#88 - No "llava_llama_3" template

Issue - State: open - Opened by yqy2001 9 months ago - 2 comments

#88 - No "llava_llama_3" template

Issue - State: open - Opened by yqy2001 9 months ago - 2 comments

#88 - No "llava_llama_3" template

Issue - State: open - Opened by yqy2001 9 months ago - 2 comments

#87 - Adding microsoft/Phi-3-vision-128k-instruct model.

Pull Request - State: closed - Opened by vfragoso 9 months ago

#87 - Adding microsoft/Phi-3-vision-128k-instruct model.

Pull Request - State: closed - Opened by vfragoso 9 months ago

#87 - Adding microsoft/Phi-3-vision-128k-instruct model.

Pull Request - State: closed - Opened by vfragoso 9 months ago

#87 - Adding microsoft/Phi-3-vision-128k-instruct model.

Pull Request - State: closed - Opened by vfragoso 9 months ago

#86 - Question on evaluation across multiple tasks

Issue - State: closed - Opened by AtsuMiyai 9 months ago

#86 - Question on evaluation across multiple tasks

Issue - State: closed - Opened by AtsuMiyai 9 months ago

#86 - Question on evaluation across multiple tasks

Issue - State: closed - Opened by AtsuMiyai 9 months ago

#86 - Question on evaluation across multiple tasks

Issue - State: closed - Opened by AtsuMiyai 9 months ago

#85 - how to get three pope results [rad,pop,adv] with lmms-eval?

Issue - State: closed - Opened by baiyuting 9 months ago - 2 comments

#85 - how to get three pope results [rad,pop,adv] with lmms-eval?

Issue - State: closed - Opened by baiyuting 9 months ago - 2 comments

#85 - how to get three pope results [rad,pop,adv] with lmms-eval?

Issue - State: closed - Opened by baiyuting 9 months ago - 2 comments

#85 - how to get three pope results [rad,pop,adv] with lmms-eval?

Issue - State: closed - Opened by baiyuting 9 months ago - 2 comments

#84 - [Week 1] Adding 5 examples for Hindi

Pull Request - State: closed - Opened by simran-khanuja 9 months ago

#84 - [Week 1] Adding 5 examples for Hindi

Pull Request - State: closed - Opened by simran-khanuja 9 months ago

#84 - [Week 1] Adding 5 examples for Hindi

Pull Request - State: closed - Opened by simran-khanuja 9 months ago

#84 - [Week 1] Adding 5 examples for Hindi

Pull Request - State: closed - Opened by simran-khanuja 9 months ago

#83 - [Feature Request] Evaluating quantized models

Issue - State: closed - Opened by zjysteven 9 months ago - 3 comments

#83 - [Feature Request] Evaluating quantized models

Issue - State: closed - Opened by zjysteven 9 months ago - 3 comments

#83 - [Feature Request] Evaluating quantized models

Issue - State: closed - Opened by zjysteven 9 months ago - 3 comments

#83 - [Feature Request] Evaluating quantized models

Issue - State: closed - Opened by zjysteven 9 months ago - 3 comments

#82 - Error while testing Qwen on POPE

Issue - State: closed - Opened by hxhcreate 9 months ago - 1 comment

#82 - Error while testing Qwen on POPE

Issue - State: closed - Opened by hxhcreate 9 months ago - 1 comment

#82 - Error while testing Qwen on POPE

Issue - State: closed - Opened by hxhcreate 9 months ago - 1 comment

#82 - Error while testing Qwen on POPE

Issue - State: closed - Opened by hxhcreate 9 months ago - 1 comment

#81 - LLaVA Benchmark

Issue - State: open - Opened by yqy2001 9 months ago - 2 comments

#81 - LLaVA Benchmark

Issue - State: open - Opened by yqy2001 9 months ago - 2 comments

#81 - LLaVA Benchmark

Issue - State: open - Opened by yqy2001 9 months ago - 2 comments

#81 - LLaVA Benchmark

Issue - State: open - Opened by yqy2001 9 months ago - 2 comments

#80 - Extremely Low Performance with LLaVA-1.5-7B on MM-Vet Benchmark

Issue - State: open - Opened by lambert-x 9 months ago - 6 comments

#80 - Extremely Low Performance with LLaVA-1.5-7B on MM-Vet Benchmark

Issue - State: open - Opened by lambert-x 9 months ago - 6 comments

#80 - Extremely Low Performance with LLaVA-1.5-7B on MM-Vet Benchmark

Issue - State: open - Opened by lambert-x 9 months ago - 6 comments

#80 - Extremely Low Performance with LLaVA-1.5-7B on MM-Vet Benchmark

Issue - State: open - Opened by lambert-x 9 months ago - 6 comments

#79 - [MultiGPU Evaluation Error]: model on differencet device

Issue - State: closed - Opened by hxhcreate 9 months ago - 2 comments

#79 - [MultiGPU Evaluation Error]: model on differencet device

Issue - State: closed - Opened by hxhcreate 9 months ago - 2 comments

#79 - [MultiGPU Evaluation Error]: model on differencet device

Issue - State: closed - Opened by hxhcreate 9 months ago - 2 comments

#79 - [MultiGPU Evaluation Error]: model on differencet device

Issue - State: closed - Opened by hxhcreate 9 months ago - 2 comments

#78 - "llava_hf" is not registered

Issue - State: open - Opened by kyleliang919 9 months ago - 5 comments

#78 - "llava_hf" is not registered

Issue - State: open - Opened by kyleliang919 9 months ago - 5 comments

#78 - "llava_hf" is not registered

Issue - State: open - Opened by kyleliang919 9 months ago - 5 comments

#78 - "llava_hf" is not registered

Issue - State: open - Opened by kyleliang919 9 months ago - 5 comments

#77 - Too many requests for MathVista

Issue - State: open - Opened by baichuanzhou 9 months ago - 1 comment

#77 - Too many requests for MathVista

Issue - State: open - Opened by baichuanzhou 9 months ago - 1 comment

#77 - Too many requests for MathVista

Issue - State: open - Opened by baichuanzhou 9 months ago - 1 comment

#77 - Too many requests for MathVista

Issue - State: open - Opened by baichuanzhou 9 months ago - 1 comment