dvlab-research/MGM issues and pull requests

#136 - article error

Issue - State: open - Opened by TuuSiwei 2 months ago

#135 - Unable to Merge LoRA Weights with Base Model: ValueError: Can't find 'adapter_config.json' at ...

Issue - State: open - Opened by PARSA-MHMDI 3 months ago

#134 - CHAIR Evaluation

Issue - State: open - Opened by itsqyh 4 months ago

#133 - TypeError: MGMllamaForCausalLM.forward() got an unexpected keyword argument 'cache_position'

Issue - State: open - Opened by czm708033 4 months ago - 2 comments

#132 - ImportError: cannot import name 'packaging' from 'pkg_resources'

Issue - State: closed - Opened by LiuRicky 5 months ago - 1 comment

#131 - when use stable-diffusion,AttributeError: 'NoneType' object has no attribute 'tokenize'

Issue - State: open - Opened by ALR-alr 5 months ago - 1 comment

#130 - llama3 result is repeated many times

Issue - State: open - Opened by pennypengpm 5 months ago - 1 comment

#129 - Does MGM support in-context(few-shot) inference?

Issue - State: open - Opened by waltonfuture 5 months ago

#128 - Will there be support for Qwen2?

Issue - State: open - Opened by huxian0402 5 months ago

#127 - How to access hidden states?

Issue - State: open - Opened by Divyanshsingh1910 5 months ago - 1 comment

#126 - I get this error: WARNING: tokenization mismatch: 156 vs. 161. (ignored) when I finetune llama3

Issue - State: open - Opened by shidingz 6 months ago - 1 comment

#125 - 使用多gpu启动worker，对话时报错

Issue - State: closed - Opened by kimi360 6 months ago - 1 comment

#124 - Do you meet the error "MGMConfig"?

Issue - State: open - Opened by strawberryrs620 6 months ago

#123 - loss 0 and grad nan

Issue - State: closed - Opened by TuuSiwei 6 months ago - 3 comments

#122 - error in loading

Issue - State: closed - Opened by TuuSiwei 6 months ago

#121 - Requirement for pretraining weights of LLaMa-3-8B-Instruct

Issue - State: open - Opened by shiwk23 6 months ago

#120 - Can provide laion-gpt4v dataset images zip?

Issue - State: open - Opened by TuuSiwei 6 months ago

#119 - mgm-34b-hd, should have a 'model_type' key in its config.json

Issue - State: open - Opened by chrisx599 6 months ago - 2 comments

#118 - Inference problem about the demo.

Issue - State: open - Opened by ApolloRay 6 months ago - 1 comment

#117 - The data for alignment and finetuning contains duplicates. Can you please explain why this is happening?

Issue - State: open - Opened by KANGRuipeng 6 months ago - 1 comment

#116 - dataset miss problem

Issue - State: closed - Opened by TuuSiwei 6 months ago - 1 comment

#115 - | EORROR | stderr | RecursionError: Maximun recursion depth exceeded in comparison

Issue - State: open - Opened by linyf38 6 months ago - 1 comment

#114 - May I ask if the current inference code does not support multi images input

Issue - State: open - Opened by Angelalilyer 6 months ago - 1 comment

#113 - 多轮对话修改图像输入后报错

Issue - State: open - Opened by pennypengpm 7 months ago

#112 - Generation-related Instructions dataset link

Issue - State: open - Opened by berry-ding 7 months ago - 1 comment

#111 - 关于多机多卡效果不如单机多卡好的问题

Issue - State: open - Opened by DePengW 7 months ago - 1 comment

#110 - Loss does not decrease

Issue - State: closed - Opened by yfthu 7 months ago

#109 - 可以放一下生成generation_pure_text数据的代码吗

Issue - State: closed - Opened by pennypengpm 7 months ago - 4 comments

#108 - LLama 70B support

Issue - State: open - Opened by PrateekPal641 7 months ago

#107 - Inference speed

Issue - State: open - Opened by PrateekPal641 7 months ago

#106 - lora initialisation missing from builder.py

Issue - State: open - Opened by adrielkuek 7 months ago - 1 comment

#105 - Error while loading model with transformers library

Issue - State: closed - Opened by PrateekPal641 7 months ago - 3 comments

#104 - Congratulations for the best LLaVA derived models !

Issue - State: open - Opened by deepbeepmeep 7 months ago - 1 comment

#103 - Some weights of the model checkpoint were not used when initializing MGMLlamaForCausalLM

Issue - State: open - Opened by charlesCXK 7 months ago - 2 comments

#102 - how to use stage2 ckpt fine-tuning stage3？

Issue - State: open - Opened by linqinguang 7 months ago - 1 comment

#101 - 计划加入DPO训练来缓解模型幻觉问题吗

Issue - State: open - Opened by jiezhangGt 7 months ago

#100 - Take input image as condition.

Issue - State: closed - Opened by Adenialzz 7 months ago - 2 comments

#99 - How to fix [NETWORK ERROR DUE TO HIGH TRAFFIC. ] on MacOS ?

Issue - State: open - Opened by seasoncool 7 months ago - 2 comments

#98 - stage2 loss is 0

Issue - State: closed - Opened by jiezhangGt 7 months ago - 1 comment

#97 - Excessive Length of Responses from Mini Gemini

Issue - State: closed - Opened by Dopplenum 7 months ago - 1 comment

#96 - AttributeError: 'OpenCLIPVisionTower' object has no attribute 'device'

Issue - State: open - Opened by l1019008146 7 months ago - 2 comments

#95 - Use of ocr in Evaluation

Issue - State: open - Opened by bruceisme 7 months ago - 1 comment

#94 - 'LlamaForCausalLM' object has no attribute 'get_vision_tower'

Issue - State: closed - Opened by HongLouyemeng 7 months ago - 1 comment

#93 - 使用cli调用自定义微调模型，出现'OpenCLIPVisionTower' object has no attribute 'device'

Issue - State: closed - Opened by HongLouyemeng 7 months ago - 16 comments

#92 - pretrain error: lack of preprocessor_config.json

Issue - State: closed - Opened by jiezhangGt 7 months ago - 1 comment

#91 - Which deepspeed version is it

Issue - State: open - Opened by Kareneveve 7 months ago - 2 comments

#90 - 当我使用推理命令的时候出现网络错误，无法构建推理的接口

Issue - State: closed - Opened by HongLouyemeng 7 months ago - 2 comments

#89 - 请问为什么在训练llama的脚本中，预训练和微调所使用的conv不一样

Issue - State: open - Opened by shidingz 7 months ago - 1 comment

#88 - model asks self questions and answers

Issue - State: open - Opened by Bowei-Li 7 months ago - 2 comments

#87 - 为什么输出结果为nan呢

Issue - State: closed - Opened by freja-zy 7 months ago - 1 comment

#86 - how to prompt to get short response

Issue - State: open - Opened by GallonDeng 7 months ago - 2 comments

#85 - Some questions about the demo

Issue - State: open - Opened by cyy-1234 7 months ago - 3 comments

#84 - Huggingface inference script

Issue - State: open - Opened by berry-ding 7 months ago - 1 comment

#83 - Finetune

Issue - State: open - Opened by ZhangScream 7 months ago - 7 comments

#82 - Deployed mini-Gemini in the Windows system and encountered the following error during the “Launch a Graph web server” step.Seeking help to resolve the issue

Issue - State: open - Opened by Janusmsr 7 months ago - 2 comments

#81 - Deployed mini-Gemini in the Windows system and encountered the following error during the ”Launch a Graph web server“ step. seeking help from a skilled user to resolve the issue

Issue - State: closed - Opened by Janusmsr 7 months ago - 1 comment

#80 - Updated the Readme file and make it more brief

Pull Request - State: open - Opened by Dopplenum 7 months ago

#79 - Can this model do graph prediction tasks? For example, predict the future trend of personal social graph.

Issue - State: open - Opened by brainplait 7 months ago - 1 comment

#78 - About the idea to further enhance the performance.

Issue - State: open - Opened by lucasjinreal 7 months ago - 4 comments

#77 - 加载模型时卡住了

Issue - State: closed - Opened by kimi360 7 months ago - 3 comments

#76 - 请问为什么在执行”python -m minigemini.serve.controller --host 0.0.0.0 --port 10000“时会出现 404 Not Found

Issue - State: open - Opened by liuwenxin0410 7 months ago - 11 comments

#75 - 为啥我6张4090 24G,微调Mini-Gemini-8x7B会显存不够QAQ

Issue - State: closed - Opened by HongLouyemeng 7 months ago - 4 comments

#74 - ModuleNotFoundError: No module named 'open_clip'

Issue - State: closed - Opened by XHB-ZMM 7 months ago - 1 comment

#73 - local variable 'data_dict' referenced before assignment 这个问题是有同学碰到过吗

Issue - State: closed - Opened by HongLouyemeng 7 months ago - 1 comment

#72 - Inquery about simple request

Issue - State: open - Opened by madhatter349 7 months ago - 2 comments

#71 - Inquery about the missing images from ocr_vqa, sam, gpt4v-dataset and ALLaVA-4V

Issue - State: open - Opened by patrick-tssn 7 months ago

#70 - button and menu clickdown does not work

Issue - State: open - Opened by jeevikasirwani 7 months ago

#69 - add autoscroll

Issue - State: open - Opened by jeevikasirwani 7 months ago

#68 - raise ValueError(f'Not find vision tower: {vision_tower}')

Issue - State: open - Opened by freja-zy 7 months ago - 4 comments

#67 - 如何调用api

Issue - State: closed - Opened by RoronoaZoroh 7 months ago - 1 comment

#66 - fix bug when using batch size 1

Pull Request - State: closed - Opened by xylcbd 7 months ago - 2 comments

#65 - 部署成功试了后，有时会循环输出，还有对中文不是很友好

Issue - State: open - Opened by chenhaoqiang 7 months ago - 4 comments

#64 - Fixed multiple typos in README.md file

Pull Request - State: closed - Opened by Hunaid2000 7 months ago

#63 - You are using a model of type mini_gemini_mixtral to instantiate a model of type mini_gemini. This is not supported for all configurations of models and can yield errors.

Issue - State: open - Opened by lightmatmul 7 months ago - 1 comment

#62 - Update train.py

Pull Request - State: closed - Opened by lightmatmul 7 months ago

#61 - Re-opening issue: Mini-Gemini Model Fine-Tuning Anomaly

Issue - State: closed - Opened by lightmatmul 7 months ago - 4 comments

#60 - LMDeploy is gonna support the inference of MiniGemini :rocket:

Issue - State: closed - Opened by AllentDan 7 months ago - 1 comment

#59 - Failed to continous sft for yi-34B with 8x CUDA graphics card! (deepspeed zero3)

Issue - State: closed - Opened by xylcbd 7 months ago - 2 comments

#58 - For multi image

Issue - State: closed - Opened by ZhangScream 7 months ago - 2 comments

#57 - 关于eval的小问题

Issue - State: open - Opened by liuwenhaha 7 months ago - 1 comment

#56 - Regarding license for using the models

Issue - State: closed - Opened by thiner 7 months ago - 1 comment

#55 - minigemini.model.multimodal_encoder.openclip_encoder.CLIP() got NoneType

Issue - State: closed - Opened by SuSung-boy 7 months ago - 1 comment

#54 - demo page not working

Issue - State: closed - Opened by GallonDeng 7 months ago - 1 comment

#53 - API service support like vllm or sglang?

Issue - State: closed - Opened by xiechengmude 7 months ago - 1 comment

#52 - Feature Request: llama.cpp support

Issue - State: closed - Opened by deutschthomas 7 months ago - 1 comment

#51 - About vl model path

Issue - State: closed - Opened by AllentDan 7 months ago - 2 comments

#50 - minigemini_instruction.json包含了预训练的LLaVA Images，但是在readme中没有写到微调用到了这部分数据

Issue - State: closed - Opened by shidingz 7 months ago - 1 comment

#49 - Continue FT from stage 2 with custom data

Issue - State: closed - Opened by adrielkuek 7 months ago - 2 comments

#48 - Questions about how to enlarge the base vision tower input resolution

Issue - State: open - Opened by lucasjinreal 7 months ago - 3 comments

#47 - 运行代码报错AttributeError: 'list' object has no attribute 'to'， image_aux_features_raw = self.get_model().get_vision_tower_aux()(images_aux).to(dtype=image_features.dtype, device=image_features.device)

Issue - State: closed - Opened by shidingz 7 months ago - 3 comments

#46 - Questions about change ViT to 378 input resolution, but got poor results.

Issue - State: open - Opened by OpenJarvisAI 7 months ago - 5 comments

#45 - 怎么从你们的检查点启动呢

Issue - State: closed - Opened by HongLouyemeng 7 months ago - 17 comments

#44 - How many SAM images were used from ShareGPT4v?

Issue - State: open - Opened by OpenJarvisAI 7 months ago - 2 comments

#43 - 关于代码实现的疑问

Issue - State: closed - Opened by hhaAndroid 7 months ago - 5 comments

#42 - loss suddenly drop to 0 and remain 0

Issue - State: closed - Opened by huxian0402 8 months ago - 2 comments

#41 - batching giving weird outputs

Issue - State: open - Opened by mukundkhanna123 8 months ago

#40 - RuntimeError: Expected 3D (unbatched) or 4D (batched) input to conv2d, but got input of size: [1, 1, 3, 336, 336]

Issue - State: closed - Opened by plf1996 8 months ago - 3 comments

GitHub / dvlab-research/MGM issues and pull requests