haotian-liu/LLaVA issues and pull requests

#1656 - comment out torch requirement

Pull Request - State: closed - Opened by nguyendinhson-kaist 3 months ago

#1656 - comment out torch requirement

Pull Request - State: closed - Opened by nguyendinhson-kaist 3 months ago

#1655 - [Question]detailed information about tokenizer and encoder

Issue - State: open - Opened by dnkscu 3 months ago

#1654 - [Usage] why the demo of LLaVA-1.5 can not use now？

Issue - State: open - Opened by moonnnpie 3 months ago

#1654 - [Usage] why the demo of LLaVA-1.5 can not use now？

Issue - State: open - Opened by moonnnpie 3 months ago

#1653 - [Question] Could you tell me any difference between llava-v1.5-13b.jsonl and old llava-v1.5-7b.jsonl?

Issue - State: open - Opened by yukio0321 3 months ago

#1651 - [Usage] LLaVA-v1.6-vicuna-7b generates incomplete sentence

Issue - State: closed - Opened by gymbeijing 3 months ago - 1 comment

#1650 - [Question] Learning is completed, but only the weights of the projector are output.

Issue - State: closed - Opened by kouyakamada 3 months ago

#1649 - Why are the first image and the remaining images processed separately? in prepare_inputs_labels_for_multimodal()

Issue - State: open - Opened by shure-dev 3 months ago

#1648 - llava 1.2.2.post1 requires torch==2.1.2, but you have torch 2.0.1 which is incompatible.

Issue - State: open - Opened by Mike-ihr 3 months ago - 3 comments

#1647 - [Usage] Freezing Vision Encoder During LoRA Training

Issue - State: open - Opened by xie-qiang 3 months ago

#1646 - You are using a model of type llava to instantiate a model of type llava_llama. This is not supported for all configurations of models and can yield errors.

Issue - State: open - Opened by XCF-Mike 3 months ago - 1 comment

#1645 - [Question] Inference Code

Issue - State: closed - Opened by bryanwong17 3 months ago - 1 comment

#1644 - [Question] Misaligned image_grid_pinpoints

Issue - State: open - Opened by Forence1999 3 months ago

#1643 - about multi-image input

Issue - State: open - Opened by eternal8080 3 months ago

#1642 - [Question] Image and Text Embeddings For Downstream Tasks

Issue - State: open - Opened by dipikakhullar 3 months ago - 1 comment

#1641 - load_dataset('liuhaotian/LLaVA-Instruct-150K') ERROR

Issue - State: open - Opened by YerongLi 3 months ago

#1639 - [Usage] KeyError: 'LlavaMistralConfig' when I start a model-worker with a fine-tuned Llava-v1.6-mistral-7B model.

Issue - State: closed - Opened by yunsaijc 3 months ago - 2 comments

#1638 - Error when load model in 4bit

Issue - State: open - Opened by rin2401 3 months ago - 1 comment

#1637 - [Question] Fine-tuned model ignore some of the captions

Issue - State: open - Opened by AbdulrahmanSoliman1 3 months ago

#1636 - How can I custom loss function during fine-tuning?[Question]

Issue - State: open - Opened by qingyunyanran 3 months ago - 4 comments

#1635 - Error when saving model: Invalid generation config due to conflicting parameters

Issue - State: closed - Opened by ohhan777 3 months ago - 6 comments

#1634 - NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE.[Usage]

Issue - State: open - Opened by sivang 3 months ago - 1 comment

#1633 - Can this model be adapted to torch2.3？

Issue - State: open - Opened by zxy1728 3 months ago

#1632 - [Question] About how to deal with multiple-answer question in vqav2

Issue - State: open - Opened by BroJunn 4 months ago

#1632 - [Question] About how to deal with multiple-answer question in vqav2

Issue - State: open - Opened by BroJunn 4 months ago

#1630 - [Usage] Visual instruction tuning for LLaVa 1.6

Issue - State: open - Opened by mattia-re-learn 4 months ago - 6 comments

#1629 - [Usage] Error when use cli.py after merge-lora-weights.py

Issue - State: open - Opened by dctnorin 4 months ago - 1 comment

#1628 - 拉了一个多模态大模型技术交流群，大家可以加入进来进行技术交流

Issue - State: open - Opened by feihuamantian 4 months ago - 3 comments

#1625 - [Usage] There is no output from the ASSISTANT when using CLI Inference????

Issue - State: open - Opened by xlxcomputer 4 months ago - 1 comment

#1624 - [Usage] from llava.mm_utils import ( ImportError: cannot import name 'load_pretrained_model' from 'llava.mm_utils'

Issue - State: closed - Opened by chenqi-205 4 months ago

#1622 - [Usage] You are using a model of type llava to instantiate a model of type llava_llama. This is not supported for all configurations of models and can yield errors.

Issue - State: open - Opened by matengxiaotiancai 4 months ago - 2 comments

#1619 - [Question] "Size mismatch" Error when finetuning from a projector

Issue - State: open - Opened by hvgupta 4 months ago - 5 comments

#1618 - Some images of ocr vqa data in llava_v1_5_mix665k.json do not exist!

Issue - State: open - Opened by Vicent0205 4 months ago - 5 comments

#1613 - [Questio why 'mm_vision_select_layer' == -2 in config ? n]

Issue - State: open - Opened by fmy7834 4 months ago - 2 comments

#1611 - [Usage] Error when downloading the weight files manually from huggingface

Issue - State: open - Opened by hee-dongdong 4 months ago - 1 comment

#1610 - [Question] 想换掉Vicuna 模型

Issue - State: open - Opened by wanglongpeng1 4 months ago - 1 comment

#1610 - [Question] 想换掉Vicuna 模型

Issue - State: open - Opened by wanglongpeng1 4 months ago - 1 comment

#1607 - [Question] Unable to submit VQAv2 result file to the evaluation server

Issue - State: open - Opened by XiaoruiMaLU 4 months ago - 2 comments

#1605 - [Usage] How to input multiple pictures and prompts at the same time

Issue - State: open - Opened by ZihangDu 4 months ago - 1 comment

#1603 - Pretraining LLaVA with SigLIP

Pull Request - State: open - Opened by nahidalam 4 months ago - 3 comments

#1600 - Need either a `state_dict` or a `save_folder` containing offloaded weights.

Issue - State: open - Opened by zhaodaojie 4 months ago - 2 comments

#1597 - [Usage] how to load the trained lora?

Issue - State: closed - Opened by feiyangsuo 4 months ago - 4 comments

#1592 - [Usage] After fine-tuning LLaVA 1.5, mm_projector.bin file is not available

Issue - State: open - Opened by rayluo88 4 months ago - 2 comments

#1589 - [Question] I got stuck here while doing fine-tuning training.

Issue - State: open - Opened by zy1996829 4 months ago - 2 comments

#1584 - automatically added to python module search path

Pull Request - State: open - Opened by inkcherry 4 months ago - 2 comments

#1582 - device mis-match error on pre-training

Issue - State: open - Opened by oroojlooy 4 months ago - 1 comment

#1582 - device mis-match error on pre-training

Issue - State: open - Opened by oroojlooy 4 months ago - 4 comments

#1580 - [Question] Mac run but get importlib.metadata.PackageNotFoundError: No package metadata was found for bitsandbytes

Issue - State: open - Opened by Timothychen00 5 months ago - 4 comments

#1578 - [Question] Why LLaVA-1.5-13b keep repeatting answer in the first round conversation?

Issue - State: open - Opened by PoopBear1 5 months ago - 3 comments

#1573 - add qwen2 support for pretraining and finetuning

Pull Request - State: open - Opened by TobyYang7 5 months ago - 5 comments

#1570 - [Usage] 7B Inferece CUDA Out of Memory for RTX 4090 24GB VRAM

Issue - State: open - Opened by 0010SS 5 months ago - 1 comment

#1568 - "Assertion `srcIndex < srcSelectDimSize` failed" in Docker on some systems

Issue - State: closed - Opened by Careiner 5 months ago - 3 comments

#1568 - "Assertion `srcIndex < srcSelectDimSize` failed" in Docker on some systems

Issue - State: closed - Opened by Careiner 5 months ago - 3 comments

#1567 - Having issues while merging LoRA attention weights

Issue - State: closed - Opened by anas-zafar 5 months ago - 5 comments

#1554 - [Usage] Not able to fine tune the LLaVA model with llava-v1.5-7b.

Issue - State: open - Opened by ayushgupta9198 5 months ago - 6 comments

#1552 - [Usage] Merging LoRa weights into llava-13b fails with bizarre error

Issue - State: open - Opened by maxall41 5 months ago - 3 comments

#1551 - [Discussion] Request for Guidance on Stage Two: Converting LLaVA-V1.6 into 4-bit GGUF Format (PAPER)

Issue - State: open - Opened by rohithbojja 5 months ago - 2 comments

#1548 - [Usage] Llava-Gemma Pretraining + Fine tuning Usage issue and missing Fine tuned projector.bin

Issue - State: open - Opened by nlpkiddo-2001 5 months ago - 1 comment

#1536 - How to perform cross-validation during the fine-tuning process?[Question]

Issue - State: open - Opened by J0eky 5 months ago - 1 comment

#1535 - load_dataset(liuhaotian/LLaVA-Instruct-150K) fails with error, when generating train split: 394276 examples

Issue - State: open - Opened by kzos 5 months ago - 2 comments

#1529 - [Question] Why apply the causal mask to image tokens in the attention operations of LLM?

Issue - State: open - Opened by yubin1219 6 months ago - 2 comments

#1527 - errors in MME evaluation

Issue - State: open - Opened by liuliAI 6 months ago - 4 comments

#1522 - [Usage] Need to install protobuf in order to run CLI Inference

Issue - State: open - Opened by cuihenggang 6 months ago - 5 comments

#1519 - [Usage] Error while using finetuned model

Issue - State: open - Opened by yesgvinayak 6 months ago - 4 comments

#1516 - The ``device`` args doesn't take effect

Issue - State: open - Opened by Yonggie 6 months ago - 4 comments

#1512 - [Question] Clarification on JSON Format for Object Detection Task

Issue - State: open - Opened by makemecker 6 months ago - 2 comments

#1507 - [Question] add extral layers in llava

Issue - State: open - Opened by NONGFUYULANG 6 months ago - 2 comments

#1506 - [Discussion] Question about prompt

Issue - State: open - Opened by avinashsai 6 months ago - 2 comments

#1496 - [Usage] Continue training from pre-trained checkpoint

Issue - State: open - Opened by orrzohar 6 months ago - 3 comments

#1485 - Update README.md. SoM-LLaVA from Microsoft.

Pull Request - State: open - Opened by zzxslp 6 months ago

#1482 - [Usage] Extremely Slow Inference Speed

Issue - State: open - Opened by HuangZhen02 6 months ago - 1 comment

#1481 - [Usage] Deepspeed Zero Stage 3 not able to shard the model

Issue - State: open - Opened by shubhamagarwal92 6 months ago - 3 comments

#1475 - [Usage] None of the inputs have requires_grad=True. Gradients will be None

Issue - State: open - Opened by hellangleZ 6 months ago - 12 comments

#1473 - [Question] How to batch inference?

Issue - State: open - Opened by catssci 6 months ago - 3 comments

#1459 - [Question] How to use multi GPUs with LlavaLlamaForCausalLM.from_pretrained

Issue - State: open - Opened by JesseZZZZZ 7 months ago - 1 comment

#1458 - add ocr vqa images

Pull Request - State: open - Opened by Victorwz 7 months ago - 11 comments

#1456 - [Question] with default setting, LoRA has no improvement is normal? 默认设置下，LoRA 完全没效果正常吗？

Issue - State: closed - Opened by fisher75 7 months ago - 4 comments

#1455 - [LLaVa demo server down]

Issue - State: open - Opened by lizaibeim 7 months ago - 2 comments

#1455 - [LLaVa demo server down]

Issue - State: open - Opened by lizaibeim 7 months ago - 2 comments

#1446 - Add NPU support for Llava

Pull Request - State: open - Opened by HelloWorldBeginner 7 months ago - 7 comments

#1439 - Update pyproject.toml for lora dependencies;

Pull Request - State: open - Opened by fisher75 7 months ago

#1437 - Cannot resume from checkpoint when 8-bit LoRA fine-tuning with deepspeed zero2_offload

Issue - State: open - Opened by jirvin16 7 months ago - 3 comments

#1432 - Llava becomes repetitive on OCR tasks

Issue - State: open - Opened by prhbrt 7 months ago - 2 comments

#1428 - [Question] loss is 0 when I finetune on my custom dataset,and runtime error

Issue - State: open - Opened by wuwu-C 7 months ago - 5 comments

#1405 - [Question] Batch Inference Lora Model + SGLang Supported LLaVa models 请问批量推理还支持哪些LLaVa的model？Lora后的模型如何批量推理？

Issue - State: closed - Opened by fisher75 7 months ago - 3 comments

#1396 - LLaVA-1.6 training dataset and training code.

Issue - State: open - Opened by shengyuwoo 7 months ago - 2 comments

#1391 - Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

Issue - State: open - Opened by jiezhangGt 7 months ago - 6 comments

#1391 - Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

Issue - State: open - Opened by jiezhangGt 7 months ago - 5 comments

#1389 - [Usage] multiple inference getting wrong results

Issue - State: open - Opened by xinghedyc 7 months ago - 2 comments

#1373 - [Question] projector not saved in the finetuning stage

Issue - State: open - Opened by St3p99 7 months ago - 1 comment

#1364 - [Discussion] More data leading to lower indicators?

Issue - State: open - Opened by uniquehou 7 months ago - 4 comments

#1356 - [Question] google.protobuf.message.DecodeError: Error parsing message with type 'sentencepiece.ModelProto' May I ask how to solve this problem?

Issue - State: open - Opened by xffzzt 7 months ago - 1 comment

#1334 - [Usage] minimum GPU memory for LLaVA-7B

Issue - State: open - Opened by CinKKKyo 8 months ago - 2 comments

#1334 - [Usage] minimum GPU memory for LLaVA-7B

Issue - State: open - Opened by CinKKKyo 8 months ago - 2 comments

#1332 - NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE. (error_code: 1)

Issue - State: open - Opened by jeevanmac011769 8 months ago - 10 comments

#1326 - [Usage] When will the eval script for llava v1.6 be available?

Issue - State: open - Opened by xushilin1 8 months ago - 3 comments

#1317 - [Usage] Is there a python script to run infer ?

Issue - State: closed - Opened by hollyaoyaozi 8 months ago - 2 comments

#1305 - [Usage] How to do batch image inference using run_llava and LLAVA 1.6 model?

Issue - State: open - Opened by siddharthshah52 8 months ago - 6 comments

#1304 - [Discussion] ms-swift对于Llava系列微调(finetune)的支持

Issue - State: open - Opened by Jintao-Huang 8 months ago - 1 comment

GitHub / haotian-liu/LLaVA issues and pull requests