EvolvingLMMs-Lab/Otter issues and pull requests

#348 - NotImplementedError: Cannot copy out of meta tensor; no data!

Issue - State: open - Opened by chilljudaoren about 1 month ago

#347 - empty tensors when saving checkpoint

Issue - State: open - Opened by Guardian-in-the-WF 3 months ago

#346 - Total Trainable param

Issue - State: open - Opened by Guardian-in-the-WF 3 months ago

#345 - OtterHD pretrain checkpoint

Issue - State: open - Opened by FightingFighting 4 months ago

#344 - Paper picture drawing problem

Issue - State: open - Opened by r1ng13 7 months ago

#343 - [dataset] When will the MagnifierBench be released?

Issue - State: open - Opened by jungle-gym-ac 10 months ago - 1 comment

#342 - About use module "github.com/dolthub/maphash"

Issue - State: closed - Opened by woodliu 11 months ago

#341 - Inference with "luodian/OTTER-MPT7B-Init" model

Issue - State: open - Opened by iz2late 11 months ago

#340 - Demo Page Error

Issue - State: open - Opened by aliman80 11 months ago

#339 - What is the relationship between this repository and Vision Pro? They seem to be completely unrelated

Issue - State: closed - Opened by KIDJourney 12 months ago - 1 comment

#338 - Download failure: MIMICIT/CGD/train-00008-of-00053.parquet

Issue - State: closed - Opened by HiHiAllen 12 months ago - 8 comments

#337 - [demo]: Inferring Multiple choice Questions

Issue - State: open - Opened by appledora 12 months ago

#336 - loading pretrained model

Issue - State: closed - Opened by Oliveyo 12 months ago - 5 comments

#335 - Where are the data for SD and GSD?

Issue - State: closed - Opened by VictorSanh about 1 year ago - 2 comments

#334 - Architecture question OtterHD

Issue - State: closed - Opened by SinanAkkoyun about 1 year ago - 4 comments

#333 - OtterHD Fuyu weights?

Issue - State: open - Opened by SinanAkkoyun about 1 year ago - 1 comment

#332 - How to perform inference after successfully configuring the environment?

Issue - State: closed - Opened by Bin-ze about 1 year ago - 2 comments

#331 - Where can I find the checkpoint of OtterHD?

Issue - State: closed - Opened by Labmem009 about 1 year ago - 2 comments

#330 - [bug] missing `import pyarrow.parquet as pq` in mimicit_dataset.py

Issue - State: closed - Opened by kxqt about 1 year ago - 1 comment

#329 - CGD_instructions references images that aren't there in the parquets

Issue - State: open - Opened by StrangeTcy about 1 year ago - 1 comment

#328 - MME result mismatch of luodian/OTTER-MPT-7B

Issue - State: closed - Opened by Li-Qingyun about 1 year ago

#327 - Question about missed values in `batch_mimicit`, maybe because of using `delete_tensors_from_dict` and `pop`

Issue - State: closed - Opened by Li-Qingyun about 1 year ago

#326 - concating error in collate_fn when replicate otter with mimicit

Issue - State: closed - Opened by Li-Qingyun about 1 year ago

#325 - Link to download the old Otter-HD models

Issue - State: closed - Opened by codonna9 about 1 year ago - 2 comments

#324 - otter-9b-hf disappeared, where to get otter support.

Issue - State: closed - Opened by Li-Qingyun about 1 year ago - 4 comments

#323 - `gradio_web_server` is outdated

Issue - State: closed - Opened by StrangeTcy about 1 year ago - 1 comment

#322 - Import error with otter_ai in demo script

Issue - State: closed - Opened by Andrew-Zhang about 1 year ago - 1 comment

#321 - Recommended next steps

Issue - State: closed - Opened by StrangeTcy about 1 year ago - 5 comments

#320 - Problem with `La.parquet` size

Issue - State: closed - Opened by StrangeTcy about 1 year ago - 3 comments

#319 - GPUs not being recognised properly

Issue - State: closed - Opened by StrangeTcy about 1 year ago - 2 comments

#318 - mismatch the size of datasets

Issue - State: closed - Opened by LinB203 about 1 year ago - 24 comments

#317 - Pufanyi/clip vision model

Pull Request - State: closed - Opened by pufanyi about 1 year ago

#316 - update no split modules

Pull Request - State: closed - Opened by pufanyi about 1 year ago

#315 - Multiple issues with OtterHD

Issue - State: closed - Opened by StrangeTcy about 1 year ago - 2 comments

#314 - hello，The fine-tuning of the OtterHD-8B model requires at least how much GPU resources?

Issue - State: closed - Opened by dgo2dance about 1 year ago - 1 comment

#313 - Question about the pre-training process

Issue - State: closed - Opened by yuezewang about 1 year ago - 4 comments

#312 - Partial Model Weights Saved During Multi-GPU Training

Issue - State: closed - Opened by superjamessyx about 1 year ago - 3 comments

#311 - Where's the data at for the shared script for OtterHD?

Issue - State: closed - Opened by StrangeTcy about 1 year ago - 7 comments

#310 - A question about the resizing operation

Issue - State: open - Opened by Richar-Du about 1 year ago - 1 comment

#309 - [model] Upgrade LiteLLM Dependencies

Pull Request - State: closed - Opened by pufanyi over 1 year ago
Labels: area:model

#308 - ERROR: Cannot install otter-ai and otter-ai==0.0.0a4 because these package versions have conflicting dependencies.

Issue - State: closed - Opened by githubwqj2 over 1 year ago

#307 - about the specific prompt strategies from llava-1.5 (otter-hd-8b)

Issue - State: closed - Opened by peiliu0408 over 1 year ago - 2 comments

#306 - About choosing dataset format and pre-training weights

Issue - State: open - Opened by xmc-andy over 1 year ago - 2 comments

#305 - how to generate labels for FUYU

Issue - State: open - Opened by aamir-gmail over 1 year ago - 1 comment

#304 - Gradient accumulation appears to skip data

Issue - State: closed - Opened by RefractAI over 1 year ago - 4 comments

#303 - ETA on OtterHD-8B weights

Issue - State: closed - Opened by somepago over 1 year ago - 3 comments

#302 - pre-trained weights

Issue - State: closed - Opened by hxan over 1 year ago - 2 comments

#301 - Checkpoint update?

Issue - State: open - Opened by xmc-andy over 1 year ago - 2 comments

#300 - the error loading large parquet file

Issue - State: open - Opened by peiliu0408 over 1 year ago - 13 comments

#299 - ModuleNotFoundError: No module named 'pipeline.constants'

Issue - State: open - Opened by twmht over 1 year ago - 2 comments

#298 - Update README.md

Pull Request - State: open - Opened by eltociear over 1 year ago

#297 - Creating custom use cases based on specific images

Issue - State: open - Opened by smtabatabaie over 1 year ago - 1 comment

#296 - [dataset] How to prepare OtterHD datasets?

Issue - State: closed - Opened by yxchng over 1 year ago - 4 comments

#295 - Questions about processing multiple images in the form of context and video frames🤔?

Issue - State: closed - Opened by xmc-andy over 1 year ago

#294 - if there is any data augmentation operation in the code？

Issue - State: closed - Opened by xmc-andy over 1 year ago - 3 comments

#293 - [Code Refactor] Large Updates Since Aug

Pull Request - State: closed - Opened by Luodian over 1 year ago

#292 - [Code Refactor] Large Updates Since Aug

Pull Request - State: closed - Opened by Luodian over 1 year ago

#291 - When loading the DC.json dataset, I encountered the 'ijson.common.IncompleteJSONError: parse error: premature EOF' error.

Issue - State: closed - Opened by hcwei13 over 1 year ago - 1 comment

#290 - Coco Captioning Performance

Issue - State: open - Opened by essamsleiman over 1 year ago - 1 comment

#289 - HI, I'm evaluating Otter in Coco Image captioning and am getting quite poor performance for 0-shot vs 2-shot or 4-shot.

Issue - State: closed - Opened by essamsleiman over 1 year ago

#288 - [model] change torch requirements to 2.0.1

Pull Request - State: closed - Opened by pufanyi over 1 year ago
Labels: area:model

#287 - [Evaluation]Evaluation with Otter pre-trained model on COCO dataset return 0 CIDER score

Issue - State: open - Opened by Thedatababbler over 1 year ago - 4 comments

#286 - Update LICENSE

Pull Request - State: closed - Opened by cliangyu over 1 year ago

#285 - How to enable Lora Training?

Issue - State: open - Opened by ElegantLin over 1 year ago

#284 - How to train the whole MIMIC-IT dataset?

Issue - State: closed - Opened by ElegantLin over 1 year ago - 2 comments

#283 - How to evaluate the image difference description?

Issue - State: open - Opened by ElegantLin over 1 year ago - 4 comments

#282 - Wrong type hint

Issue - State: closed - Opened by SCZwangxiao over 1 year ago - 1 comment

#281 - Image processor in mimic dataset

Issue - State: open - Opened by baibizhe over 1 year ago
Labels: area:train

#280 - Delete data

Pull Request - State: closed - Opened by kassy11 over 1 year ago

#279 - Question about choosing multi-image input mode and replacing image decoder

Issue - State: open - Opened by charlierabea over 1 year ago - 3 comments
Labels: area:dataset

#278 - having import issues

Issue - State: open - Opened by natashamaniarnew over 1 year ago - 1 comment

#277 - how syphus for ego4d get object_description field?

Issue - State: closed - Opened by Maxlinn over 1 year ago - 2 comments

#276 - [model]Understanding video with images as in-context

Issue - State: open - Opened by kassy11 over 1 year ago - 1 comment
Labels: area:model

#275 - batch size and gpu settings?

Issue - State: open - Opened by yichaoshen-MS over 1 year ago

#274 - Is it correct to set up fsdp for a machine (V100) that does not support bf16?

Issue - State: open - Opened by xmc-andy over 1 year ago - 6 comments

#273 - How to configure acceleration so that main thread memory usage does not exceed the range (make full use of other graphics cards)

Issue - State: closed - Opened by xmc-andy over 1 year ago - 1 comment

#272 - [model] remove horovod dependency

Pull Request - State: closed - Opened by pufanyi over 1 year ago
Labels: area:model

#271 - Question about the evaluation of classification tasks

Issue - State: open - Opened by gray311 over 1 year ago

#270 - Some weights of OtterForConditionalGeneration were not initialized from the model

Issue - State: open - Opened by xmc-andy over 1 year ago - 23 comments

#269 - Recommended Batch Size for Fine-tuning

Issue - State: closed - Opened by da-yama82 over 1 year ago
Labels: area:train

#268 - Improving LiteLLM?

Issue - State: open - Opened by krrishdholakia over 1 year ago - 4 comments
Labels: area:model

#267 - [model] Rebasing the otter structures, make it able to be packed as a pip package

Pull Request - State: closed - Opened by pufanyi over 1 year ago
Labels: area:model

#266 - Merge main to dev_pip_package

Pull Request - State: closed - Opened by pufanyi over 1 year ago

#265 - Jupyter test funqa

Pull Request - State: closed - Opened by Wangmerlyn over 1 year ago

#264 - [training] Change current data related args to a yaml file

Pull Request - State: closed - Opened by pufanyi over 1 year ago
Labels: area:train

#263 - Gradio

Issue - State: open - Opened by WangQvQ over 1 year ago - 2 comments
Labels: area:demo

#262 - base64 image encoding and visual model differences for training and inference （dataset && evaluation）

Issue - State: closed - Opened by xmc-andy over 1 year ago

#261 - Update README.md

Pull Request - State: closed - Opened by ZhangYuanhan-AI over 1 year ago

#260 - Add LiteLLM instructions on docs

Pull Request - State: closed - Opened by ishaan-jaff over 1 year ago - 1 comment

#259 - [Doc] add requirement

Pull Request - State: closed - Opened by Luodian over 1 year ago

#258 - [Model] Fix bugs and improve code logic/performance

Pull Request - State: closed - Opened by Luodian over 1 year ago

#256 - Load pretrained weight error

Issue - State: open - Opened by baibizhe over 1 year ago - 13 comments
Labels: area:model

#254 - which dataset are used in train image-text model?

Issue - State: open - Opened by wjfwjfwjf over 1 year ago - 5 comments
Labels: area:train

#248 - Hope there will be instructions about downloading from the HuggingFace Dataset

Issue - State: open - Opened by tingxueronghua over 1 year ago - 2 comments
Labels: area:dataset

#244 - [model] Add support for Llama2, Palm, Cohere, Replicate Models - using litellm

Pull Request - State: closed - Opened by ishaan-jaff over 1 year ago - 3 comments
Labels: branch:under review, area:dataset

#234 - [train] Single- or multi-round multi-image training

Issue - State: open - Opened by codybum over 1 year ago - 7 comments
Labels: area:train, area:model

#182 - Can I use a single gpu to train this model?

Issue - State: open - Opened by ElegantLin over 1 year ago - 9 comments
Labels: area:train

#179 - [Fix/Train] `image_processor` is missing in `get_mimicit_dataset`

Issue - State: open - Opened by ElegantLin over 1 year ago - 6 comments
Labels: area:train

#100 - xformers implementation in CLIP & LlaMA

Pull Request - State: closed - Opened by king159 almost 2 years ago

#99 - how to train the model on 24g ram gpu with deepspeed

Issue - State: closed - Opened by haodongze almost 2 years ago - 1 comment

GitHub / EvolvingLMMs-Lab/Otter issues and pull requests