facebookresearch/multimodal issues and pull requests

#527 - set bias=True for linear layer (#526)

Pull Request - State: closed - Opened by zhangtemplar 3 months ago - 4 comments
Labels: CLA Signed, fb-exported, Merged

#526 - unify how to freeze some parameters for coca pre-training

Pull Request - State: closed - Opened by zhangtemplar 4 months ago - 8 comments
Labels: CLA Signed, fb-exported

#525 - Cannot fine-tune CLIP model in GPU

Issue - State: closed - Opened by LioYao 4 months ago - 5 comments

#524 - update mypy version

Pull Request - State: closed - Opened by ebsmothers 4 months ago - 3 comments
Labels: CLA Signed, Merged

#523 - Fix docstring of CLIP model

Pull Request - State: closed - Opened by willyfh 4 months ago - 3 comments
Labels: CLA Signed, Merged

#522 - Replace torch.jit.instance with instance to assist torch.compile

Pull Request - State: closed - Opened by Bellaktris 4 months ago - 4 comments
Labels: CLA Signed, fb-exported, Merged

#521 - create configuration file for MaMMUT training

Pull Request - State: closed - Opened by zhangtemplar 5 months ago - 18 comments
Labels: CLA Signed, fb-exported, Merged

#520 - implement MaMMUT

Pull Request - State: closed - Opened by zhangtemplar 5 months ago - 10 comments
Labels: CLA Signed, fb-exported, Merged

#519 - Add optional field for multimodal pooled embeddings

Pull Request - State: closed - Opened by zhangtemplar 5 months ago - 5 comments
Labels: CLA Signed, fb-exported, Merged

#518 - Fixes for CoCa cascaded attention poolers

Pull Request - State: closed - Opened by ebsmothers 6 months ago - 3 comments
Labels: CLA Signed, Merged

#517 - CoCa model implementation

Issue - State: open - Opened by seungkyuK 6 months ago - 1 comment

#516 - Allow non-zero text pad token for Clip text transform

Pull Request - State: closed - Opened by ktnag 7 months ago - 3 comments
Labels: CLA Signed, fb-exported, Merged

#515 - mini-imageNet

Issue - State: open - Opened by HeYiyang2 7 months ago - 1 comment

#514 - add another shape comment

Pull Request - State: closed - Opened by ebsmothers 7 months ago - 1 comment
Labels: CLA Signed

#513 - add shape comment

Pull Request - State: closed - Opened by ebsmothers 7 months ago - 1 comment
Labels: CLA Signed

#512 - update readme

Pull Request - State: closed - Opened by ebsmothers 8 months ago - 5 comments
Labels: CLA Signed, Merged

#511 - Correcting cls token issues and enabling unified ViT-MAE

Pull Request - State: closed - Opened by ebsmothers 8 months ago - 2 comments
Labels: CLA Signed, fb-exported

#510 - Change default logit scale in contrastive loss with temperature from parameter to float

Pull Request - State: closed - Opened by ebsmothers 8 months ago - 4 comments
Labels: CLA Signed, Merged

#509 - Fix contrastive loss with temperature test

Pull Request - State: closed - Opened by ebsmothers 8 months ago - 4 comments
Labels: CLA Signed, Merged

#508 - Fix mypy errors in diffusion labs/models/vae

Pull Request - State: closed - Opened by ebsmothers 8 months ago - 3 comments
Labels: CLA Signed, Merged

#507 - Script for comparison of CoCa zero-shot to open_clip implementation

Pull Request - State: open - Opened by ebsmothers 8 months ago - 2 comments
Labels: CLA Signed

#506 - Add CoCa model

Pull Request - State: closed - Opened by ebsmothers 8 months ago - 4 comments
Labels: CLA Signed, Merged

#505 - Train diffusion on MNIST

Issue - State: closed - Opened by sudhir2016 8 months ago - 2 comments

#504 - Remaining fixes to unblock nightly build

Pull Request - State: closed - Opened by ebsmothers 8 months ago - 4 comments
Labels: CLA Signed, Merged

#503 - test commit for nightly build w/o torchtext

Pull Request - State: closed - Opened by ebsmothers 8 months ago
Labels: CLA Signed

#502 - fix blip2 gpu test

Pull Request - State: closed - Opened by pikapecan 8 months ago - 2 comments
Labels: CLA Signed, fb-exported, Merged

#501 - Drop torchtext dependency in torchmultimodal/

Pull Request - State: closed - Opened by ebsmothers 8 months ago - 3 comments
Labels: CLA Signed, Merged

#500 - Drop torchtext dep in OSS CLIP transform

Pull Request - State: closed - Opened by ebsmothers 8 months ago - 8 comments
Labels: CLA Signed, fb-exported, Merged

#499 - Move BERTTextTransform into MUGEN example

Pull Request - State: closed - Opened by ebsmothers 8 months ago - 4 comments
Labels: CLA Signed, Merged

#498 - Relax atol and rtol for MUGEN video VQVAE unit test

Pull Request - State: closed - Opened by ebsmothers 8 months ago - 4 comments
Labels: CLA Signed, Merged

#497 - Utilities for deterministic unit tests

Pull Request - State: closed - Opened by ebsmothers 8 months ago - 4 comments
Labels: CLA Signed, fb-exported, Merged

#496 - Add code for LDM

Pull Request - State: open - Opened by abhinavarora 8 months ago - 2 comments
Labels: CLA Signed, fb-exported

#495 - Added readme and example notebook for diffusion_labs

Pull Request - State: closed - Opened by pbontrager 9 months ago - 3 comments
Labels: CLA Signed, Merged

#494 - [bugfix] Fix missing self in res/attn cond proj definitions

Pull Request - State: closed - Opened by ebsmothers 9 months ago - 4 comments
Labels: CLA Signed, Merged

#493 - Add LDM VAE and its components to diffusion_labs

Pull Request - State: closed - Opened by abhinavarora 9 months ago - 7 comments
Labels: CLA Signed, fb-exported, Merged

#492 - [just testing] Move off of nightly releases

Pull Request - State: closed - Opened by ebsmothers 9 months ago - 1 comment
Labels: CLA Signed

#491 - Update and Add Diffusion Components

Pull Request - State: closed - Opened by pbontrager 9 months ago - 6 comments
Labels: CLA Signed, fb-exported, Merged

#490 - [diffusion_labs] Move attention and res blocks under modules

Pull Request - State: closed - Opened by ebsmothers 9 months ago - 9 comments
Labels: CLA Signed

#489 - ADMStack and ADMResBlock changes

Pull Request - State: closed - Opened by ebsmothers 9 months ago - 6 comments
Labels: CLA Signed, Merged

#488 - Refactor ADM classes

Pull Request - State: closed - Opened by ebsmothers 9 months ago - 6 comments
Labels: CLA Signed, Merged

#487 - Port BLIP-2 entire model to OSS folder

Pull Request - State: closed - Opened by pikapecan 9 months ago - 2 comments
Labels: CLA Signed, fb-exported, Merged

#486 - add qformer model to torchmm/models

Pull Request - State: open - Opened by pikapecan 9 months ago - 7 comments
Labels: CLA Signed, fb-exported

#485 - add blip2 loss under torchmultimodal/modules/losses

Pull Request - State: open - Opened by pikapecan 9 months ago - 10 comments
Labels: CLA Signed, fb-exported

#484 - add blip2 layer under torchmm/models

Pull Request - State: open - Opened by pikapecan 9 months ago - 9 comments
Labels: CLA Signed, fb-exported

#483 - create blip2 folder and add qformer layers under torchmm/models

Pull Request - State: open - Opened by pikapecan 9 months ago - 12 comments
Labels: CLA Signed, fb-exported

#482 - Add support for LLaVA model

Issue - State: open - Opened by youssefadr 9 months ago - 7 comments

#481 - Update linter versions

Pull Request - State: closed - Opened by ebsmothers 9 months ago - 2 comments
Labels: CLA Signed, Merged

#480 - Reorg OSS Diffusion Components to diffusion_labs folder

Pull Request - State: closed - Opened by pbontrager 9 months ago - 5 comments
Labels: CLA Signed, fb-exported

#479 - Add pretrained MAE weights, option to load checkpoints in ViT builder

Pull Request - State: closed - Opened by ebsmothers 9 months ago - 3 comments
Labels: CLA Signed, Merged

#478 - Consolidate VideoGPT components

Pull Request - State: closed - Opened by kartikayk 9 months ago - 4 comments
Labels: CLA Signed, fb-exported, Merged

#477 - add AnyPrecision optimizer

Pull Request - State: closed - Opened by lessw2020 9 months ago - 3 comments
Labels: CLA Signed, Merged

#476 - Increase tolerance for audio MAE test

Pull Request - State: closed - Opened by ebsmothers 9 months ago - 3 comments
Labels: CLA Signed, Merged

#475 - add triton based Softmax and unit tests

Pull Request - State: open - Opened by lessw2020 9 months ago
Labels: CLA Signed

#474 - Rename patch_embed and transformer_encoder fields in MAE model

Pull Request - State: closed - Opened by ebsmothers 9 months ago - 3 comments
Labels: CLA Signed, fb-exported, Merged

#473 - Rename image embeddings -> patch embeddings

Pull Request - State: closed - Opened by ebsmothers 9 months ago - 2 comments
Labels: CLA Signed, fb-exported, Merged

#472 - fix MHA dropout under eval mode

Pull Request - State: closed - Opened by pikapecan 9 months ago - 8 comments
Labels: CLA Signed, fb-exported, Merged

#471 - Add init.py to MAE directory

Pull Request - State: closed - Opened by ebsmothers 10 months ago - 3 comments
Labels: CLA Signed, Merged

#470 - Add init.py to MAE directory

Pull Request - State: closed - Opened by ebsmothers 10 months ago
Labels: CLA Signed

#469 - Add enum for backprop type in all-gather utility

Pull Request - State: closed - Opened by ebsmothers 10 months ago - 3 comments
Labels: CLA Signed, fb-exported, Merged

#468 - Add test util to split tensor across devices for distributed tests

Pull Request - State: closed - Opened by ebsmothers 10 months ago - 2 comments
Labels: CLA Signed, fb-exported, Merged

#467 - add new flag to gather_tensor for cases we need to fully turn off backprop

Pull Request - State: closed - Opened by pikapecan 10 months ago - 2 comments
Labels: CLA Signed, fb-exported, Merged

#466 - Add back the mae tests

Pull Request - State: closed - Opened by ankitade 10 months ago - 3 comments
Labels: CLA Signed, Merged

#465 - add SimpleRMSNorm

Pull Request - State: closed - Opened by lessw2020 10 months ago - 3 comments
Labels: CLA Signed, Merged

#464 - Add mae transforms

Pull Request - State: closed - Opened by ankitade 10 months ago - 4 comments
Labels: CLA Signed, fb-exported, Merged

#463 - Add reconstruction loss

Pull Request - State: open - Opened by ankitade 10 months ago - 1 comment
Labels: CLA Signed

#462 - add Alibi positional embeddings

Pull Request - State: open - Opened by lessw2020 10 months ago - 4 comments
Labels: CLA Signed

#461 - blip2 stage1 losses

Pull Request - State: closed - Opened by pikapecan 10 months ago - 4 comments
Labels: CLA Signed, fb-exported, Merged

#460 - add tensor_hash

Pull Request - State: closed - Opened by lessw2020 10 months ago - 5 comments
Labels: CLA Signed, Merged

#459 - Add mae components and model

Pull Request - State: closed - Opened by ankitade 10 months ago - 3 comments
Labels: CLA Signed, Merged

#458 - Add vision transformer

Pull Request - State: closed - Opened by ankitade 10 months ago - 4 comments
Labels: CLA Signed, fb-exported, Merged

#457 - Add Parallel_Attention_Blocks (3 of 3)

Pull Request - State: open - Opened by lessw2020 10 months ago - 1 comment
Labels: CLA Signed

#456 - Add RMSNorm (part of parallel_attn_blocks)

Pull Request - State: closed - Opened by rohan-varma 10 months ago - 4 comments
Labels: CLA Signed, fb-exported, Merged

#455 - Add image embeddings

Pull Request - State: closed - Opened by ankitade 10 months ago - 4 comments
Labels: CLA Signed, Merged

#454 - Return either hidden state or projected embeddings not both

Pull Request - State: closed - Opened by abhinavarora 10 months ago - 4 comments
Labels: CLA Signed, fb-exported, Merged

#453 - Remove strict=True from zip in transformer test

Pull Request - State: closed - Opened by ankitade 10 months ago - 3 comments
Labels: CLA Signed, Merged

#452 - Add patch masking functions

Pull Request - State: closed - Opened by ankitade 10 months ago - 4 comments
Labels: CLA Signed, Merged

#451 - Add transformer decoder layer

Pull Request - State: closed - Opened by ankitade 11 months ago - 3 comments
Labels: CLA Signed, Merged

#450 - Add Rotary Positional Embeddings (RoPE) - part 2 of parallel attention blocks

Pull Request - State: open - Opened by lessw2020 11 months ago - 3 comments
Labels: CLA Signed

#449 - Batch matmul fast path in MHAWithCache

Pull Request - State: open - Opened by rohan-varma 11 months ago - 8 comments
Labels: CLA Signed, fb-exported

#448 - Add RMSNorm (part of parallel_attn_blocks)

Pull Request - State: open - Opened by lessw2020 11 months ago - 2 comments
Labels: CLA Signed

#447 - Add new transformer implementation to oss

Pull Request - State: closed - Opened by ankitade 11 months ago - 3 comments
Labels: CLA Signed, Merged

#446 - add parallel_attention_blocks

Pull Request - State: closed - Opened by lessw2020 11 months ago
Labels: CLA Signed

#445 - Transformer decoder

Pull Request - State: closed - Opened by ebsmothers 11 months ago - 5 comments
Labels: CLA Signed, fb-exported, Merged

#444 - How to perform multimodal multitask instance segmentation in torchmultimodal?

Issue - State: closed - Opened by Nandtiw 11 months ago - 4 comments

#443 - Copy transformer impl to oss folder

Pull Request - State: open - Opened by ankitade 11 months ago - 3 comments
Labels: CLA Signed, fb-exported

#442 - Support returning hidden state in CLIPTextEncoder

Pull Request - State: closed - Opened by 22quinn 11 months ago - 5 comments
Labels: CLA Signed, Merged

#441 - Fix attn_mask shape in MHA docstrings

Pull Request - State: closed - Opened by ebsmothers 11 months ago - 5 comments
Labels: CLA Signed, fb-exported, Merged

#440 - Bump requests from 2.27.1 to 2.31.0 in /examples/flava

Pull Request - State: closed - Opened by dependabot[bot] 11 months ago - 4 comments
Labels: CLA Signed, Merged, dependencies

#439 - Move transformer impl to flava folder

Pull Request - State: closed - Opened by ankitade 11 months ago - 4 comments
Labels: CLA Signed, Merged

#438 - Implement Spatial Transformer used by LDM UNet

Pull Request - State: closed - Opened by abhinavarora 11 months ago - 11 comments
Labels: CLA Signed, fb-exported, Merged

#437 - Move CA to albef dir

Pull Request - State: closed - Opened by ankitade 11 months ago - 3 comments
Labels: CLA Signed, Merged

#436 - Add option to remove bias from q,k,v projections in MHA

Pull Request - State: closed - Opened by abhinavarora 11 months ago - 5 comments
Labels: CLA Signed, fb-exported, Merged

#435 - Bump scipy from 1.8.1 to 1.10.0 in /examples/mdetr

Pull Request - State: closed - Opened by dependabot[bot] 12 months ago - 3 comments
Labels: CLA Signed, Merged, dependencies

#434 - Fix AC order for FLAVA example

Pull Request - State: open - Opened by rohan-varma 12 months ago - 2 comments
Labels: CLA Signed, fb-exported

#433 - Dont use scaler for bf16

Pull Request - State: open - Opened by rohan-varma 12 months ago - 2 comments
Labels: CLA Signed, fb-exported

#432 - Qformer layer impl.

Pull Request - State: closed - Opened by pikapecan about 1 year ago - 6 comments
Labels: CLA Signed, fb-exported, Merged

#431 - Enable PT2 for dalle2 with old FSDP api

Pull Request - State: closed - Opened by Yuzhen11 about 1 year ago - 4 comments
Labels: CLA Signed, fb-exported, Merged

#430 - Update transformers version in examples to 4.30.0

Pull Request - State: closed - Opened by ebsmothers about 1 year ago - 3 comments
Labels: CLA Signed, Merged

#429 - Attention with caching

Pull Request - State: open - Opened by pikapecan about 1 year ago - 2 comments
Labels: CLA Signed, fb-exported

#428 - Add init.py to diffusion folder to make a package

Pull Request - State: closed - Opened by pbontrager about 1 year ago - 3 comments
Labels: CLA Signed, Merged

GitHub / facebookresearch/multimodal issues and pull requests