mlfoundations/open_lm issues and pull requests

#100 - main.py: log number of model parameters

Pull Request - State: closed - Opened by iejMac 12 months ago

#100 - main.py: log number of model parameters

Pull Request - State: closed - Opened by iejMac 12 months ago

#99 - tests: run in GitHub Actions

Pull Request - State: closed - Opened by iejMac 12 months ago - 7 comments

#99 - tests: run in GitHub Actions

Pull Request - State: closed - Opened by iejMac 12 months ago - 7 comments

#98 - Fused implementation of squared loss

Pull Request - State: open - Opened by saurabhgarg1996 12 months ago

#98 - Fused implementation of squared loss

Pull Request - State: open - Opened by saurabhgarg1996 12 months ago

#97 - Shared epoch common for datasets.

Pull Request - State: closed - Opened by GeorgiosSmyrnis 12 months ago

#97 - Shared epoch common for datasets.

Pull Request - State: closed - Opened by GeorgiosSmyrnis 12 months ago

#96 - Add assertion for flags.

Pull Request - State: closed - Opened by GeorgiosSmyrnis 12 months ago

#96 - Add assertion for flags.

Pull Request - State: closed - Opened by GeorgiosSmyrnis 12 months ago

#95 - error checking params.py

Issue - State: closed - Opened by sagadre 12 months ago - 1 comment

#95 - error checking params.py

Issue - State: closed - Opened by sagadre 12 months ago - 1 comment

#94 - lr hotfix

Pull Request - State: closed - Opened by sagadre 12 months ago - 1 comment

#94 - lr hotfix

Pull Request - State: closed - Opened by sagadre 12 months ago - 1 comment

#93 - Dataloading Epoch Update Bug

Issue - State: closed - Opened by sedrick-keh-tri 12 months ago - 2 comments

#93 - Dataloading Epoch Update Bug

Issue - State: closed - Opened by sedrick-keh-tri 12 months ago - 2 comments

#92 - Z loss mean square instead of square mean

Pull Request - State: closed - Opened by jmercat 12 months ago

#92 - Z loss mean square instead of square mean

Pull Request - State: closed - Opened by jmercat 12 months ago

#91 - evaluate: set tokens only when args.train_num_samples is not None

Pull Request - State: closed - Opened by iejMac almost 1 year ago - 3 comments

#91 - evaluate: set tokens only when args.train_num_samples is not None

Pull Request - State: closed - Opened by iejMac almost 1 year ago - 3 comments

#90 - open_lm chronicles

Issue - State: open - Opened by iejMac almost 1 year ago
Labels: enhancement

#90 - open_lm chronicles

Issue - State: open - Opened by iejMac almost 1 year ago
Labels: enhancement

#89 - HF Integration

Issue - State: open - Opened by sedrick-keh-tri almost 1 year ago - 3 comments
Labels: enhancement

#89 - HF Integration

Issue - State: open - Opened by sedrick-keh-tri almost 1 year ago - 3 comments
Labels: enhancement

#88 - Fix S3 streaming issues

Pull Request - State: closed - Opened by shahromil16 almost 1 year ago - 6 comments

#88 - Fix S3 streaming issues

Pull Request - State: closed - Opened by shahromil16 almost 1 year ago - 6 comments

#87 - README update re: dataloader.

Pull Request - State: closed - Opened by GeorgiosSmyrnis about 1 year ago - 4 comments

#87 - README update re: dataloader.

Pull Request - State: closed - Opened by GeorgiosSmyrnis about 1 year ago - 4 comments

#86 - Reintegrate manifest creation.

Pull Request - State: closed - Opened by GeorgiosSmyrnis about 1 year ago - 1 comment

#86 - Reintegrate manifest creation.

Pull Request - State: closed - Opened by GeorgiosSmyrnis about 1 year ago - 1 comment

#85 - Vaishaal/tokenize shuffle generator

Pull Request - State: closed - Opened by Vaishaal about 1 year ago - 4 comments

#85 - Vaishaal/tokenize shuffle generator

Pull Request - State: closed - Opened by Vaishaal about 1 year ago - 4 comments

#84 - tokenize_shuffle.py, re-adding accidentally removed line

Pull Request - State: closed - Opened by jeffreywpli about 1 year ago - 2 comments

#84 - tokenize_shuffle.py, re-adding accidentally removed line

Pull Request - State: closed - Opened by jeffreywpli about 1 year ago - 2 comments

#83 - main.py: divide val_num_samples by seq_len

Pull Request - State: closed - Opened by iejMac about 1 year ago

#83 - main.py: divide val_num_samples by seq_len

Pull Request - State: closed - Opened by iejMac about 1 year ago

#82 - Add automatic download of dataloader test files.

Pull Request - State: closed - Opened by GeorgiosSmyrnis about 1 year ago - 2 comments

#82 - Add automatic download of dataloader test files.

Pull Request - State: closed - Opened by GeorgiosSmyrnis about 1 year ago - 2 comments

#81 - Fix off by one in dataloading

Pull Request - State: closed - Opened by sagadre about 1 year ago

#81 - Fix off by one in dataloading

Pull Request - State: closed - Opened by sagadre about 1 year ago

#80 - Fill-in-the-middle training

Pull Request - State: closed - Opened by sedrick-keh-tri about 1 year ago - 1 comment

#80 - Fill-in-the-middle training

Pull Request - State: closed - Opened by sedrick-keh-tri about 1 year ago - 1 comment

#79 - support individual padding token

Pull Request - State: closed - Opened by sagadre about 1 year ago - 2 comments

#79 - support individual padding token

Pull Request - State: closed - Opened by sagadre about 1 year ago - 2 comments

#78 - 120 line length, add pyproject.toml

Pull Request - State: closed - Opened by achalddave about 1 year ago

#78 - 120 line length, add pyproject.toml

Pull Request - State: closed - Opened by achalddave about 1 year ago

#77 - data.py: only x%vocab_size when x < 0

Pull Request - State: closed - Opened by iejMac about 1 year ago - 1 comment

#77 - data.py: only x%vocab_size when x < 0

Pull Request - State: closed - Opened by iejMac about 1 year ago - 1 comment

#76 - pre-commit hook

Pull Request - State: closed - Opened by sagadre about 1 year ago - 1 comment

#76 - pre-commit hook

Pull Request - State: closed - Opened by sagadre about 1 year ago - 1 comment

#75 - Add params to MockArgs

Pull Request - State: closed - Opened by achalddave about 1 year ago

#75 - Add params to MockArgs

Pull Request - State: closed - Opened by achalddave about 1 year ago

#74 - Implement kv cache

Pull Request - State: closed - Opened by jmercat about 1 year ago - 13 comments

#74 - Implement kv cache

Pull Request - State: closed - Opened by jmercat about 1 year ago - 13 comments

#73 - black reformat everything

Pull Request - State: closed - Opened by Vaishaal about 1 year ago

#73 - black reformat everything

Pull Request - State: closed - Opened by Vaishaal about 1 year ago

#72 - Make torch.compile work with fsdp and xformers

Issue - State: closed - Opened by sagadre about 1 year ago - 6 comments

#72 - Make torch.compile work with fsdp and xformers

Issue - State: closed - Opened by sagadre about 1 year ago - 6 comments

#71 - Minimize how often we load args.resume

Issue - State: open - Opened by achalddave about 1 year ago - 1 comment
Labels: enhancement, good first issue

#71 - Minimize how often we load args.resume

Issue - State: open - Opened by achalddave about 1 year ago - 1 comment
Labels: enhancement, good first issue

#70 - Improve dataloading.

Issue - State: closed - Opened by GeorgiosSmyrnis about 1 year ago - 2 comments
Labels: enhancement

#70 - Improve dataloading.

Issue - State: closed - Opened by GeorgiosSmyrnis about 1 year ago - 2 comments
Labels: enhancement

#69 - Integrate manifest creation

Pull Request - State: closed - Opened by GeorgiosSmyrnis about 1 year ago - 1 comment

#69 - Integrate manifest creation

Pull Request - State: closed - Opened by GeorgiosSmyrnis about 1 year ago - 1 comment

#68 - Fix sample/step counting in the case of uneven epochs.

Pull Request - State: closed - Opened by GeorgiosSmyrnis about 1 year ago - 2 comments

#68 - Fix sample/step counting in the case of uneven epochs.

Pull Request - State: closed - Opened by GeorgiosSmyrnis about 1 year ago - 2 comments

#67 - Check if batch size changed when resuming.

Issue - State: closed - Opened by GeorgiosSmyrnis about 1 year ago - 1 comment
Labels: enhancement

#67 - Check if batch size changed when resuming.

Issue - State: closed - Opened by GeorgiosSmyrnis about 1 year ago - 1 comment
Labels: enhancement

#66 - ray tokenize: support tar file input format and pre-tokenized data

Pull Request - State: closed - Opened by iejMac about 1 year ago - 6 comments

#66 - ray tokenize: support tar file input format and pre-tokenized data

Pull Request - State: closed - Opened by iejMac about 1 year ago - 6 comments

#65 - Wrong token count when using --accurate-total-tokens.

Issue - State: closed - Opened by GeorgiosSmyrnis about 1 year ago

#65 - Wrong token count when using --accurate-total-tokens.

Issue - State: closed - Opened by GeorgiosSmyrnis about 1 year ago

#64 - support for training AutoModelForCausalLM directly

Pull Request - State: closed - Opened by sagadre about 1 year ago - 2 comments

#64 - support for training AutoModelForCausalLM directly

Pull Request - State: closed - Opened by sagadre about 1 year ago - 2 comments

#63 - Fix naming inconsistencies.

Pull Request - State: closed - Opened by GeorgiosSmyrnis about 1 year ago

#63 - Fix naming inconsistencies.

Pull Request - State: closed - Opened by GeorgiosSmyrnis about 1 year ago

#62 - Use s3 cp for file load

Pull Request - State: closed - Opened by achalddave about 1 year ago - 5 comments

#62 - Use s3 cp for file load

Pull Request - State: closed - Opened by achalddave about 1 year ago - 5 comments

#61 - Target masking left of special tok

Pull Request - State: closed - Opened by sagadre about 1 year ago - 1 comment

#61 - Target masking left of special tok

Pull Request - State: closed - Opened by sagadre about 1 year ago - 1 comment

#60 - Support no_sync when using grad accumulation

Pull Request - State: closed - Opened by achalddave about 1 year ago

#60 - Support no_sync when using grad accumulation

Pull Request - State: closed - Opened by achalddave about 1 year ago

#59 - Load LLaMA2 from HF checkpoint

Pull Request - State: closed - Opened by sedrick-keh-tri about 1 year ago - 2 comments

#59 - Load LLaMA2 from HF checkpoint

Pull Request - State: closed - Opened by sedrick-keh-tri about 1 year ago - 2 comments

#58 - added 41m model config

Pull Request - State: closed - Opened by reinhardh about 1 year ago

#58 - added 41m model config

Pull Request - State: closed - Opened by reinhardh about 1 year ago

#57 - standardize tokenization and basic dataloader test

Pull Request - State: closed - Opened by sagadre about 1 year ago - 2 comments

#57 - standardize tokenization and basic dataloader test

Pull Request - State: closed - Opened by sagadre about 1 year ago - 2 comments

#56 - Standardize tokenization for json and txt datasets

Issue - State: closed - Opened by sagadre about 1 year ago

#56 - Standardize tokenization for json and txt datasets

Issue - State: closed - Opened by sagadre about 1 year ago

#55 - Tokenization on-the-fly without slowdown

Issue - State: open - Opened by sagadre about 1 year ago

#55 - Tokenization on-the-fly without slowdown

Issue - State: open - Opened by sagadre about 1 year ago

#54 - Update hf_config and hf_model to support HELM eval

Pull Request - State: closed - Opened by ruixin31 about 1 year ago - 3 comments

#54 - Update hf_config and hf_model to support HELM eval

Pull Request - State: closed - Opened by ruixin31 about 1 year ago - 3 comments

#53 - filter_lt_seqlen warning

Pull Request - State: closed - Opened by sagadre about 1 year ago

#53 - filter_lt_seqlen warning

Pull Request - State: closed - Opened by sagadre about 1 year ago

#52 - llama2 unit tests

Issue - State: open - Opened by sagadre about 1 year ago
Labels: good first issue

#52 - llama2 unit tests

Issue - State: open - Opened by sagadre about 1 year ago
Labels: good first issue

#51 - Use inference_mode in evaluate

Pull Request - State: closed - Opened by PythonNut about 1 year ago

#51 - Use inference_mode in evaluate

Pull Request - State: closed - Opened by PythonNut about 1 year ago

GitHub / mlfoundations/open_lm issues and pull requests