Ecosyste.ms: Issues

An open API service for providing issue and pull request metadata for open source projects.

GitHub / allenai/OLMo issues and pull requests

#735 - fixed up changelog

Pull Request - State: closed - Opened by revbucket 25 days ago

#733 - Update version.py

Pull Request - State: closed - Opened by revbucket 25 days ago

#732 - OLMo Checkpoints Website Down?

Issue - State: open - Opened by jhsansom 25 days ago
Labels: type/bug

#730 - Add regression tests for training

Pull Request - State: open - Opened by 2015aroras about 1 month ago - 1 comment

#729 - I added some script to help people set up the env on vista

Pull Request - State: closed - Opened by leo-liuzy about 1 month ago

#728 - Getting training data by sources

Issue - State: open - Opened by chawins about 1 month ago
Labels: type/question

#727 - Compile support for peteish13

Pull Request - State: closed - Opened by dirkgr about 1 month ago

#726 - Missing OLMo checkpoints

Issue - State: open - Opened by mirandrom about 1 month ago - 1 comment

#725 - Fix build errors

Pull Request - State: closed - Opened by 2015aroras about 1 month ago

#724 - Update LUMI scripts

Pull Request - State: closed - Opened by 2015aroras about 1 month ago

#723 - docker

Issue - State: open - Opened by jacky080808 about 1 month ago
Labels: type/question

#721 - Bump torch version

Pull Request - State: closed - Opened by vwxyzjn about 2 months ago - 1 comment

#706 - OLMoThreadError: generator thread data thread 0 failed

Issue - State: open - Opened by ybdesire 3 months ago - 1 comment
Labels: type/question

#687 - Extend functionality of Wandb Config Diff script

Pull Request - State: open - Opened by kyleclo 3 months ago

#682 - DNM: Loss issue checkpoint with refine1b setups

Pull Request - State: open - Opened by undfined 3 months ago

#678 - Initial Loss increased from 10 (0.3.0 v) to 60 (0.4.0) !

Issue - State: open - Opened by Xuekai-Zhu 3 months ago - 9 comments
Labels: type/bug

#677 - Ladder 1xC

Pull Request - State: open - Opened by AkshitaB 4 months ago

#675 - Alternative evals

Pull Request - State: open - Opened by AkshitaB 4 months ago

#639 - MoE

Pull Request - State: open - Opened by Muennighoff 4 months ago - 3 comments

#549 - Gradient Checkpointing

Issue - State: closed - Opened by fakerybakery 7 months ago - 3 comments
Labels: type/feature

#500 - Make R2 intermediate checkpoints of official runs easy to access

Pull Request - State: closed - Opened by 2015aroras 8 months ago - 7 comments

#100 - Url deduper

Pull Request - State: closed - Opened by rodneykinney over 1 year ago

#99 - Minor additions to LUMI doc

Pull Request - State: closed - Opened by epwalsh over 1 year ago

#98 - Makes torch compilation work

Pull Request - State: closed - Opened by dirkgr over 1 year ago

#97 - Bring back 6 CPUs per GPU

Pull Request - State: closed - Opened by dirkgr over 1 year ago

#96 - Ensure tokenizer is thread safe

Pull Request - State: closed - Opened by epwalsh over 1 year ago

#95 - Ensuring Data Order Tracking for Reproducibility

Issue - State: closed - Opened by IanMagnusson over 1 year ago - 14 comments
Labels: type/feature

#94 - Allow multiple document roots per stream

Pull Request - State: closed - Opened by rodneykinney over 1 year ago

#93 - avoid buffers

Pull Request - State: closed - Opened by epwalsh over 1 year ago - 1 comment

#92 - Computing stats for the stack

Pull Request - State: closed - Opened by AkshitaB over 1 year ago - 1 comment

#91 - Update dataset statistics

Pull Request - State: closed - Opened by AkshitaB over 1 year ago

#90 - Some settings from the AMD branch

Pull Request - State: closed - Opened by dirkgr over 1 year ago - 3 comments

#89 - Freeze Merger Requirements

Pull Request - State: closed - Opened by rodneykinney over 1 year ago

#88 - Update mypy requirement from <1.2,>=1.0 to >=1.0,<1.3

Pull Request - State: closed - Opened by dependabot[bot] over 1 year ago - 1 comment
Labels: dependencies/python

#87 - Save config at start of run

Pull Request - State: closed - Opened by epwalsh over 1 year ago - 1 comment

#86 - rename DOLMA -> OLMo

Pull Request - State: closed - Opened by epwalsh over 1 year ago

#85 - Upload artifact

Pull Request - State: closed - Opened by dirkgr over 1 year ago - 2 comments

#84 - Merger fixes

Pull Request - State: closed - Opened by rodneykinney over 1 year ago

#83 - Add a beam search implementation and a `.generate()` method to the model

Pull Request - State: closed - Opened by epwalsh over 1 year ago - 3 comments

#82 - Update README.md

Pull Request - State: closed - Opened by soldni over 1 year ago

#80 - Add option back for default "sequential" transformer block

Pull Request - State: closed - Opened by epwalsh over 1 year ago - 1 comment
Labels: project/model

#79 - Try fusing output projections in the block

Pull Request - State: closed - Opened by epwalsh over 1 year ago - 5 comments
Labels: project/model

#78 - Rename "DOLMA" to "OLMo"

Issue - State: closed - Opened by epwalsh over 1 year ago
Labels: project/model, severity/must, status/blocked

#77 - Starting a llm-eval branch

Pull Request - State: open - Opened by IanMagnusson over 1 year ago

#76 - Modified S2 format to conform to pretrain data spec

Pull Request - State: closed - Opened by soldni over 1 year ago

#75 - Fuse attention and feed-forward projections

Pull Request - State: closed - Opened by epwalsh over 1 year ago - 4 comments

#74 - Merger

Pull Request - State: closed - Opened by rodneykinney over 1 year ago - 2 comments

#73 - Add `generate` method to our model implementation

Issue - State: closed - Opened by soldni over 1 year ago - 2 comments
Labels: project/model, severity/must, type/feature

#72 - Quick script to verify data format works for all the sources

Pull Request - State: closed - Opened by kyleclo over 1 year ago

#71 - Adding `v0` and `v1` of Wikipedia

Pull Request - State: closed - Opened by soldni over 1 year ago

#70 - Investigate RMSNorm as an alternative to LayerNorm

Issue - State: closed - Opened by epwalsh over 1 year ago - 11 comments
Labels: project/model, severity/should, difficulty/medium, type/feature

#69 - Integrate latest throughput improvements from Mosaic

Issue - State: closed - Opened by epwalsh over 1 year ago - 2 comments
Labels: project/model, severity/must, difficulty/medium

#68 - CC Notes

Pull Request - State: closed - Opened by rodneykinney over 1 year ago

#67 - Deduped version of S2 Corpus

Pull Request - State: closed - Opened by soldni over 1 year ago

#66 - Integrate down-stream evaluation code

Issue - State: closed - Opened by yakazimir over 1 year ago - 1 comment
Labels: type/feature

#65 - CC notes

Pull Request - State: closed - Opened by rodneykinney over 1 year ago

#64 - Does using `Dropout` layers, even if the probability is 0, have a performance penalty?

Issue - State: closed - Opened by epwalsh over 1 year ago - 1 comment
Labels: project/model, type/question

#63 - Implement Rotary Positional Embedding (RoPE)

Issue - State: closed - Opened by epwalsh over 1 year ago - 1 comment
Labels: project/model, severity/should, difficulty/easy

#62 - Version 2 of the S2 dataset.

Pull Request - State: closed - Opened by soldni over 1 year ago

#61 - Torch 2.0

Pull Request - State: closed - Opened by dirkgr over 1 year ago - 6 comments
Labels: project/model, status/blocked

#60 - Adds a config for a 7B model

Pull Request - State: closed - Opened by dirkgr over 1 year ago - 2 comments

#59 - Checkpoint saving code needs to never delete or overwrite certain checkpoints

Issue - State: closed - Opened by dirkgr over 1 year ago - 3 comments
Labels: project/model, severity/must, difficulty/medium

#58 - Add docs and make target for Beaker IB cluster

Pull Request - State: closed - Opened by mewil over 1 year ago

#57 - Logging to the cloud

Pull Request - State: closed - Opened by dirkgr over 1 year ago - 4 comments

#56 - Add (decoupled) Lion optimizer

Pull Request - State: closed - Opened by epwalsh over 1 year ago

#55 - Use composer's built-in speed monitor

Pull Request - State: closed - Opened by epwalsh over 1 year ago - 1 comment

#54 - Replace our speed monitor with composer's built in

Issue - State: closed - Opened by epwalsh over 1 year ago
Labels: project/model, severity/should, difficulty/easy

#53 - Log into some online logging service

Issue - State: closed - Opened by dirkgr over 1 year ago - 1 comment
Labels: project/model, severity/should

#52 - Get GPU metrics into wandb

Issue - State: closed - Opened by dirkgr over 1 year ago - 4 comments
Labels: project/model, severity/should, status/blocked

#51 - Find out what running with a profiler is like

Issue - State: closed - Opened by dirkgr over 1 year ago - 1 comment
Labels: project/model, severity/must, difficulty/medium

#50 - Add a 7B config

Issue - State: closed - Opened by dirkgr over 1 year ago
Labels: project/model, severity/must, difficulty/easy

#49 - Try bf16 on AMD hardware

Issue - State: closed - Opened by dirkgr over 1 year ago - 2 comments
Labels: project/model, severity/must, status/blocked, difficulty/easy

#48 - Try torch 2.0

Issue - State: closed - Opened by dirkgr over 1 year ago
Labels: project/model, difficulty/medium

#47 - Run different, and bigger

Pull Request - State: closed - Opened by dirkgr over 1 year ago

#46 - Add decoupled AdamW, use by default

Pull Request - State: closed - Opened by epwalsh over 1 year ago

#45 - Add (decoupled) LION optimizer

Issue - State: closed - Opened by epwalsh over 1 year ago
Labels: project/model, severity/should, difficulty/easy

#44 - add config section for speed monitor

Pull Request - State: closed - Opened by epwalsh over 1 year ago - 1 comment

#43 - add a 70b config

Pull Request - State: closed - Opened by epwalsh over 1 year ago - 1 comment

#42 - Get rid of some hardcoded paths

Pull Request - State: closed - Opened by dirkgr over 1 year ago - 1 comment

#41 - merge lumi and cirrascale configs

Pull Request - State: closed - Opened by epwalsh over 1 year ago - 1 comment

#40 - add more docs about data specification

Pull Request - State: closed - Opened by kyleclo over 1 year ago - 1 comment

#39 - Populate W&B config

Pull Request - State: closed - Opened by epwalsh over 1 year ago

#38 - Try PyTorch FSDP "HYBRID_SHARD" strategy

Issue - State: closed - Opened by epwalsh over 1 year ago - 1 comment
Labels: project/model, severity/should, status/blocked, difficulty/easy

#37 - Lumi

Pull Request - State: closed - Opened by dirkgr over 1 year ago

#36 - Update NOTES.md

Pull Request - State: closed - Opened by AkshitaB over 1 year ago

#35 - Update NOTES.md

Pull Request - State: closed - Opened by rodneykinney over 1 year ago - 1 comment

#34 - Added v1 of S2ORC provided to MosaicML

Pull Request - State: closed - Opened by soldni over 1 year ago

#33 - Add eval loop to training script

Issue - State: closed - Opened by epwalsh over 1 year ago - 2 comments
Labels: project/model, severity/must, difficulty/easy

#32 - Run Stas' bandwidth testing code on LUMI

Issue - State: closed - Opened by dirkgr over 1 year ago - 2 comments
Labels: project/model, severity/should, difficulty/medium

#31 - Minor fixes, add a small 300m param training config

Pull Request - State: closed - Opened by epwalsh over 1 year ago

#30 - Try Multi-Query Attention from PaLM

Issue - State: closed - Opened by epwalsh over 1 year ago
Labels: project/model, severity/should, difficulty/easy

#29 - Try SwiGLU activation

Issue - State: closed - Opened by epwalsh over 1 year ago
Labels: project/model, severity/should, difficulty/easy

#28 - Improve logging

Pull Request - State: closed - Opened by epwalsh over 1 year ago - 3 comments

#27 - Add option to omit bias terms

Pull Request - State: closed - Opened by epwalsh over 1 year ago

#26 - Upgrade `triton` and `flash_attn`

Issue - State: closed - Opened by epwalsh over 1 year ago - 2 comments
Labels: project/model, status/blocked, difficulty/medium

#24 - Add FlashAttention

Pull Request - State: closed - Opened by epwalsh over 1 year ago