kempnerinstitute/tatm issues and pull requests

#48 - 0.1.0 Release

Pull Request - State: closed - Opened by mbsabath 4 months ago - 3 comments
Labels: bump:minor

#48 - 0.1.0 Release

Pull Request - State: closed - Opened by mbsabath 4 months ago - 3 comments
Labels: bump:minor

#47 - Tokenized Data User Stores

Pull Request - State: closed - Opened by mbsabath 4 months ago - 1 comment

#47 - Tokenized Data User Stores

Pull Request - State: closed - Opened by mbsabath 4 months ago - 1 comment

#46 - installation instructions

Pull Request - State: closed - Opened by mbsabath 4 months ago

#46 - installation instructions

Pull Request - State: closed - Opened by mbsabath 4 months ago

#45 - limit test workflow to run on src/tests changes

Pull Request - State: closed - Opened by mbsabath 4 months ago - 1 comment

#45 - limit test workflow to run on src/tests changes

Pull Request - State: closed - Opened by mbsabath 4 months ago - 1 comment

#44 - Only run tests on changes to source code or tests

Issue - State: closed - Opened by mbsabath 4 months ago

#44 - Only run tests on changes to source code or tests

Issue - State: closed - Opened by mbsabath 4 months ago

#43 - use pip sphinx rtd theme

Pull Request - State: closed - Opened by mbsabath 4 months ago

#43 - use pip sphinx rtd theme

Pull Request - State: closed - Opened by mbsabath 4 months ago

#42 - Fix issue with doc deployment not running

Pull Request - State: closed - Opened by mbsabath 4 months ago

#42 - Fix issue with doc deployment not running

Pull Request - State: closed - Opened by mbsabath 4 months ago

#41 - Add documentation describing how to feed a tokenized dataset to a Pytorch dataloader for use downstream

Issue - State: closed - Opened by mbsabath 4 months ago
Labels: documentation

#41 - Add documentation describing how to feed a tokenized dataset to a Pytorch dataloader for use downstream

Issue - State: closed - Opened by mbsabath 4 months ago
Labels: documentation

#40 - Add documentation to Sphinx and Readme covering installation

Issue - State: closed - Opened by mbsabath 4 months ago
Labels: documentation

#40 - Add documentation to Sphinx and Readme covering installation

Issue - State: closed - Opened by mbsabath 4 months ago
Labels: documentation

#39 - Create a documentation page discussing how to tokenize a text dataset with `tatm`

Issue - State: closed - Opened by mbsabath 4 months ago
Labels: documentation

#39 - Create a documentation page discussing how to tokenize a text dataset with `tatm`

Issue - State: closed - Opened by mbsabath 4 months ago
Labels: documentation

#38 - Create documentation page describing how to prepare a text dataset for use with `tatm`

Issue - State: closed - Opened by mbsabath 4 months ago
Labels: documentation

#38 - Create documentation page describing how to prepare a text dataset for use with `tatm`

Issue - State: closed - Opened by mbsabath 4 months ago
Labels: documentation

#37 - Use pip installed sphinx rtd theme v3.0.0

Issue - State: closed - Opened by mbsabath 4 months ago

#37 - Use pip installed sphinx rtd theme v3.0.0

Issue - State: closed - Opened by mbsabath 4 months ago

#36 - docs build workflow

Pull Request - State: closed - Opened by mbsabath 4 months ago - 1 comment

#36 - docs build workflow

Pull Request - State: closed - Opened by mbsabath 4 months ago - 1 comment

#35 - Build Versioned Documentation on release

Issue - State: closed - Opened by mbsabath 4 months ago

#35 - Build Versioned Documentation on release

Issue - State: closed - Opened by mbsabath 4 months ago

#34 - Automatically Create Tags and Releases based on PR labels on PRs to main

Pull Request - State: closed - Opened by mbsabath 4 months ago - 4 comments

#34 - Automatically Create Tags and Releases based on PR labels on PRs to main

Pull Request - State: closed - Opened by mbsabath 4 months ago - 4 comments

#33 - Build dev documentation on github pages

Issue - State: closed - Opened by mbsabath 4 months ago

#33 - Build dev documentation on github pages

Issue - State: closed - Opened by mbsabath 4 months ago

#32 - Use tatm base path in metadata, not hardcoded path

Issue - State: open - Opened by mbsabath 4 months ago
Labels: enhancement

#32 - Use tatm base path in metadata, not hardcoded path

Issue - State: open - Opened by mbsabath 4 months ago
Labels: enhancement

#31 - use importlib versions

Pull Request - State: closed - Opened by mbsabath 4 months ago

#31 - use importlib versions

Pull Request - State: closed - Opened by mbsabath 4 months ago

#30 - `importlib.metadata.version` should be used to get package versions, not `version`

Issue - State: closed - Opened by mbsabath 4 months ago - 1 comment
Labels: bug

#30 - `importlib.metadata.version` should be used to get package versions, not `version`

Issue - State: closed - Opened by mbsabath 4 months ago - 1 comment
Labels: bug

#29 - Create versioned package on tag

Issue - State: closed - Opened by mbsabath 4 months ago
Labels: enhancement

#29 - Create versioned package on tag

Issue - State: closed - Opened by mbsabath 4 months ago
Labels: enhancement

#28 - add document mask creation

Pull Request - State: closed - Opened by mbsabath 4 months ago

#28 - add document mask creation

Pull Request - State: closed - Opened by mbsabath 4 months ago

#27 - add vocab size field to tokenized dataset

Pull Request - State: closed - Opened by mbsabath 4 months ago

#27 - add vocab size field to tokenized dataset

Pull Request - State: closed - Opened by mbsabath 4 months ago

#26 - make dataset return array, not memmap

Pull Request - State: closed - Opened by mbsabath 4 months ago

#26 - make dataset return array, not memmap

Pull Request - State: closed - Opened by mbsabath 4 months ago

#25 - Tokenized Dataset should return numpy arrays not memmaps

Issue - State: closed - Opened by mbsabath 4 months ago

#25 - Tokenized Dataset should return numpy arrays not memmaps

Issue - State: closed - Opened by mbsabath 4 months ago

#24 - Tokenized Dataset should include number of token IDs in tokenizer

Issue - State: closed - Opened by mbsabath 4 months ago

#24 - Tokenized Dataset should include number of token IDs in tokenizer

Issue - State: closed - Opened by mbsabath 4 months ago

#23 - set ray temp dir, handle ray address

Pull Request - State: closed - Opened by mbsabath 4 months ago - 1 comment

#23 - set ray temp dir, handle ray address

Pull Request - State: closed - Opened by mbsabath 4 months ago - 1 comment

#22 - Fix issue with metadata redirecting data load

Pull Request - State: closed - Opened by mbsabath 4 months ago - 1 comment

#22 - Fix issue with metadata redirecting data load

Pull Request - State: closed - Opened by mbsabath 4 months ago - 1 comment

#21 - Ray fails to start when sharing a headnode with another user in slurm

Issue - State: closed - Opened by mbsabath 4 months ago

#21 - Ray fails to start when sharing a headnode with another user in slurm

Issue - State: closed - Opened by mbsabath 4 months ago

#20 - Loading dataset from path pointing to difference directory overwrites metadata path

Issue - State: closed - Opened by mbsabath 4 months ago
Labels: bug

#20 - Loading dataset from path pointing to difference directory overwrites metadata path

Issue - State: closed - Opened by mbsabath 4 months ago
Labels: bug

#19 - Handle ray submit with no GPUs

Pull Request - State: closed - Opened by mbsabath 4 months ago

#19 - Handle ray submit with no GPUs

Pull Request - State: closed - Opened by mbsabath 4 months ago

#18 - `tatm` run fails to start ray when there are no GPUs

Issue - State: closed - Opened by mbsabath 4 months ago

#18 - `tatm` run fails to start ray when there are no GPUs

Issue - State: closed - Opened by mbsabath 4 months ago

#17 - Update author/copyright in docs

Pull Request - State: closed - Opened by mbsabath 4 months ago - 1 comment

#17 - Update author/copyright in docs

Pull Request - State: closed - Opened by mbsabath 4 months ago - 1 comment

#16 - remove personal info

Issue - State: closed - Opened by mmshad 4 months ago

#16 - remove personal info

Issue - State: closed - Opened by mmshad 4 months ago

#15 - Emit Document IDs From Token Datasets

Pull Request - State: closed - Opened by mbsabath 5 months ago

#15 - Emit Document IDs From Token Datasets

Pull Request - State: closed - Opened by mbsabath 5 months ago

#14 - Tokenized Dataset should create document mask for each example

Issue - State: closed - Opened by mbsabath 5 months ago

#14 - Tokenized Dataset should create document mask for each example

Issue - State: closed - Opened by mbsabath 5 months ago

#13 - Tokenized Dataset should emit document IDs along with tokens

Issue - State: closed - Opened by mbsabath 5 months ago

#13 - Tokenized Dataset should emit document IDs along with tokens

Issue - State: closed - Opened by mbsabath 5 months ago

#12 - add corpus option to tatm data

Pull Request - State: closed - Opened by mbsabath 5 months ago

#12 - add corpus option to tatm data

Pull Request - State: closed - Opened by mbsabath 5 months ago

#11 - Can't Specify the corpus for loading data

Issue - State: closed - Opened by mbsabath 5 months ago

#11 - Can't Specify the corpus for loading data

Issue - State: closed - Opened by mbsabath 5 months ago

#10 - update author

Pull Request - State: closed - Opened by mbsabath 5 months ago

#10 - update author

Pull Request - State: closed - Opened by mbsabath 5 months ago

#9 - Add gpu flag to slurm template to start ray cluster with available gpus

Pull Request - State: closed - Opened by timothyngo 5 months ago - 2 comments

#9 - Add gpu flag to slurm template to start ray cluster with available gpus

Pull Request - State: closed - Opened by timothyngo 5 months ago - 2 comments

#8 - Remove personal info

Issue - State: closed - Opened by mmshad 5 months ago

#8 - Remove personal info

Issue - State: closed - Opened by mmshad 5 months ago

#7 - Tokenized data metadata

Pull Request - State: closed - Opened by mbsabath 5 months ago

#6 - Create Pytorch Dataset-like Interface to tokenized data

Pull Request - State: closed - Opened by mbsabath 5 months ago

#5 - Support Wrapping `tatm` commands in slurm ray jobs

Pull Request - State: closed - Opened by mbsabath 5 months ago - 2 comments

#4 - Tokenization

Pull Request - State: closed - Opened by mbsabath 6 months ago

#3 - Load Raw Dataset

Pull Request - State: closed - Opened by mbsabath 6 months ago

#2 - add metadata creation flow

Pull Request - State: closed - Opened by mbsabath 6 months ago

#1 - Add ci

Pull Request - State: closed - Opened by mbsabath 6 months ago

GitHub / kempnerinstitute/tatm issues and pull requests