google/seqio issues and pull requests

#755 - Add option to TfdsDataSource to specify only the data dir pointing to a single dataset

Pull Request - State: open - Opened by copybara-service[bot] 7 days ago

#754 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] 21 days ago

#753 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] 22 days ago

#752 - internal change

Pull Request - State: closed - Opened by copybara-service[bot] 24 days ago

#751 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] 24 days ago

#750 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] 25 days ago

#749 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] 25 days ago

#748 - Expose size attribute of PassThroughVocabulary

Pull Request - State: closed - Opened by copybara-service[bot] about 1 month ago

#747 - depend on sentencepiece version that uses newer protobuf

Pull Request - State: closed - Opened by copybara-service[bot] about 1 month ago

#746 - remove dependency on old protobuf

Pull Request - State: closed - Opened by copybara-service[bot] about 1 month ago

#745 - Supports `builder_kwargs` in `TfdsDataSource`

Pull Request - State: closed - Opened by copybara-service[bot] about 1 month ago

#744 - Replace deprecated `jax.tree_` functions with `jax.tree.`

Pull Request - State: closed - Opened by copybara-service[bot] about 2 months ago

#743 - internal only

Pull Request - State: open - Opened by copybara-service[bot] about 2 months ago - 1 comment

#742 - Allow data sources to specify that they can be shuffled without a buffer.

Pull Request - State: closed - Opened by copybara-service[bot] about 2 months ago - 1 comment

#741 - Truncate seed if too large during seqio caching jobs.

Pull Request - State: closed - Opened by copybara-service[bot] about 2 months ago

#740 - minor changes.

Pull Request - State: closed - Opened by copybara-service[bot] about 2 months ago

#739 - Set return type of tasks property to list[Task] instead of Sequence[Task]

Pull Request - State: closed - Opened by copybara-service[bot] about 2 months ago - 1 comment

#738 - Set return type of tasks property to list[Task] instead of Sequence[Task]

Pull Request - State: open - Opened by copybara-service[bot] about 2 months ago - 1 comment

#737 - Allow for no splits to be defined in the tasks and raise a warning.

Pull Request - State: open - Opened by copybara-service[bot] about 2 months ago - 1 comment

#736 - Make load_model cache across threads.

Pull Request - State: closed - Opened by copybara-service[bot] 2 months ago - 1 comment

#735 - Expose `tf_data_options` from `GrainTask/Mixture`

Pull Request - State: closed - Opened by copybara-service[bot] 2 months ago

#734 - What is a number?

Issue - State: closed - Opened by justzh 3 months ago

#733 - `scores` is a sequence of floats.

Pull Request - State: closed - Opened by copybara-service[bot] 3 months ago - 1 comment

#732 - my public commit msg

Pull Request - State: closed - Opened by copybara-service[bot] 4 months ago - 1 comment

#731 - Allow '#' in the task name. This will be used as a separator for denoting different versions of the same task.

Pull Request - State: closed - Opened by copybara-service[bot] 4 months ago - 1 comment

#730 - Improve exception error messages for mismatched features in mixtures.

Pull Request - State: closed - Opened by copybara-service[bot] 4 months ago

#729 - prevent timeout for cached dir

Pull Request - State: closed - Opened by copybara-service[bot] 4 months ago - 1 comment

#728 - Prevent duplicate cache dirs in global cache dirs

Pull Request - State: closed - Opened by copybara-service[bot] 4 months ago

#727 - Add support for write caching results to ArrayRecord.

Pull Request - State: closed - Opened by copybara-service[bot] 4 months ago - 1 comment

#726 - Replace `tensorflow.compat.v2` import with `tensorflow` in vocabularies.

Pull Request - State: closed - Opened by copybara-service[bot] 4 months ago

#725 - Prevent multiple copies of a directory being added to global_cache_dirs.

Pull Request - State: closed - Opened by copybara-service[bot] 5 months ago - 1 comment

#724 - Minor fix: metrics computed on scores have signature `scores`.

Pull Request - State: closed - Opened by copybara-service[bot] 5 months ago - 1 comment

#723 - Better error message when MIXTURE_OR_TASK_NAME is None by accident

Pull Request - State: closed - Opened by copybara-service[bot] 5 months ago - 1 comment

#722 - Adjust `read_file_fn` pytype.

Pull Request - State: open - Opened by copybara-service[bot] 5 months ago

#721 - Adjust `read_file_fn` pytype.

Pull Request - State: closed - Opened by copybara-service[bot] 5 months ago

#720 - Adjust `read_file_fn` pytype.

Pull Request - State: closed - Opened by copybara-service[bot] 5 months ago - 1 comment

#719 - Adjust `read_file_fn` pytype.

Pull Request - State: closed - Opened by copybara-service[bot] 6 months ago

#718 - Copy source info in mixture_or_task_with_new_vocab helper function

Pull Request - State: closed - Opened by copybara-service[bot] 6 months ago

#717 - Fix behavior when `Iterable` is actually given as `split_to_filepattern`.

Pull Request - State: closed - Opened by copybara-service[bot] 6 months ago

#716 - Use `rate_per_task_name` instead of protected member

Pull Request - State: closed - Opened by copybara-service[bot] 6 months ago

#715 - Revert part of some breaking changes in using `rate_per_task_name`

Pull Request - State: closed - Opened by copybara-service[bot] 6 months ago

#714 - Require Python 3.9 and run tests with Python 3.9

Pull Request - State: closed - Opened by copybara-service[bot] 6 months ago

#713 - Update Github Actions to use more recent versions of checkout and setup-python

Pull Request - State: closed - Opened by copybara-service[bot] 6 months ago

#712 - Add method to get mixture rate per task name

Pull Request - State: closed - Opened by copybara-service[bot] 6 months ago

#711 - Remove unused hash_and_tile_subtask_id.

Pull Request - State: closed - Opened by copybara-service[bot] 6 months ago

#710 - Helpers to add metrics to SeqIO task or mixture.

Pull Request - State: open - Opened by copybara-service[bot] 6 months ago - 1 comment

#709 - Update `add_to_seqio_registry` docstring.

Pull Request - State: closed - Opened by copybara-service[bot] 6 months ago - 1 comment

#708 - internal change

Pull Request - State: closed - Opened by copybara-service[bot] 6 months ago - 1 comment

#707 - Add benchmarks that run on TPUs.

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago

#706 - Resolve lint issues in SeqIO.

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago

#705 - Internal change.

Pull Request - State: open - Opened by copybara-service[bot] 7 months ago

#704 - Add benchmarks for preprocessors.

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago

#703 - Internal change.

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago

#702 - Internal change.

Pull Request - State: open - Opened by copybara-service[bot] 7 months ago

#701 - Stop using deprecated logging.warn and use logging.warning

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago

#700 - Internal Changes

Pull Request - State: open - Opened by copybara-service[bot] 7 months ago - 1 comment

#699 - Internal Changes

Pull Request - State: open - Opened by copybara-service[bot] 7 months ago - 1 comment

#698 - Internal change.

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago

#697 - Internal change.

Pull Request - State: open - Opened by copybara-service[bot] 7 months ago

#696 - Internal change.

Pull Request - State: open - Opened by copybara-service[bot] 7 months ago

#695 - Internal change.

Pull Request - State: open - Opened by copybara-service[bot] 7 months ago

#694 - Internal change.

Pull Request - State: open - Opened by copybara-service[bot] 7 months ago

#693 - Internal change.

Pull Request - State: open - Opened by copybara-service[bot] 7 months ago

#692 - Resolve lint issues in dataset_providers_test.

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago

#691 - Internal change.

Pull Request - State: open - Opened by copybara-service[bot] 7 months ago

#690 - Internal change.

Pull Request - State: open - Opened by copybara-service[bot] 7 months ago

#689 - Internal change.

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago

#688 - Fix lint issues.

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago

#687 - Speed up seqio.Vocabulary.decode_tf for empty tensors.

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago - 1 comment

#686 - Cache the `tf_tokenizer` in `seqio.SentencePieceVocabulary`.

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago - 1 comment

#685 - Add method to get the TFDS builder class

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago

#684 - Allow provider redefinition when reloading modules.

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago - 1 comment

#683 - Add fast sentencepiece tokenziation to T5.

Pull Request - State: open - Opened by copybara-service[bot] 7 months ago - 1 comment

#682 - Add support for FastSentencepieceTokenizer.

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago - 1 comment

#681 - task registration for fewshot binding

Pull Request - State: open - Opened by copybara-service[bot] 7 months ago - 1 comment

#680 - Setup Fewshot Binding Task and Mixture

Pull Request - State: open - Opened by copybara-service[bot] 7 months ago - 1 comment

#679 - Remove argument override in `add_kwargs_to_transform()`.

Pull Request - State: closed - Opened by copybara-service[bot] 7 months ago - 1 comment

#678 - internal changes

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago - 1 comment

#677 - Internal change.

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago

#676 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago - 1 comment

#675 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago

#674 - Include the case of new key not being in the old mapping.

Pull Request - State: open - Opened by copybara-service[bot] 8 months ago - 1 comment

#673 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago

#672 - internal changes

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago - 1 comment

#671 - allow passing source info to PyGloveTunableMixture

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago

#670 - Remove argument override in `add_kwargs_to_transform()`.

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago - 1 comment

#669 - Add PreprocessTask.preprocess_postcache option to perform preprocessing for tasks that don't support caching.

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago - 1 comment

#668 - Add _get_dataset() helper function.

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago - 1 comment

#667 - Internal features.

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago - 1 comment

#666 - Since we know this class will pass the argument to be the `tasks` argument of `Mixture.init()`, it's OK to use

Pull Request - State: open - Opened by copybara-service[bot] 8 months ago - 1 comment

#665 - Performance fix when passing a Tensor into seqio.SentencePieceVocabulary decode

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago

#664 - Update seqio.preprocessors.print_dataset for TF2

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago - 1 comment

#663 - Performance fix when passing a Tensor into seqio.SentencePieceVocabulary decode

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago

#662 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago - 1 comment

#661 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago - 1 comment

#660 - int...

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago

#659 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago - 1 comment

#658 - Better error message when SeqIO metric computation fails

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago - 1 comment

#657 - internal change

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago

#656 - internal change

Pull Request - State: closed - Opened by copybara-service[bot] 8 months ago - 1 comment

GitHub / google/seqio issues and pull requests