google/seqio issues and pull requests

#499 - Internal change

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago

#498 - Move prefetcing to after preprocessing in SeqIO caching.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#497 - internal change

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#496 - internal change

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago - 1 comment

#495 - Tracking module-level seqio TaskRegistry use back to where the tasks were registered

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#494 - Disable counting characters by default during SeqIO caching.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#493 - disable tests that fail because of bug in TF

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#492 - Handle function names of partial functions better

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#491 - Silence some pytype errors.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#490 - BEHAVIOR CHANGE: By default, SeqIO mixtures should complete an epoch when any of the subtasks complete. Note, does not affect the default cause where a mixture is repeated indefinitely.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#489 - Add a sequence_length argument to test_postprocessing.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#488 - A feature converter to convert a prefix LM corpus to one suitable for RL finetuning.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#487 - More cached properties to speed up task initialization

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#486 - More-threadsafe construction of metric functions.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#485 - Allow Task source to be any DatasetProvider.

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago

#484 - Fixes `Task.replace()` by passing in metric_fns and postprocess_fn to the initializer.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#483 - internal

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago - 1 comment

#482 - Speed up task registry by using lazy properties

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#481 - internal

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago - 1 comment

#480 - internal

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#479 - Patch Mixture.num_input_examples to only use split if it exists

Pull Request - State: closed - Opened by TheExGenesis over 1 year ago - 1 comment

#478 - Make `mixture_or_task_with_new_vocab` able to take an actual `Task` or `Mixture` instance, rather than just a string name.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#477 - Make `mixture_or_task_with_truncated_data` able to take an actual `Task` or `Mixture` instance, rather than just a string name.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#476 - Bump version number

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#475 - Update `mixing_rate_num_examples` doc string.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#474 - internal.

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago

#473 - internal

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago

#472 - internal.

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago

#471 - Update the mixing_rate_num_examples function to take in a split to use for calculating the mixing rate. This enables easier mixing for eval-only mixtures.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#470 - Bump version number

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#469 - Allow mixture_or_task_with_new_vocab users to override the validation step.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#468 - `mixture_or_task_with_new_vocab` should respect `add_to_seqio_registry` when creating subtasks.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#464 - Internal

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago - 1 comment

#462 - Experimental tool to disable SeqIO registries.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#455 - The current implementation of tf.io.gfile.glob may return sorted results on some filesystems.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#451 - Seqio task works with multiple decodes.\n

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#431 - Add DatasetProvider interface. The clear advantage of this over simply renaming DatasetProviderBase to DatasetProvider is that current uses of DatasetProviderBase do not break (implementing DatasetProviders still extend DPBase).

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago

#420 - Refactor how tasks and mixtures are cached for easier addition of other strategies

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago

#409 - rollback fix minor ptype mismatch

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#406 - How to apply the huggingface tokenizer in seqio.vocabulary

Issue - State: closed - Opened by nawnoes over 1 year ago

#394 - Adds LegacyMetricFactory which creates LegacyMetric from metric_fn and pp_fn

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago

#310 - unable to train mt5 from t5x using mixtures ValueError: Dataset is missing an expected feature during input_validation validation: 'inputs'

Issue - State: closed - Opened by StephennFernandes almost 2 years ago - 3 comments

#306 - Introduces seqio.CollectingMetric class

Pull Request - State: closed - Opened by copybara-service[bot] almost 2 years ago

#283 - This CL unifies `split` argument behavior between TfdsDataSource and FunctionDataSource.

Pull Request - State: closed - Opened by copybara-service[bot] almost 2 years ago - 1 comment

#261 - Please include installation instructions

Issue - State: open - Opened by leiterenato about 2 years ago - 2 comments

#243 - Support for "Deterministic Pipelines"

Issue - State: closed - Opened by ruomingp about 2 years ago - 4 comments

#228 - Feature lenghts are sometimes used to identify the data/task (i.e. use a different feature converter depending on which lenghts are expected), for example DecoderFeatureConverter. This works great in training but during evaluation the feature lengths used are the ones present in the dataset not the ones set in the config. This CL makes the task_feature_lenghts to be passed instead of the inferred feature_lengths during evaluation.

Pull Request - State: closed - Opened by copybara-service[bot] about 2 years ago

GitHub / google/seqio issues and pull requests