google/seqio issues and pull requests

#661 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] 11 months ago - 1 comment

#660 - int...

Pull Request - State: closed - Opened by copybara-service[bot] 11 months ago

#659 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] 11 months ago - 1 comment

#658 - Better error message when SeqIO metric computation fails

Pull Request - State: closed - Opened by copybara-service[bot] 11 months ago - 1 comment

#657 - internal change

Pull Request - State: closed - Opened by copybara-service[bot] 11 months ago

#656 - internal change

Pull Request - State: closed - Opened by copybara-service[bot] 11 months ago - 1 comment

#655 - Make type annotation more precise.

Pull Request - State: open - Opened by copybara-service[bot] 12 months ago

#654 - Update dataset_providers.py

Pull Request - State: open - Opened by anukriti0009 12 months ago - 2 comments

#653 - Refactor metrics to use 'metric_objs' instead of 'metric_fns'.

Pull Request - State: closed - Opened by copybara-service[bot] 12 months ago - 1 comment

#652 - Fix subclass type mismatch

Pull Request - State: closed - Opened by copybara-service[bot] 12 months ago - 1 comment

#651 - ping tfds to a specific version

Pull Request - State: closed - Opened by copybara-service[bot] 12 months ago

#650 - X

Pull Request - State: open - Opened by copybara-service[bot] 12 months ago - 1 comment

#649 - X

Pull Request - State: open - Opened by copybara-service[bot] 12 months ago - 1 comment

#648 - Internal changes.

Pull Request - State: closed - Opened by copybara-service[bot] almost 1 year ago - 1 comment

#647 - Internal change.

Pull Request - State: open - Opened by copybara-service[bot] about 1 year ago

#646 - Refactor `TfdsDataSource`.

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago

#645 - Fix overflow bug in ByteVocabulary._encode_tf

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago - 1 comment

#644 - #SeqIO unnest the element specs to allow for validation of nested inputs.

Pull Request - State: open - Opened by copybara-service[bot] about 1 year ago - 1 comment

#643 - Add HuggingFace GPT2 BPE Slow Tokenizer into Seqio Vocabulary

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago - 1 comment

#642 - --

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago - 1 comment

#641 - e2e test for saxml seqio huggingface tokenizer

Pull Request - State: open - Opened by copybara-service[bot] about 1 year ago - 1 comment

#640 - Refactor metrics to use 'metric_objs' instead of 'metric_fns'.

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago - 1 comment

#639 - Don't decode predictions in PassthroughLegacyMetric if not needed.

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago - 1 comment

#638 - Optionally config atol and rtol for assert_dataset.

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago - 1 comment

#637 - Optionally config atol and rtol for assert_dataset.

Pull Request - State: open - Opened by copybara-service[bot] about 1 year ago - 1 comment

#636 - Refactor paxml metrics to use 'metric_objs' instead of 'metric_fns'.

Pull Request - State: open - Opened by copybara-service[bot] about 1 year ago - 1 comment

#635 - Allow negative values for unbounded epochs in GrainTask.

Pull Request - State: open - Opened by copybara-service[bot] about 1 year ago

#634 - Add support for resolving wildcards in versions of TFDS datasets

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago

#633 - Add a setup function to beam_utils.PreprocessTask.

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago - 1 comment

#632 - Call `preprocess_postcache` in beam_utils.PreprocessTask

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago - 1 comment

#631 - fix ValueError

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago - 1 comment

#630 - [Seqio] Speed up shard globbing when the pattern is already a list of non-glob entries.

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago

#629 - Bump version number

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago

#628 - Bump version number

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago

#628 - Bump version number

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago

#627 - Update docs to make it more clear how use_cached works for Mixtures.

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago

#627 - Update docs to make it more clear how use_cached works for Mixtures.

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago

#626 - Update behavior of rename_feature preprocessor:

Pull Request - State: open - Opened by copybara-service[bot] about 1 year ago - 1 comment

#625 - Add unused parameter to fix code breakage.

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago - 1 comment

#623 - Skip sequence_length information logging for SeqIO deterministic tasks when the information is missing.

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago

#623 - Skip sequence_length information logging for SeqIO deterministic tasks when the information is missing.

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago

#620 - Make seqio caching work when some dataset elements are ragged tensors.

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago

#620 - Make seqio caching work when some dataset elements are ragged tensors.

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago

#612 - Allow custom Seqio vocabulary as tokenizer

Pull Request - State: open - Opened by copybara-service[bot] about 1 year ago - 1 comment

#611 - Add HuggingFace GPT2 BPE Slow Tokenizer into Seqio Vocabulary

Pull Request - State: closed - Opened by copybara-service[bot] about 1 year ago - 1 comment

#564 - Add `try_in_mem_cache` config in SeqIOInput.

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago

#563 - Enables the caching of postprocessed targets at Evaluator level

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago

#562 - ValueError: mutable default <class 'seqio.vocabularies.PassThroughVocabulary'> for field vocabulary is not allowed: use default_factory

Issue - State: open - Opened by jli262 over 1 year ago - 1 comment

#561 - loosen protobuf version requirement to unblock dependent packages.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#560 - Fix sentencepiece protobuf version issue.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#559 - Minor refactor in SeqIO Mixture.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#558 - Adds warnings when attempting to compute subtasks of a DeterministicMixture.

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago

#557 - Fixes off-by-one in printing of shard index.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#556 - unimax sampling ?

Issue - State: open - Opened by StephennFernandes over 1 year ago

#555 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#554 - Avoid deadlock in decode_tf.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#553 - seqio.get_mixture_or_task('bool_q_template_0_no_opt_five_shot') failed

Issue - State: open - Opened by liuzhiyong01 over 1 year ago

#552 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#550 - Make block_length configurable for tf.data.Dataset.interleave in seqio data_provider.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#549 - Make cycle_length configurable for tf.data.Dataset.interleave.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#548 - Make cycle_length configurable for tf.data.Dataset.interleave.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#547 - Dataset performance

Issue - State: open - Opened by KeremTurgutlu over 1 year ago

#545 - TfdsDataProvider gives error with non-None tfds_data_dir

Issue - State: closed - Opened by lucaslingle over 1 year ago - 2 comments

#544 - Raise an exception when no shards are found during task caching. This prevents the later error of malformed info json files during mixture caching.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#543 - internal

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago - 1 comment

#542 - Refactor Mixture.get_dataset() to take task.get_dataset() call out in a separate method.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#541 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#540 - fix sentencepiece proto issue

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#539 - Add preprocessor hash_and_tile_subtask_id for packing task IDs.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#538 - fixed typo in loggers_test

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#537 - Specify which of inferences, targets, and/or dataset is unset.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#536 - seqio data_providers filepatterns to ensure that matching files are unique, deduplicating the files(shards) that are matched by more than one pattern.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#535 - How to choose minimum sequence length while avoiding truncation

Issue - State: open - Opened by marcospiau over 1 year ago

#534 - Internal change

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#533 - Makes SeqIO SentencePieceVocabulary thread-safe.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#531 - Increase parallelism for seqio tokenization to mitigate head of line blocking.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#530 - Adds MetricManager to manage metrics for the upcoming ShardedEvaluator

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#529 - Add versioned tasks and mixtures to SeqIO

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago

#528 - internal

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#527 - Task registration tracking: usability improvements

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#526 - Read tasks concurrently in Mixture.get_dataset

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago

#525 - Add SourceInfo dataclass to explicitly record where a Task or Mixture is defined.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#524 - internal libraries

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#523 - Add caching to `list_shards` method to avoid redundant disk access

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#522 - temp, won't be submited untill diff

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago - 1 comment

#521 - Add generics to dataset provider registries

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#520 - explicitly specify the parameter names when call init

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#519 - Add generics to dataset provider registries

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#518 - Don't run `prediction` over the entire dataset if we are already doing `prediction_with_aux`

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#517 - Make in-memory caching in task.get_dataset optional.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#516 - Expose tokenizer bos_id on the seqio Vocabulary.

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago - 1 comment

#515 - Expose tokenizer bos_id on the seqio Vocabulary.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago - 1 comment

#514 - caching tasks goes out of memory due to apache beam

Issue - State: closed - Opened by mayurnewase over 1 year ago - 2 comments

#513 - Concatenating Tasks?

Issue - State: closed - Opened by gahdritz over 1 year ago - 2 comments

#512 - # - z_code: string - contains asr hypothesis.

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago - 1 comment

#511 - Updates Metric class

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#510 - Replace usage of deprecated `abc.abstractproperty`.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

#509 - internal libraries

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago - 1 comment

#508 - internal

Pull Request - State: open - Opened by copybara-service[bot] over 1 year ago

#507 - prefetch data pipeline at the end of preprocessing.

Pull Request - State: closed - Opened by copybara-service[bot] over 1 year ago

GitHub / google/seqio issues and pull requests