Ecosyste.ms: Issues
An open API service for providing issue and pull request metadata for open source projects.
GitHub / huggingface/tokenizers issues and pull requests
#308 - remove use of parallel iterators except in batch methods
Pull Request -
State: closed - Opened by epwalsh over 4 years ago
- 3 comments
Labels: Stale
#299 - tokenizer.train with strings not files
Issue -
State: closed - Opened by geg00 over 4 years ago
- 4 comments
#299 - tokenizer.train with strings not files
Issue -
State: closed - Opened by geg00 over 4 years ago
- 4 comments
#296 - ❓ How to see the token frequency ?
Issue -
State: closed - Opened by astariul over 4 years ago
- 4 comments
#282 - Exception: stream did not contain valid UTF-8
Issue -
State: closed - Opened by phamdinhkhanh over 4 years ago
- 5 comments
#259 - Why doesn't this library share the same tokenizer api as the transformers library?
Issue -
State: closed - Opened by sabetAI over 4 years ago
- 10 comments
Labels: Stale
#255 - Upgrade bin-package/index.node to node v12.16.3
Issue -
State: closed - Opened by micah5 over 4 years ago
- 2 comments
#251 - XLM tokenizer?
Issue -
State: closed - Opened by mkumar10 over 4 years ago
- 14 comments
#242 - Any plan to binding to java?
Issue -
State: closed - Opened by ericxsun over 4 years ago
- 4 comments
Labels: Stale
#232 - Whitespace tokenizer for training BERT from scratch
Issue -
State: closed - Opened by aqibsaeed over 4 years ago
- 5 comments
Labels: Stale
#232 - Whitespace tokenizer for training BERT from scratch
Issue -
State: closed - Opened by aqibsaeed over 4 years ago
- 5 comments
Labels: Stale
#232 - Whitespace tokenizer for training BERT from scratch
Issue -
State: closed - Opened by aqibsaeed over 4 years ago
- 5 comments
Labels: Stale
#230 - Convert saved pretrained tokenizers from transformers to tokenizers
Issue -
State: closed - Opened by NonaryR over 4 years ago
- 7 comments
Labels: Stale
#230 - Convert saved pretrained tokenizers from transformers to tokenizers
Issue -
State: closed - Opened by NonaryR over 4 years ago
- 7 comments
Labels: Stale
#194 - How to use tokenizers library for my own dataset with a mix of existing and new vocabulary
Issue -
State: closed - Opened by nikhilno1 over 4 years ago
- 1 comment
Labels: Stale
#194 - How to use tokenizers library for my own dataset with a mix of existing and new vocabulary
Issue -
State: open - Opened by nikhilno1 over 4 years ago
- 1 comment
Labels: Stale
#185 - C/C++ binding interface
Issue -
State: closed - Opened by roman-kruglov over 4 years ago
- 24 comments
Labels: Stale
#157 - How can we output the progress bars within a jupyter notebook?
Issue -
State: closed - Opened by ohmeow over 4 years ago
- 3 comments
Labels: bug, Stale
#123 - Encoding.pad/truncate return () but could return the Encoding to chain calls
Issue -
State: closed - Opened by mandubian over 4 years ago
- 3 comments
Labels: Stale
#123 - Encoding.pad/truncate return () but could return the Encoding to chain calls
Issue -
State: open - Opened by mandubian over 4 years ago
- 3 comments
Labels: Stale
#123 - Encoding.pad/truncate return () but could return the Encoding to chain calls
Issue -
State: open - Opened by mandubian over 4 years ago
- 3 comments
Labels: Stale
#119 - Exposed Unknown Tokens in Tokenizers ?
Issue -
State: closed - Opened by mandubian over 4 years ago
- 5 comments
Labels: enhancement, Stale
#119 - Exposed Unknown Tokens in Tokenizers ?
Issue -
State: closed - Opened by mandubian over 4 years ago
- 5 comments
Labels: enhancement, Stale
#100 - Feature Request: Customizable Word Tokenizers - Spacy
Issue -
State: closed - Opened by sai-prasanna over 4 years ago
- 3 comments
Labels: Stale
#69 - Compatibility with torchtext
Issue -
State: closed - Opened by joeddav over 4 years ago
- 2 comments
Labels: Stale
#67 - Why Rust?
Issue -
State: closed - Opened by andyalmonte over 4 years ago
- 2 comments
Labels: Stale
#63 - JS / WebAssembly binding planned ?
Issue -
State: closed - Opened by mikbry over 4 years ago
- 27 comments
Labels: Stale
#59 - Automatically loading vocab files
Issue -
State: closed - Opened by phosseini over 4 years ago
- 7 comments
Labels: Stale
#51 - Support for multiple language!
Issue -
State: closed - Opened by adhaamehab over 4 years ago
- 1 comment
Labels: Stale
#51 - Support for multiple language!
Issue -
State: open - Opened by adhaamehab over 4 years ago
- 1 comment
Labels: Stale
#51 - Support for multiple language!
Issue -
State: open - Opened by adhaamehab over 4 years ago
- 1 comment
Labels: Stale
#51 - Support for multiple language!
Issue -
State: open - Opened by adhaamehab over 4 years ago
- 1 comment
Labels: Stale
#51 - Support for multiple language!
Issue -
State: closed - Opened by adhaamehab over 4 years ago
- 1 comment
Labels: Stale
#10 - Rust documentation
Issue -
State: open - Opened by n1t0 almost 5 years ago
- 6 comments
#10 - Rust documentation
Issue -
State: closed - Opened by n1t0 almost 5 years ago
- 7 comments