soaxelbrooke/python-bpe issues and pull requests

#21 - [clarifications] This is not following the BPE paper you've mentioned in encoder.py

Issue - State: closed - Opened by edoson over 2 years ago - 1 comment

#20 - It amazes me that this implementation of BPE doesn't require LOOP. How to make it correct?

Issue - State: open - Opened by lsy641 over 2 years ago

#19 - Update encoder.py

Pull Request - State: closed - Opened by tuanio over 2 years ago - 1 comment

#18 - Update encoder.py

Pull Request - State: closed - Opened by tuanio over 2 years ago - 1 comment

#17 - Remove broken link from comment

Pull Request - State: closed - Opened by zouharvi almost 3 years ago

#16 - How to install by python2.7

Issue - State: open - Opened by FengXuas almost 4 years ago

#15 - How to generate a GPT-2 (openAI) encoder.json ?

Issue - State: open - Opened by Skylixia over 4 years ago - 1 comment

#14 - Naive vocab checking sometimes uses word index instead of byte pair index

Issue - State: closed - Opened by soaxelbrooke over 5 years ago
Labels: bug

#13 - Allow lowercasing

Issue - State: closed - Opened by soaxelbrooke over 5 years ago

#12 - Prefilter learn_bpe_vocab to prevent duplicates

Pull Request - State: closed - Opened by thien over 5 years ago - 4 comments

#11 - use tok as fallback tokenizer

Pull Request - State: open - Opened by kootenpv over 5 years ago - 3 comments

#10 - Consider using `tok` as tokenizer; faster and more customizable

Issue - State: open - Opened by kootenpv over 5 years ago - 1 comment

#9 - make lowercasing optional

Pull Request - State: closed - Opened by soumyagk over 5 years ago - 5 comments
Labels: enhancement

#8 - Is it possible to finetune BPE embeddings along with the rest of my network?

Issue - State: closed - Opened by JoaoLages almost 6 years ago - 4 comments

#7 - Is it possible to apply BPE without SOW/EOW tokens?

Issue - State: closed - Opened by universome about 6 years ago - 2 comments

#6 - Add python 3.7

Pull Request - State: closed - Opened by soaxelbrooke about 6 years ago

#5 - Add LICENSE